I selected the label settings page and 'all mail' was all ready selected to 'show'.Maybe there is something else afoot. Thanks for the help though.
You need to go to GMail settings then the Labs tab and enable IMAP advanced controls.... then go back to labels and a new setting will be there allowing you to show/hide each label in IMAP. Hide ALL Mail label and you will no longer see in in Mail app in OS X.