I'm in this boat with you MRU. I don't think Samsung's most recent devices are ugly, per se, but rather feel less substantial and/or premium. I think much of it comes from using a design that allows for a removable battery and the required removable back plate. I'm personally a big fan of metal/glass, practical or not, and seem to generally prefer devices using these materials (iPhones/iPads, Nexus 4, HTC One, LG G Pad, and now the Xperia Z line are my recent personal favorites in design and materials). But there are certainly quite a few devices made from plastic that I found quite pleasant--Nexus 5/7, quite a few Nokia Lumia's--all of which have a better feel to me than Galaxy devices--more heft, more substantial and solid, so blaming the materials used is a cop out, IMO.
I agree with Technarchy in that Samsung changing for the sake of change would be foolish. Moving a new dramatic direction could very well alienate a large portion of their customers. I'd personally would just like to see a different build design using materials that feel (and look) better--go ahead keep the general Samsung design. Absolutely doesn't mean my design tastes are better, just what I'd need to see before considering using a future Samsung device.
I don't think a removable plate has anything to do with, I have seen several phones in the past that had removable plates be metal. Off the top of my head there was the original flip phone RAZR, krazr, slzr, RAZR 2, HTC inspire, Nokia e71 and another flip phone Nokia. All had metal back plates that were removable.
And really, the plastic isn't the problem, its the design of the plastic. It would've been cool had they used the same texture as the Nexus 5, if they did that and cleaned up touchwiz I might have got one because of the water proof ness.