Actually, in fairness, I will amend this statement. While stainless steel is very hard and tough compared to aluminium, the watch body can sometimes exhibit 'buffs' on its super shiny surface if it's knocked about a lot. But in the same circumstances the aluminium would be scratched and chipped, so I guess it's personal preference to an extent. Either way I feel that, on balance, the stainless body is more hardwearing.
Of course none of the above necessarily means that anyone should choose one over the other. It's still personal preference, and definitely a matter of perspective as to whether or not the SS is worth the price premium.