Sean, you have had company making that decision to switch so go with what you feel works best for you in the long run and whether you want it more for sport/casual or everyday/dress. Also keep in mind the differences in the screen. IonX glass - will scratch without protection but not as prone to break. Screen will be brighter in sunlight. Sapphire - like the luxury watches won't scratch but if dropped is prone to cracking. Dimmer in sunlight. The SS can be polished to remove light scratches like other SS body watches. If you don't have or haven't thought about getting Apple Care+, now would be the time. Read some of the experiences people have had and do what you think is right for you. Most of all hope you enjoy using it which ever model you end up with.
If you have a store nearby I'd also recommend stopping in and trying on the SS for a little while. It is a bit heavier than the aluminium, and as such some people have preferred the sport over SS. It's something you will want to wear throughout the day into the night so comfort should be a concern.