That's how I understand it, as well. But just because Apple sells a watch in the thousands of dollars doesn't mean that you are getting any less of an experience if you opt for a $1000 stainless steel watch or $350 Sport. Regular watches sell in a wide range of prices.
And from what we know, the internals are the same whether you're buying the aluminum watch or the gold one. Basically you're paying more based on materials not specs/functionality.