I actually have quite a few matte stainless steels items. It's my favorite material for many things. Many of these things stand a daily treatments a lot rougher than a phone would, and for years. They are still as good looking now as the day I bought them.
The thing about matte steel is that they are scratched to start with. As you use them, additional scratches don't stand out the way they do on polished steel. Take my stainless steel pans for example. I use metal utensil on them, and and scrap the inside every time I cook with them. After 10 years they still look great.
A phone doesn't experience any where near the rough treatment my stainless steel pan does, so I think it will be fine.
Another example is my watch which has both matte and polished surface. The matte area holds up beautifully all these years, but in the polished area, there are a few distracting scratches.