For a non pro user can someone explain why the Mac Pro wasn’t refreshed with the M3 Ultra like the Studio? What was the benefit of not updating it versus updating it?
The benefit of not updating is should be clear.... it is less expensive. No R&D is always going to be cheaper than some effort that takes time, money , and resources.
If the upgrade user base is very small , then run the risk of doing the expense and not making the money back. Folks who just bought a > $8K Mac Pro very probably would not upgrade. The folks still on Intel MP who didn't upgrade are extremely likely content to just coast until the end of the Intel era on what they have got. (or switch to Linux/WinPC and stay on x86-64). If vast majority buyers are only upgrading on 5-7 year cycles then shorter product upgrade cycles are not going to generate more sales.
No upgrade is also a cheaper 'test' to see how much traction TBv5 could get.
Possible PCC coupling. There are some reports that the custom servers for Apple's Private Cloud Compute are M2 Ultra. Keeping the Mac Pro on M2 Ultra at least keeps a tiny income coming in for the chips, since PCC generates nothing.
The 'upside' benefit for doing an upgrade for a Mac Pro wouldn't be that much.
i. The M4 Max Studio would outclass it on single threan and lightweight-medium weight threaded apps.
( For the Mac Studio that isn't as much of a problem since that still results in a Studio sale. No M4 Max for Mac Pro would be no Mac Pro sale. Unit sales of Mac Pro would go down even more. )
ii. The M3 Ultra only is an advantage over the M4 Max in some corner cases. Very 'embarrassingly parallel' GPU work where must more cores that are a bit slower outpace a smaller number that are faster.
( a Mac Pro might actually 'dilute' a bit of the Studio Ultra sales which are very likely smaller this generation to begin with. ). The render upgrades were good , but that is probably not the folks more likely to get.
iii. If Apple did not upgrade the PCI_e backhaul of the M3 Ultra , then the PCI-e througput would have been the same. The Mac Pro would only really get additional Thunderbolt 5 ports as any I/O 'upgrade'. Tracking poeple into TBv5 is primiarly only tracking people off the Mac Pro. Apple seems to be have been looking at legacy PCI-e cards that were in the "faster than TBv3 but slower than TBV5" bandwidth range with the MP 2023.
( pretty good chance Apple threw any internal chip network bus increase allocated for I/O at TBv5 and not the backhaul.)
iv. Reputational help. If Apple got on to a predictable upgrade schedule then these gaps in Mac Pro upgrades would go over with less 'drama'. However, they are somewhat already crumbled that reputation. At this point, it is hard to sink any lower. It could be 4 years. Last iterations were 6 years , 4 years . It wasn't going to be less than 2 years. It would make the MP fan base happier to see quick cycle upgrades but would that generate substnatially any more sales. ( Probably not ).
If the M2 Mac Pro had not been so late then perhaps there was uplift for a M3 generation ( e.g., if had gone M1 -> M3 ).
The M3 Ultra arrived relatively very late. Basically after the run for M3 Max was done. Some reports were that the M3 Max used in the MBP 14/16" didn't have UltraFUsion. That would make the M3 Max-v2 a special case chip. (and if a special case chip , then pretty likey it has a lame or no update on the PCI-e backhaul . In that case, the plain M3 Max probably dumped the PCI backhaul function to save die space. ). On top of TSMC N3B to boot and very large ( so also more expensive packaging). It wouldn't be too surprising if it were a money loosing chip. Apple decided to take they lumps for the Mac Studio , but placing it in MP would be just digging a deeper hole. [ They may be an option for Apple PCC since that is a money-pit anyway. Web services for zero fees. ]
The M3 generation was a bit of a transitional chip that probably wasn't worth 'betting the farm' on. Ultra's are too big and expensive to throw in the 'trash can' every year; so likely not coming every generation. I think the original Apple plan was M1 -> M3 -> M5 , but that got scrambled by pandemic and TSMC N3 quirks. If M5 Ultra comes and no Mac Pro that would be the odd-duck situation.
P.S. Since Apple has been putting together some custom server nodes , there is a decent chance parts of the MP team that did the rack version got 'hijacked' into doing non-customer facting work. Until Apple gets there PCC situation stabilized and sorted out the MP would get starved of some resources. If Apple diverts a large chunk of M5 Ultras into PCC then MP might just get left out.