Not yearly, chiply. The thinking is the top of the line machine should always have the top of the line CPU. Instead Apple put the M4 into a tablet best used to watch videos, read e-books, and check on email. I admit to being puzzled by this decision as well.
Apple has largely explained this. The disconnect here is that most of "iPad gets the M4 is horrible" first folks tend to have a hyper single threaded drag racing performance bias. The M2 Ultra although a bigger , more powerful chip can't drive the new screens that the iPad Pro got. The M3 couldn't do it either (which in part is why it got 'skipped' for the iPad Pro). The updates to the display controllers were just as much as critical factor in the iPad Pro . CPU cores were not the major driving factor.
The M4 isn't going to beat the M2 Max on any multithreaded , RAM capacity , or a very wide variety of graphics tasks. The M2 Ultra has 'slower' NPU but also twice as many, so again any 'wide' , 'embarrassingly parallel' NPU task it is still likely to win a wide variety of situations.
So single through approximiately six CPU core 'drag racing' a Mini with M4/M4 Pro would work and be way more cost effective for vast block of users.
Roughly similar issue why some folks will hold onto their MP2019 with mulitple GPUs and stick the single GPU Mn SoC solution for probably a few more years. Similarly the hyper modular GPU user base won't really care if there M4 Ultra GPU is faster as it is still won't be modular. The RTX 5080 and RX 8800 are going to be far more 'shiny' than anything Apple is going to roll out M4 or M5 generation.
There is a decent chance the Mac Pro will be on a n+2 iteration schedule ( M2 'big' -> M4 'big' -> M6 'big' or M5 'big' -> M7 'big' etc. ) Apple churning 400-800 mm^2 dies every year and then completely dropping the old one does make any economic sense. Nobody does that ( e.g., look at big die GPUs. , big die server chips ,etc. ). There is no 'hand me down' product for the SoC at the top end of the Mac spectrum. Leading edge iPhone SoC spend 3 years in the same product and can branch out into low end iPad's , AppleTV , SE , etc. The 'plain' Mn product can trickle down into the iPad Air ( e.g., won't be surprising if iPad Air gets M4 when iPad Pro moves to either M5/M6 depending upon timing. )
Typically there is a "But Apple killed off the M2 Max in the MBP 14/16 inch quickly" comment ... and likely the 'blowback' on that is a contributing factor why the MP and perhaps Studio spend extra time on the M2 Max die component ( so Apple has time to do return on investment on a smaller unit volume over a longer period of time. ) There is a meme that Apple could just 'dogfood' dump older Max/Ultras into their datacenters to create a 'hand me down' category that is a 'non-product'. After an initial larger x86-64 retirement wave , it seems doubtful they could year dump excess into that limited 'pit' yearly without it overflowing. ( dumping the datacenter SoCs yearly isn't likely going to work ecnomically long term either. ). "Oh but Apple will dump them into the AI/ML hardware deployment growth hype train"... and what happens when the hype train eventually stalls? That isn't a 'plan' , it is a 'prayer'.
The 'big screen iMac' and Mac Pro get replaced last in the the M-series transition and Apple Mac sales didn't crumble. Big systems regularly go last isn't going to crush the Mac ecosystem.
