My understanding was that the clause Apple had did not prevent a publisher from selling a book on Amazon or any other store for a lower price. It merely said that if it did, Apple could lower the price on their store as well.
Your understanding is correct, the problems are in the consequences which sadly are
not lower prices for everyone.
Before Apple the game was played with the wholesale model. In wholesale the publisher doesn't decide the final price (the price the consumer pays), it instead sells to the retailer at a given price, and the retailer sells to the consumer at whatever price it sees fit. This means that in the wholesale model the publishers get the same money per-sale no matter what the consumer pays. With the agency model it's not so. With the agency model the publishers sets the price the consumer pays, and gets 70% of that, the retailer gets the remaining 30% (assuming Apple's fees).
The key issue is that with wholesale the price the consumer pays
is irrelevant to the retailer's per-sale revenue (which is fixed) but in the agency model the price the consumer pays
does affect to the retailer's per-sale revenue. Mix that with the MFN clause (which allows to match the lowest price) and you get the following scenario:
Let's assume Amazon sells through wholesale and Apple sells through agency with MFN clause. The publishers sell the book through Apple at a consumer price of 15$, so for every sale they get 10.50$ (the 70%). Through Amazon they set a wholesale price of 10.50$, so that's what they get and Amazon is free to sell the ebook to the consumers at the price it wants. So from the publisher's revenue point of view selling though Amazon or Apple is the same, they always get 10.50$ every sale.
Amazon decides to set the book on sale to the consumers at 12$ because they are happy with a lower margin than 30%. Every sale the publishers gets 10.50$ (wich is fixed) and Amazon 1.50$. Since the publishers always get their 10.50$ they are happy (at least about the revenue per-sale). Apple matches the price according to the MFN clause, so now they too sell the book at 12$, but in this case the publishers get only 8.40$ per-sale from Apple instead of 10.50$. And this makes them definately
not happy.
Let's assume now that Amazon wants to use the book as loss-leader. It sets the price at 5$, so for every sale the publishers get 10.50$ and Amazon
loses 5.50$. Apple matches the price and through Apple the publishers now get 3.50$ per-sale. If the publishers were not happy before, now they are contemplating suicide...
So the publishers cannot afford the agency model with MFN clause without fixing consumer prices overall, or their revenues would be at the mercy of the lowest bidder. This is to explain the consequences of the MFN clause, the collusion issues and the involvement of Apple in it are another story.
TL;DR: With the agency model the final price affects the publisher's revenues directly. With MFN the wholesale model had to go and prices had to be fixed by publishers, or the whole system would not work (at least from the publishers' point of view).