Wanting more RAM in these things isn't just "spec mongering." More RAM may not be necessary NOW, but people want it for forward compatibility.
It will likely make a huge difference when (hypothetically) iOS 7 come out alongside the iPhone 6S, which has 1GB of RAM, and all of a sudden a whole lot of apps need it.
The only reason the first gen iPad can't run iMovie, remember, is because of RAM. (Well, technically, it's because Apple doesn't allow it, since it CAN run iMovie. But the reason Apple doesn't allow it is that it doesn't run well enough, because of the limited RAM.)