21 vs 27, diffs not documented in comparision table

Depending on which comparisson chart you looked at, it may not have mentioned that the hard drive in the 21.5" is 5400RPM and in the 27" it's 7200RPM.

Another thing to note is that the larger SSDs tend to be faster than smaller ones, but it's not something you'd notice under normal use.

 

The differences look pretty clear here: http://www.apple.com/imac/specs/

The 21-inch is configurable for more memory.
And, specifically states that the 27-inch can be configured for twice the memory, and has user-accessible slots.

That same page lists a lot of differences between the two screen sizes.
Bear mentioned that hard drives are different speed. That's listed, too, but not that the drives are different physical size. 21.5-inch is 2.5-inch, and 27-inch is 3.5-inch drive. That's why you have the option of 3 TB drive in the 27-inch, but not in the 21.5-inch, there's not space for the full size drive in that smaller case.
There's probably other considerations, such as the fact that the smaller screen will fit into locations where the 27-inch will be too large - but that is simply something that you have to decide for yourself - no reason for that to be in a printed comparison table.

 

The 21" doesn't have the access cover for the RAM like the 27" does.

 

Yes, exactly.
An access cover, or door, would make the memory user-accessible, and there's nothing about the 21.5-inch that is user-accessible.
Apple does say this about the memory - http://support.apple.com/kb/HT1423