OK, an update I've just finished downloading the 720p version of the same file and the quality is fine, here's some stats for you:-
The 720p file has a video bitrate of 3964kbps while the 1080p has a video bitrate of 4921kbps, the 1080p file has a 2.25 times the pixels but only a 1.24 times increase in bitrate, meaning the 1080p encode suffers as a result. To get the same level of quality you'd expect an equal increase in bitrate as there is in pixel density.
That is only part of the math. And that part is wrong.
1080p24 is actually 1088x1920x24 (there are actually 1088 lines encoded in 1080p because 1080 can't be equally divided into full macroblocks) which is a raw pixel rate of 50,135,040. 720p30 is 1280x720x30 which yields 27,648,000. The difference is then x1.81 rather than x2.25.
To get the actual bit rate for 8-bit video at 4:2:2 you have to multiply those numbers by 12 (x8 for luminance + x2 for Pr and + x2 for Pb). Using your delivered bit rates, that means that the compression rate for 1080p24 is 122:1, while the compression rate for 720p30 is 83:1. But this is probably 4:2:0, meaning that the compression rates are probably closer to 102:1 and 70:1. But since it is download, there is no streaming bit starving to cannabalize motion; if more bits need to be allocated to motion, they can be, unlike live OTA TV.
That's not really all that bad; MPEG4 AVC can be compressed at about 30% higher than MPEG2, and MPEG2 delivery OTA is typically compressed at about 100:1. But this does support your theory that compression is more severe for 1080p than for 720p from iTunes.
But what really causes the artifacts you are seeing is the fact that it is originally 8-bit video compressed secondarily for iTunes distribution. This does not manifest so much as a reduction of resolution (in fact the images posted seem to maintain resolution pretty well) as it does in a contouring effect, where mild changes in luminance or pastel colors are represented by banding.
This happens because the quantization levels are so far apart in 8 bit that they tend to show a lot of rounding error. The difference between quantization levels in 8-bit (232 levels) is over 4 times greater than it is for 10-bit (1006 levels), for instance, and we can see an obvious difference between adjacent levels. If a part of the picture gradually increases in luminance, at some point there is a true "quantum leap", where that part of the image is noticebly brighter, and we can see the line of demarcation there. And the noisier or busier the picture is in that area, the more that line jumps around. It aint pretty.
A good example of this is the face of the lead actor; it appears that his lower face is much darker than his forehead, almost giving him the appearance of a 5 o'clock shadow. You can see this on the actress in the other grab as well. In real video it also moves around, so it is really noticable and really annoying; enough to take us out of our suspension of disbelief that we are watching real images instead of replicated images.
And this is the result when 8-bit video is compressed severely a second time at 4:2:0. But this is all tradeoff; if they compressed less, it would give a better picture, but it would take longer to download. Online download is between a rock (high compression compromising the quality) and a hard place (internet speeds are shamefully low and ridiculously overpriced in the US). And 8-bit video really aggravates that quality problem. If they could create these downloads from better-quality masters, as they are starting to do in iTunes for audio, this would solve much of the problem.
But anyone who has spent any time moving content from a Mac to an iPad will see that this is pretty common there, too. Or if you manipulate something in iMovie. You get the same reduction in quality. I find iMovie so bad that I just can't use it because I can't tolerate how bad the quality is, and there is no option to increase the quality at the expense of processing/rendering time as there should be.
Also, how good the original is makes a real difference. I downloaded a free pre-air copy of Deception last month, and it looked remarkably good. Movies usually look pretty good, because they usually are from better masters than TV shows. All consumer delivery of TV is 8-bit 4:2:0, and it is difficult to go back up the chain to a mezzanine level of higher quality, Deception nonwithstanding. More accurately, it is more a matter of availability; TV shows are typically widely available at TV delivery quality so they get them there rather than going back up the chain, while movies may only be available as high-quality telecine dubs from the studios, meaning they are forced to accept a higher quality copy.