Everything You Need to Know About High-Quality, Lossless Video Zooming

This article is targeted at advanced iPhone 5, 5c and 5s users using iOS7, video shooters, programmers coding video apps and Apple's engineers in charge of bugfixing.

Executive Summary:

Apple has done it! iOS7 indeed offers lossless zooming while shooting video on the iPhone 5, 5c and 5s. Some points to keep in mind:

- the stock Camera client does use lossless zooming, delivering the best possible image quality over the lossless limit. Under the lossless limit, however, its image quality is somewhat worse than the maximum achievable. Hope this bug is soon fixed by Apple. Before that, however, prefer it to anything AppStore-based, unless you're a developer and can, after developing further for your needs, deploy my proof-of-concept video recorder. The latter delivers definitely better IQ in the lossless zone.

- the current, non-iOS7-updated versions of the tested AppStore apps (FiLMiC Pro, KitCam) do NOT support lossless zooming. Should you have a lossless zooming-capable iPhone, for the time being, in no way use any third party app if you extensively zoom! Use the stock Camera app instead. Or, if you know how to program, feel free to modify / further develop my sources, which give you the best possible image quality.

Full article:

You may have heard the built-in, stock Camera app of the iPhone is also capable of zooming. All you need to do is doing a “pinch” before / while recording. A zoom slider will be, then, displayed, which you can slide to set any zoom level between 1x and 3x. The following screenshot shows it being at 3x:



(annotated by a red rectangle)

(Note: as with all my articles, you can access the full, quality-wise non-compromised version of the in-article thumbnails by clicking it and checking out the original at Flickr. As my articles are full of in-line images but, as I write on image quality-specific questions, I do need to provide you with full-quality and -size images as well – ones with sizes far exceeding the page width here –, this is the best approach.)

You may also have heard (article 1, 2) of contradicting information on iOS7 adding not only zoom, but also doing it in a(n almost) lossless way because of oversampling. This has, so far, been a territory of Nokia's high-end “PureView” handsets only (the 808, the 1020 etc.) With iOS7, iPhone 5's (and newer models) can at least in this respect play catch-up with Nokia's (otherwise, camera-wise, on the whole significantly better) cameraphone models. (The iPhone 4 and 4S, while they can be updated to iOS7 (the pre-requisite of lossless zooming), are not capable of lossless zooming. That is, this article doesn't apply to them.)

Well, I have some good news for you. It DOES work when properly(!) programmed. And for the bad news: the built-in, stock Camera app still doesn't properly support this feature (assuming you only zoom moderately and not over the lossless limit), not even as late as iOS 7.0.2, that is, more than two weeks after the first official, final iOS7 release. In addition, the most famous third-party AppStore apps with support for shooting video (FiLMiC Pro and KitCam) capable of zooming and I've tested (with FiLMiC Pro, hopefully only still), as they aren't (and, with KitCam, won't be) updated with the new iOS7 features, don't deliver quality image when zoomed in, either. The latter apps, as of 07/Oct, are not recommended at all for zoomed-in shooting.

With this article, I hope I can make both Apple fix this bug in their stock Camera app and application programmers implement proper zoom functionality too in their AppStore apps. To help the latter, I provide the full sources of a proof-of-concept, simple-but-working video recorder recording proper, lossless footage. After all, I also work as an iOS / Android lecturer loving sharing his programming knowledge with others. (Incidentally, HERE are two shots of my teaching Android programming to relatives' children even during my Summer holiday. The slides' language are, as you may have guessed, Finnish.)

Why do you need to know all this?

iPhones, apart from the lack of stereo audio recording, optical image stabilization (OIS) and wide field-of-view lens (all three supported by other flagships like the LG G2 running Android or the Nokia 92x/1020 running Windows Phone 8), produce really decent video footage under ideal circumstances. Actually, if you shoot on a tripod (where the lack of OIS doesn't have any detrimental effect on the image quality), you can easily have significantly(!) better image quality than the above-mentioned, high-end LG G2. Similarly, all iPhones starting with the iPhone 4 produce generally much better-detailed footage than most dedicated point-and-shoot cameras. They can, in video quality, easily beat even large-sensor DSLR's or ILC cameras because there's no aliasing or, unless you zoom in, moire in the image (unlike in the image of most large-sensor cameras). Of course, a large-sensor cameras will always have better dynamic range and less noise; that is, you'll only want to prefer the iPhone in absolutely ideal circumstances (no vast differences in lighting requiring a sensor with significantly better dynamic range; no low-light shooting etc.)

Also, there can be cases when you won't have any other camera with you. Recently, I have travelled to a conference – with my iOS7 (beta) iPhone 5 (and, just to be on the safe side, a tripod) only. It was there that I decided to record the entire conference on video. With knowing the limitations and range of the lossless zoom, I had much more liberty in placing my iPhone. As I could use a zoom up to 1.45x, I could place the phone a bit more far away from the speaker, knowing there will be almost no image quality degradation. Without lossless zoom, I would have had to place the camera significantly closer to the speaker.

This is what 1.45x zoom is capable of, framing liberty-wise, assuming the same camera & subject locations. When not zoomed in at all, the subject only fills part of the screen (and the output frame):



and this is after zooming in to the maximum possible lossless zoom level:


(Both screenshots show the stock Camera app, with its zoom slider visible so that you can also see the current zoom factor.)

As you can see, with all iPhone models starting with the 5, even in the default 1080p there is some way for zooming without sacrificing any quality (or having an optical zoom, of course). In no way as large as with Nokia's 41 Mpixel cameraphones, but still more than nothing, nonetheless. In addition, the lossless zoom helps a LOT with zoom factors over the 1.4545x limit - as we'll see when comparing the image quality acquirable with 3x zoom using both the partly lossless (new) and the fully lossy (old) approach.

How should it work?

If you read the above-linked articles on iOS7's lossless zoom and/or have read for example the excellent comparison of the effective resolution at different zoom levels of the Nokia 808 (the first cameraphone to have true lossless zoom) HERE, at the bottom of the page, you'll know lossless isn't really lossless. Oversampling will always(!) result in better pixel-level detail and less moire / aliasing. The quality deterioration, however, will still be much-much lower than with traditional, digital zooming - as we'll promptly see.

Let's see how this all works! Let me present you with three ISO 12233 resolution chart (reschart for short) shots. (The inline images are crops. You'll want to check out the original, full-resolution ones instead. Please consult my earlier articles - for example, THIS - on how these reschart shots should be evaluated.)

Let's start with 720p video framegrabs as they offer a whopping 2.18x (almost-)lossless zoom. Now, we'll see the vast quality improvement lossless zoom provides us! (Note that, for an introduction, I've chosen 720p over 1080p as it much more readily presents the vast quality difference between lossless and lossy zoom as it can zoom in up to 2.18x, while the default 1080p mode can only zoom up to 1.4545x losslessly. Later on, I'll also present lossless 1080p zoomed framegrabs.)

Here's a 720p framegrab crop (again, click the thumbnail for the full ISO 12233 chart shot!) showing the effective resolution of the recorded footage without zooming in at all:




Note: in these crops, as with my previous, camera-related articles, I used a red line to denote the point after which, towards higher resolutions, the horizontal lines can't reliably be counted any more. (There are nine such lines on the right. If, at a given position, you can still easily count them, then, that resolution is still truly rendered by the camera.) For example, in the above crop, it's exactly at 7.2 because 7.2 (720 lines) is the Nyquist threshold or 720p video modes and all iPhones are capable of recording that much of true detail when not zoomed in.

Now, let's take a look at a framegrab from a video shot at the (for 720p) maximum 2.18x lossless zoom:




There is significantly more color moire in the image (which is pretty common with such dynamic oversampling-based zoom methods) but, otherwise, the nine lines on the right can still be easily counted at even 700 lines or resolution. With traditional, lossy digital zoom, the value would be less(!) than half (after all, the zoom factor is more than two) of this.

Now, let's see how the effective resolution drops and the image quality becomes worse when entering the already-lossy zoom range, which, with 720p, is over 2.18x. Take a look at the same resolution chart with 3x zoom:




Ouch! While the zoom factor (0.82) between the previous 2.18 and the current 3 is significantly lower than between the first two (1.18), the difference in image quality is staggering. The resolution dropped a lot; now, the nine lines are absolutely uncountable on the right and can only be reliably counted on the left up to around 480 lines of resolution. (See the red annotation, which is at around the 4.8 mark.) That's a HUGE drop in resolution – much-much more than between the first two images!

We'll soon see the same setup with 1080p (the native recording format of all iPhones starting with the 4S). First, however, let me introduce the test setup – and my zooming app I've written specifically for this article.

Test setup

I've used a (with a specialized video LED reflector) well-lit ISO 12233 resolution chart printed in A4. (Being very well-lit is very important so that the camera can record at base ISO and won't introduce detail smearing because of noise reduction.) I've continuously moved the iPhone, fastened to a pro, expensive (Manfrotto) tripod, to a more far position when zooming in:




When testing third-party apps where it's not possible to zoom right to the end of the lossless zoom range, I started my camera client (see below), set it to either the maximum lossless zoom factor with a single touch on the “1.4x” button, carefully positioned the tripod and, then, switched to the to-be-tested app and zoomed with its controls until I got the entire reschart properly framed. This way, I could be absolutely sure I used exactly the same zoom ratios in all the tested apps.

Note that I also checked for the electronic image stabilization (EIS for short) of each app. With FiLMiC Pro, it's disabled by default (and can be enabled); with the other apps (including the stock Camera client), it's always-on. With the latter, before I positioned the tripod by moving it closer to the reschart / more far away from it, I enabled the “EIS” switch in my video recorder app so that I can account for the different field-of-view when EIS is enabled.

My application

As has already been stated, I've written an app specifically for the purposes of this article: to reliably test whether the iOS7 zoom is indeed lossless, and to switch between zoom levels and en/disabled EIS state as easily as possible.

It's a really simple app (I didn't bother implementing for example touch-to-focus, GUI rotation or even exporting to the Camera Roll to keep the code as simple as possible) but can be used for even true video shooting if you need lossless zoom now. It's available as an Xcode project HERE. If you don't know how it should be compiled and deployed on your own iPhone, ask a friend. There are iOS a lot of programmers – you may also know one that you can quickly give your iPhone to for a quick installation. Needless to say, it's a source(-only) distribution – feel free to examine the code.

Here's the main interface:




Let's start with the upper left corner and introduce the controls clockwise.

The “Start” button in the upper left corner (annotated by a green rectangle) starts video recording. While it's recording, the title changes to “Stop”. Tap the latter to stop recording.

The “1x 1.4x 3x Max” control (annotated by a red rectangle) sets the zoom level. It can be done any time, even during recording. 1x doesn't use any zoom; 1.4x uses the maximum available lossless zoom (1.4545454545x for 1080p, 2.18x for 720p etc.); 3x zooms in three-fold and, finally, “Max” zooms in as much as possible. With 1080p, you can zoom up to 108 times (programmers: the value of activeFormat.videoMaxZoomFactor). (Of course, there isn't much point in zooming in THAT much. No wonder the stock Camera client tops up at 3x zoom and third-party AppStore apps don't go over 5-6x zoom either.)

At the bottom of the screen, the pink rectangle encloses shows the zoom currently used. In the screenshot above, it shows "1.454545" because I've, beforehand, tapped the 1.4x button (see above) to zoom in to the maximal lossless zoom level.

The back rectangle has an “IS:” label and a switch. With the latter, you can quickly en/disable EIS, even (as with the zoom level) during recording. Finally, the slider at the bottom (annotated by a blue rectangle) dynamically zooms between 1x and 3x, should you need continuous, arbitrary zooming.

Below, I use section numbering to denote a section being a subsection of another one. Otherwise, the article would be a mess. This also means you'll be able to easily skip entire sections. For example, if you aren't interested in the question of “should I always disable Image Stabilization”, just skip the entire section 1 straight to Section 2, where I explain what you can expect of currently available camera apps, starting with the stock Camera app.

(contd. below)

(contd. from above)


1. The effects of the image stabilization on the image quality

As has also been explained in my dedicated article, unfortunately, none of the current iOS devices have optical image stabilization (OIS). Electronic image stabilization (the one they use), on the other hand, use the outermost 10% of all the pixels for some rudimentary stabilization – as do all electronic image stabilization (EIS) methods on all non-OIS-capable cameras and phones (including even flagships like the Samsung GS4).

Note that, should you want to do these comparisons yourself, feel free to use the zoom and IS controls of my above app. To switch it back to 720p mode (see section 1.2 below), you'll also need to change the selectedAVCaptureDeviceFormatIdx = 15 assignment to selectedAVCaptureDeviceFormatIdx = 11 in VideoZoomerWithEISSwitchViewController. (11 is the full mode of 720p, while the original 15 is that of 1080p.)

Fortunately, should you need to use EIS (you're shooting handheld and not on a tripod), it won't have any negative effect on the effectivity of the lossless zoom. Let me prove this. First, a set of 1080p reschart images.

1.1 1080p tests

1, 1x zoom, IS OFF:




2, 1x zoom, IS ON:



As you can see, there is absolutely no difference between the IS-enabled/disabled states in the non-zoomed-in state.

Let's continue with zoomed-in images. Start with the maximal lossless zoom, that is, with 1080p, 1.454545:

3, max. lossless (1.45x) zoom, IS OFF:



4, max. lossless (1.45x) zoom, IS ON:




While the latter image has a little bit more color moire, the resolution is exactly the same as that of the non-IS shot.

Now, let's apply 3x zoom, which is way beyond the non-lossless range (which, again, is 1.454545x in the default 1080p mode).

5, 3x zoom, IS OFF:



6, 3x zoom, IS ON:



The non-IS version is a little bit sharper – but don't make this mislead you, it doesn't have better effective resolution, it's just sharpened. That is, you can use both modes equally well. Just don't forget that, if you zoom, you'd better use a tripod for the best results. (Nevertheless, if you already carry a tripod with you, you could also take a dedicated camera / camcorder. Even the worst, cheapest ones would produce better resolution around 3x+ zoom levels, assuming the camera does have optical zoom...)

1.2 720p tests

Now, let's take a look at something only achievable via third-party apps (the stock Camera app only records in 1080p): comparing IS-eabled to IS-disabled framegrabs taken from 720p footage (recorded using my video recorder).

As, in the very beginning, I've already provided three IS-disabled 720p framegrabs, I only give you their IS-enabled counterparts, in exactly the same order. Scroll up so that directly compare them to the IS-disabled versions.

1x zoom:




2.18x (maximal lossless at 720p) zoom:




3x (already lossy) zoom:




While the second two IS-enabled shots are somewhat better & sharper than the IS-disabled counterparts, the difference is still well beyond variable test variance. That is, both modes can equally well be used.

2. What about the other apps (including the stock Camera app) on iPhones?

So far, I've only provided you with framegrabs right from my dedicated (and very simple) video recorder. In this section, I shed light on how stock and AppStore apps record videos.

2.1 The stock Camera app

I have some mixed news. It doesn't seem to properly support the lossless zoom. Let me show you an example of this.

Compare the following shot (that of the stock Camera app) made at 1.4545x zoom factor (again, the maximum lossless zoom factor)




to the above-presented 1080p shots made with my client in bullets 3 and 4 of section “1.1 1080p tests”. The difference is obvious: while, in the above (inline) crop, my app could still resolve truly 1080p of horizontal resolution. This is why all vertical lines are visible in the lower right corner up to the mark “10”. With the stock Camera app's shot, there are no distinguishable vertical lines to the right of the mark “8”. That is, while my app did deliver full 1080p worth or resolution, the stock Camera app only delivered around 810-820p.

The latter result is still over the that of simple, traditional digital zoom. With digital zoom, the resolution would maximize out at 1080p / 1.45 = 745p. (Later, we'll see current AppStore apps not making use of lossless zoom at all are indeed only capable of true 745p at the 1.4545x zoom.) This all means the stock Camera client does lossless zoom, but, for some reason, it still delivers worse-than-expected results in the lossless zone. These results are, however, still better than those of apps that haven't been (properly) updated to support the new iOS7 zooming API.

Note that outside the lossless area (and at 1x zoom) the two apps produce exactly the same resolution. This is a 1x (that is, no-) zoom framegrab of the stock Camera app:




and this is the 3x one:




The latter should be compared to the IS-enabled version of my camera app – again, the IS-disabled one has somewhat sharper (but not more detailed!) output. The effective resolution of the two (that of my and the stock Camera app) shots are identical.

By the way, this is the app's GUI when recording at exactly 1.45x zoom factor:




Again, I used my app to properly move the tripod first so that I can be absolutely sure I'm setting the slider OK – after all, the Camera app doesn't display the current zoom ratio.

2.1.1 And the reason for the stock Camera app's delivering sub-par results compared to the ideal case?

Frankly – I don't know. I've made tons of tests to make absolutely sure the above-presented results are right. They are. In my app, I'm doing a simple AVCaptureDevice.videoZoomFactor assignment in control callbacks to change the zoom factor on the fly.

Finally, note that direct Full HD framegrabs while shooting video (see the “still shot” icon during video recording) will behave in exactly the same way as a framegrab grabbed from the video footage during post processing (after shooting), zoomed image quality-wise.

3. Third-party apps

Now that we've realized the stock Camera app is buggy and, when it comes to zooming inside the lossless range, delivers worse-than-expected results, let's take a look at the output of zooming-capable(!), popular third-party apps (except for KitCam) available in AppStore: KitCam, FiLMiC Pro and, when shooting in high-framerate modes only, SlowCam.

(cont'd below)

(contd. from above)


3.1 KitCam

KitCam (no longer in AppStore) supports zooming if you drag your finger at the bottom of the screen. Then, a zoom slider will be displayed, along with a label displaying the zoom level. I've annotated both in the following screenshot:




Unfortunately, the zoom it has is fully lossy. This is perfectly understandable as KitCam has been removed from the AppStore way before iOS7's (and the new, above-mentioned videoZoomFactor property needed for lossless zooming) has become available.

Let's start with the 1.4545 zoom (again, the highest lossless zoom factor available for 1080p shooting):




As you can see, the resolution is somewhat worse than that of the stock Camera client (about 820p) and much-much worse than that of my app (full 1080p). The vertical lines start to entirely be blurred at around 745p. This means there's absolutely no lossless zooming, as the traditional digital zoom would simple use 1080p / 1.4545 = 745p resolution. (As we'll see, the current version of FiLMiC Pro doesn't make use of lossless zoom either.)

Now, take a look at the 3x zoom:



Awful, isn't it? MUCH worse than that of either the stock Camera client or my app (the latter two, again, produce exactly the same image quality). The effective resolution is 360p only, which you can easily see from the shot above. This 360p, incidentally, is one-third of 1080p, showing all zooming has been done digitally, not making use of the lossless region.

Finally, as was easy to predict, at 1x zoom, KitCam produces exactly the same, excellent results (true 1080p resolution) as all the other camera apps:




All in all, stay away from KitCam on your lossless zoom-capable (iPhone 5+ with iOS7) phones if you ever plan to zoom! Even the stock Camera client has better quality (particularly over the lossless limit) and this won't ever be fixed!

3.2 FiLMiC Pro

FiLMiC Pro ($4.99) is also capable of zooming. I've annotated (with blue rectangles) the two (+ / -) icons you can use for zooming in the following screenshot:



I've tested the current, 3.3 version, released 15 Jul, 2013 – that is, two months before the official release of iOS7. This also means it cannot use the new zoom API using lossless zoom, which is certainly visible in the following two test shots:

This is the 1.4545x zoom resolution:



And this is that of the 3x zoom:



(Note that THIS flickr set also has EIS-enabled shots of FiLMiC, and 24p (as opposed to the 30p ones above) ones as well. Just look for images starting with “filmic”. After the first hyphen, the frame rate is listed (either 30p or 24p) and, at the end, after the zoom factor (1.4x or 3x) whether IS was on or off (EIS on or EIS off). I don't elaborate on them separately as both 30p/24p and EIS on/EIS off delivers the same image quality.

4. What about slow-mo recording?

One of the major selling points of the iPhone 5s is its new “slow motion” mode, also available in the iPhone 5 (iOS7+ only) and 5c. (And, I assume, the 4S too running on iOS7, but I couldn't confirm this myself. Note that the 4S can't do lossless zoom in iOS7 at all, regardless the video mode.)

While there is lossless zoom in the 720p60 (iPhone 5 / 5c) / 720p120 (iPhone 5s only) slow-mo too, its range is waaaaay smaller than that of the regular 720p mode and its 2.18x lossless zoom: 1.08x. That is, it's practically non-existing. And, as all these iPhones use pixel binning (halving the originally 720p resolution) when shooting in 720p60 / 720p120 mode, the originally already low resolution will become painfully low if you start zooming. You should never do that. If you still want to zoom while shooting slow-mo videos, consider getting a separate point-and-shoot camera with high-speed video recording. For example, even the really cheap Nikon P300 (released in early 2011 and received very good reviews – an example) is capable of 120 fps shooting at 640*480 and the effective resolution doesn't decrease if you zoom in.

Nevertheless, should you still zoom in, let me present you with some reschart shots of both my app and SlowCam ($1.99), one of the iOS7-compliant and pretty cool slowmo apps (review; pay special attention to my ('Menneisyys') comments under the article):

1, 1x zoom, my app (to be able to shoot 720p60 / 720p120 footage, make sure you change the selectedAVCaptureDeviceFormatIdx = 15 assignment to selectedAVCaptureDeviceFormatIdx = 13 in VideoZoomerWithEISSwitchViewController. (13 is the full mode of 720p60, while the original 15 is that of 1080p.)):




And that of SlowCam:



As you can see, there's no difference: both apps deliver way inferior results because of the pixel binning (meaning, again, 360p effective resolution).

2, 3x mode:

My app:



SlowCam:



No differences here either. The effective resolution is way worse than that of even 360p.

Nevertheless, at least the latter test shows you'll have exactly the same (bad) image quality using third-party, new, updated-for-iOS7 slow-mo apps like SlowCam than using my app or waiting for a third-party developer to come out with a solution.

UPDATE (07/Oct/2013 14:17 GMT): I've also scrutinized a new photo & video shooter in AppStore, ProCam ($free), which, at the moment, is available for free and is definitely a must at this price point as it covers mostly everything settable via the iOS camera API and more. For example, it allows for changing the video recording resolution, framerate and, which is very important if you need as wide field of view as possible without sacrificing resolution and supported by VERY few apps, also lets for disabling the stabilization (dedicated article on the advantages of doing so). I've annotated this in the next screenshot:




While the app (current version: 3.1.1) was last updated on 18/Sep/2013, it still doesn't make use of the new iOS7 zoom features, including being lossless. Instead, it always shoots at the default resolution (whatever you set) and downsamples afterwards, while saving the video. This not only adds a lengthy post-processing encoding to the video shooting process, but also, at it's just downsampling the 1080p original, works in exactly the same way as simple digital zooms, meaning inferior image quality.

The app warns users of the post-processing requirements when using the zoom while shooting video:




and here's a screenshot of its post-processing zoomed footage after shooting:




The zoom lever is around the shutter icon; I've annotated it below (red rectangle). The app also displays the current zoom factor. The latter, as the app doesn't make use of the iOS7 lossless zoom, is of not of much use (to set the maximal possible but still-lossless zoom without using third-party calibration like the way I've set other apps shoot at the lossless threshold calibrating the position of the camera beforehand using my zoomer app) now, though.




Unfortunately, the image quality of the zoomed-in footage is equally bad as with other, non-iOS7-updated apps (KitCam, FiLMiC Pro) and, consequently, is to be avoided on lossless zooming-capable iOS7 handsets (iPhone 5 / 5c / 5s). Two examples:

1.4545x zoom (lossless threshold at 1080p):



3x zoom (way beyond the lossless threshold at 1080p):