Register FAQ / Rules Forum Spy Search Today's Posts Mark Forums Read
Go Back   MacRumors Forums > Apple Systems and Services > Programming > iPhone/iPad Programming

Reply
 
Thread Tools Search this Thread Display Modes
Old May 5, 2013, 04:34 PM   #1
Menneisyys2
macrumors 601
 
Join Date: Jun 2011
TUTORIAL: Why does your TV Out Picture Have a Black Border and How can it be Fixed?

The following article is targeted at
- people using any kind of external display with their iDevices
- programmers that would like to provide as good external display output in their apps as possible. Programmers may want to go right at the bottom, to section "1.2.4 Disable overscan in third-party apps" in the second post, for my remarks and my example app.


There have been a lot of misconceptions regarding TV output on iDevices; most importantly, with regards to the black borders on not only the left and right sides of the external image, but also on the top and bottom. As an engineer particularly well versed in both video technology and engineering (you can ask me any question about anything video-related) and iOS programming, I found it necessary to get the facts right in a manner that should be digestable at least by advanced iOS users (and all programmers).

So, you've purchased a HDMI adapter or a component / composite cable, hooked it up to your external TV / projector only to find out the picture has sometimes thick borders around them almost all the time, except for playing back videos in the stock Videos – or, prior to iOS6, stock YouTube - client? (Note: VGA adapters, which are also available for iDevices, have deliberately been left off the list above. They, in most cases, behave differently than the just-listed three “consumer” video signal transfer modes. Also note that, should you connect the HDMI output to a DVI input, you may not encounter borders either. More on these exceptions later.)

This applies to both several stock apps (for example, Photos) and, at the time of this writing, the majority of third-party apps. For example, at least half of AppStore video players display the same border around their output as, say, the 16:9 iPhone 5 in mirrored landscape mode. An example of these is BUZZ Player, of which user “Timpax” posted the following HERE, on Apr 25, 2012 11:28 AM:

“For example native apps, like YouTube, never have the problem, they always fill the 1920x1280 correctly. Also Air Video app seems to work all the time. But some apps dont, for exampe Buzz player does not manage to fill the whole resolution 1920x1280, it leaves empty space on all 4 sides. I am using video projector and find this very frustrating.”

Let's take a look at this:


(iPhone 5 mirrored mode. Every single image in this article is only a thumbnail. Click them to access the full-sized originals.)


(the standardized Monsters test video played back by Buzz Playeron an iPad 3 over HDMI. Note that the same border would be visible when playing the video in a lot of other players, for example, GoodPlayer. In the latter, the additional “TV Out Tweak mode” doesn't hide the windowbox.)

The above two pictures are direct HDMI stream captures with the excellent Elgato Game Capture HD. Let me show you a traditional shot too, showing the border in question on a real external monitor, an LG W2253V:


Note that I used the output of my custom-written display output tester app as the displayed image. (More on the app below.)

I've composed this shot to emphasize several important facts:

1, I've lit the upper left corner with an external halogen bulb so that you can more easily see where the bezel of the monitor ends, that is, where the black border of the windowbox is.
2, I've made the monitor display the on-screen display (OSD) menu with the Overscan switch. As you can see, it's off, which is the default.
3, at the bottom of the image, I show my iPad 3 displaying the control panel of my video output tester. Of particular interest is the “OverscanCompensation” (here, because I've forgotten to enlarge the label with the text, it reads as “OverscanCompensati...”) segmented control in the center, left at 0. We'll later learn what the different values mean.
4, connected to the iPad, as you can easily recognize, is the HDMI (not the VGA) adapter.

Now, what's the problem with this shot? Yes, the subject of this article, the thick windowbox border on all sides, which, as you can check yourself too by simply measuring the first two “pure” HDMI stream still captures, reduces the, theoretically Full HD (1920 * 1080) resolution to 1728*972.

1.1 Why exactly 1728*972?

Because, as is also shown HERE in the dedicated Wiki article, a commonly used overscan (extra image area around the four edges of a video image that may not be seen reliably by the viewer) is 5% per side, that is, 10% for the two sides in each (horizontal and vertical) direction. A quick calculation: 1920-192 * 1080-108= 1728*972 indeed.

Apple uses this by default over HDMI, except for some DVI cases. This is why there's a black border around the iPhone5's otherwise natively 16:9 mirrored screen instead of it filling the entire external screen – and this is why other, non-natively-16:9 iDevices' mirrored screen has black bars above and below them.

1.2 Hiding the black bars

This section is entirely dedicated to fixing the above problem, that is, “zooming” into the picture – preferably by maximizing image quality (resolution).

1.2.1 Can the black bars be hidden using the stock software?

Unfortunately, unlike in OS X, this is impossible. There simply isn't a Settings option to disable overscanning.

This is, given that no one would ever display high-resolution video signals on a CRT monitor, is, at least, questionable. Apple really should fix this problem by adding the same switch as has already done in OS X. Up until then, you'll need to use other ways of enlarging the output if you mirror the screen.

If, on the other hand, you're a programmer and work on an app with native TV output, you must pay special attention to let users disable overscanning from inside you app. Fortunately, it's very easy in iOS5, as you'll see below – for example, when discussing the monitor tester application I've written. This way, you can avoid users (like “Timpax”; see his BUZZ Player-related remark above) getting disappointed.

Regarding Apple's having made a mistake by not providing us with a system-level switch, let me also recite the post of user “AppleZombie”, at Aug 12, 2012, from HERE:

“The Airplay MIRRORING is very frustrating. OSX Mounatin Lion has no problem, thankfully. Unfortunately Apple engineers believe overscan is necessary for ios devices, which makes no sense. How many users out there have their AppleTVs connected via HDMI to an old CRT SDTV that uses overscan??? HDTVs are basically the same as computer monitors... A 1080p signal can match pixel per pixel. What old SD CRT monitors are Apple QA folks testing this stuff on? Do we need to send them a cheap 1080p TV so they can see that overscan is not needed? I just dont get why they do it this way.”

All in all, if you do want to disable overscanning, you can't do it from inside iOS – yet. You need to configure either the monitor or cleverly change the connection type, if possible. Let's start with the former.

1.2.2 Enabling overscanning on the monitor

In the previous photo, I've shown an OSD with an Overscan switch, which is disabled (off) by default. If you enable it, the borders become somewhat thinner but still don't disappear:



However, this is the least favorable solution because:

- the display receives essentially a lower-resolution image, which, then, bumps in size. This means there is two scaling operation: first, the iDevice scales the originally Full HD picture down to the size 1728*972 (see “1.1 Why exactly 1728*972” above for more info on why it's exactly 1728*972). Then, after having received the scaled image, the monitor applies another scaling operation it, further decreasing its quality. (Every scaling / resizing operation decreases quality and, in cases, can fully destroy the inner structure of the image you're trying to display, as we'll see later when I explain what the “pixel size” segmented control changes in my app.)

- you will end up having to continuously en- and disable this switch if you also play non-overscanning material on the same display. This includes, as has already been mentioned, the stock Videos app and, if you aren't on iOS 6, the built-in YouTube app. (Note that the AppStore-based YouTube app, at the time of writing, has no native TV out support.) Also, any third-party app with disabled overscan will appear without the outer 2-3%. Let me show you an example of this of the very same app but with overscanning disabled in-app (see the OverscanCompensation segmented control set to 2), while, of course, it's enabled on the monitor (see the uppermost, enabled “Overscan” switch in the OSD):



As you can see, for example in the top left corner the first four characters (“phys”) of the label “physical” have disappeared; so did the "16:9" label in all the four corners.

That is, you should only use this if there is simply no other way of reducing or getting rid of the border by using other connection types or, if you're using a third-party app and not just mirroring, by asking the developer to completely disable overscanning or making it configurable. (Again, you can't do anything in software if you're mirroring only as, on the operating system level, you can't disable overscanning.)

1.2.3 Using another kind of connection

If your monitor / projector has a VGA and/or a DVI input, you may have an always-working solution with, because of the lack of two-stage scaling, the best image quality. (Note that I elaborate on the image quality of VGA below – it, in cases, can be lower than that of its digital counterparts.)

1.2.3.1 VGA adapter

In the introduction, I've already mentioned overscan is only applied to, in addition to the standard definition component and composite ones, native HDMI connections. If you have a VGA adapter around, you won't have unnecessary overscan; that is, no borders will be shown.

Note: I don't have access to the Lightning-VGA adapter (yet). Everything below should also apply to it. The only difference you'll see is that it does an implicit rescaling 900p → 1080p operation you can't in any way disable, just like with the Lightning-HDMI adapter. I'll elaborate on the latter (and its consequences) in a separate article.




In this case,

- the monitor won't even show you an Overscan switch as there can't be overscanning
- if the source app supports this, disabling overscanning in the input app won't have any effect. This means no matter what configuration the source app has, everything will be properly rendered on the external screen without any kind of useless borders. An example:



The only difference between this and the previous shot is the value of the OverscanCompensation segmented control, which, in the second shot, is set to 1. As you can see, without any effect on the final image. This is what I've just explained in the second bullet above.

1.2.3.1.1 Image quality issues of VGA

You may already heard people claiming VGA, being analog, has inherently worse image quality than the fully digital HDMI (or DVI, see below). This is only partly true: some configurations suffer from image quality degradations, some others don't.

Of the two LCD monitors I've tested the VGA rendering quality of my app, on the Samsung 2243SW, the VGA output exhibited noise in the high-frequency image areas. I've shot two videos showing this:

http://youtu.be/do0fG-xjic8 (shot at 1080p)

http://youtu.be/tU0-EGWfUQc (half-speed playback; shot at 720p60)

Pay particular attention to the noise in the homogenous grey area in the center left.

Note that, if you pause playback, the results (horizontal lines) are pretty much the same as is shown in some other demos

The DVI-D input of the same Samsung 2243SW monitor, driven by the same iPad 3 via the HDMI adapter (that is, using an entirely digital connection), produced absolutely clean image.

The image quality was practically indistuingishable between the HDMI / DVI-D (full digital) and the VGA (full analog) inputs of the other test LCD monitor, the LG W2253V using exactly the same VGA / HDMI adapters and cables. That is, it wasn't the cable or the adapter that caused the noise on the Samsung 2243SW as they worked just fine with the LG.

All in all, it can't be stated “digital (e.g., HDMI) is definitely better than analog (VGA)”. You may never see any difference – as I haven't on the LG either.

Note that this also emphasized by a lot of posters HERE - several posters stated they haven't seen any differences between VGA and HDI / DVI; other members stated the opposite. As has been explained above, both parties are right.

1.2.3.2 Driving the DVI(-D) input via HDMI (if present)

Many monitors / projectors have DVI inputs. Note that simple single-link DVI-D is sufficient: it doesn't need to be the more advanced dual-link or DVI-I (integrated) type.

When connecting iDevices to some (not all!) monitors using a HDMI -> DVI (passive) cable or a (passive, cheap) adapter (examples of them is HERE and HERE, respectively), no overscanning will be applied on either part, as opposed to the purely HDMI-connected case, where, by default, overscanning and, consequently, image quality-degrading scaling and windowboxing will be used.

An example shot with the iPad connected to the LG W2253V monitor using the HDMI adapter, connected to the DVI(-D) input of the monitor (and not the native HDMI):



As you can see,
- the image is presented full screen, at the native Full HD resolution, with flawless (1:1) pixel mapping
- all this without touching the overscan-compensation segmented control in my tester app (that is, leaving it at the default 0)

As has been mentioned above, not all monitors disable overscan in this setup – unlike with the VGA adapter, where in no way will overscan be used. For example, the Samsung 2243SW will still use overscanning. Hope you're lucky to have a monitor making the iDevice not apply overscanning in this setup.

Note that the three available HDMI adapters behave somewhat differently:


- of the two 30-pin adapters, it's only the first-gen MC953ZM/A (model number A1388) that doesn't apply overscanning; the second-gen MD098ZM/A (A1422) does, unfortunately. This is also pointed out in my latest article on the differences of the two adapters.

- the new Lightning one (A1438) with firmware version 6.1.0 has no overscanning either, fortunately.


Again, let me point you that, even if you have an adapter (the first-gen 30-pin one or the Lightning-HDMI one) not using overscan when connected to a DVI input, your monitor may still force it to use it.

1.2.3.3 Using AirPlay with the Apple TV

You can also display the image of your apps wirelessly using the 2nd- or 3rd-gen AppleTV, assuming you can put up with the considerable lag (which can render some kinds of games unplayable), the considerably lower image quality and the reduced framerate (30 fps vs. 60 fps) compared to any kind of HDMI / VGA-based cabled solutions.

On the AppleTV, you can disable overscan right in Settings:



Just set "Adjust For AirPlay Overscan" (selected above) to off.

Note that, due to the restrictions of the AirPlay protocol, you're restricted to, in addition to 30 fps only, 720p resolution. This is the list of the usable resolution when connected to an AppleTV 3 (shot on an iPhone 5; other models would also show the same):





An example shot of my app's output over ATV. Note the major compression artefacts introduced by AirPlay particularly in the 7-8 section of the vertical resolution checker lines:





Unfortunately, unless you want to display images using the built-in stock Photos app or play iOS-native videos with hardware decoding, you simply cannot have better quality with the AppleTV.




(tbc in the next post below)

Last edited by Menneisyys2; May 11, 2013 at 07:57 AM.
Menneisyys2 is offline   1 Reply With Quote
Old May 5, 2013, 04:35 PM   #2
Menneisyys2
Thread Starter
macrumors 601
 
Join Date: Jun 2011
(continued from above)

1.2.4 Disable overscan in third-party apps

If you can make a developer do what you ask for – or are the developer itself -, there is another way of entirely disabling overscanning. Again: it'll only work in the given app and not in others, including the standard operating system-level mirroring.

All you need to do is, upon setting up the external screen (via, say, if ([[UIScreen screens] count] > 1) externalScreen = (UIScreen *)[[UIScreen screens] objectAtIndex:1];), set the overscanCompensation property of the above UIScreen instance to UIScreenOverscanCompensationInsetApplicationFrame (=2). It'll entirely get rid of both the border (overscanning) and image quality-deterioating scaling.

Note that, over at StackOverflow, people recommend using “3” instead (see for example THIS and http://stackoverflow.com/questions/1...pter-on-ipad-2 ). I've made a lot of tests (under iOS5.1.1 (iPad1), 6.1.2 (iPhone 5, iPad 3 etc., most JB'n) and 6.1.3 (iPad 3 non-JB'n)) and found it to be exactly the same as UIScreenOverscanCompensationInsetApplicationFrame (=2). The difference may have been a pre-iOS5.1.1 issue.

1.2.4.1 My demo app

I've referred to my demo app (with full sources), along with posting shots of it in action, several times in this article. It's available HERE as a freely compilable and deployable Xcode project. As has been explained, you want to test the effects of the “OverscanCompensation” segmented control, keeping in mind that it'll have any effect over connections where there indeed is overscanning – that is, definitely not over VGA or, with some monitors, an HDMI -> DVI connector.

Basically, the app creates an external screen if there is one. (If it doesn't, it just creates a moire tester surface in the 100*100 pixel area in the upper left corner. I'll explain why it does so in my future article dedicated to the inherent problems of Apple's current Lightning-HDMI / VGA adapters.) On that screen, it displays the following:

- an ISO 12233 chart with a static moire tester in the center left area. The image has been created by me and has two versions, both in the app bundle. One is of resolution 1080p; the other is 900p. (Basically, I've created these two versions so that I can test both “old” 30-pin and new Lightning adapters. The former have the resolution of 1080p, the latter 900p.) You can switch between the two versions using the “pic” segmented control (in ExtDisplayDemoViewController, its action callback method is picSizeChanged)

- a wide label at 1, 110 so that all custom settings are shown on the external screen too, making it easy to correctly and easily identify shots of the output


- the above-mentioned programmatically-generated moire tester with settable rectangle size. The default setting is 1; that is, it sets each pixel in the 100*100 upper left corner to alternate between off and on. If, on the GUI, you set the lowermost “pixel size” segmented control to 2, 3 or 4, these rectangles become 2, 3 or 4 in size.

All the drawing operations take place in the UIView-descendant CustomViewWithOverriddenDrawRect.

UPDATE (9/May/2013): it was certainly worth writing this tutorial. After talking to the nPlayer developers (original thread; discussion starts at #2 - users complain about windowboxing), they, based on my tutorial above, quickly implemented disabling overscanning. That is, the next version (1.12) will already use fullscreen playback, as opposed to the (windowboxed) current one.

Hope other developers follow.

Last edited by Menneisyys2; May 14, 2013 at 07:24 PM.
Menneisyys2 is offline   0 Reply With Quote
Old May 5, 2013, 06:02 PM   #3
ArtOfWarfare
macrumors 601
 
ArtOfWarfare's Avatar
 
Join Date: Nov 2007
Send a message via Skype™ to ArtOfWarfare
Thanks for the insightful post
__________________
Battery Status - On the Mac App Store
The only app that'll estimate when your wireless devices will need their batteries changed.
Including the ones paired with other Macs on your network.
ArtOfWarfare is offline   0 Reply With Quote
Old May 16, 2013, 10:15 PM   #4
xArtx
macrumors 6502a
 
Join Date: Mar 2012
Wow, where on Earth do you get a test pattern like that?

& thanks
xArtx is offline   0 Reply With Quote
Old May 19, 2013, 05:02 PM   #5
Duncan C
macrumors 6502a
 
Duncan C's Avatar
 
Join Date: Jan 2008
Location: Northern Virginia
 
see vendor information in user profile
Very useful, insightful post. Thanks for taking the time to share your knowledge.
__________________
Regards,
Duncan Champney, WareTo.
Check out our latest iOS app, Face Dancer, available for free on the App Store.
Duncan C is offline   0 Reply With Quote
Old May 22, 2013, 06:45 AM   #6
Menneisyys2
Thread Starter
macrumors 601
 
Join Date: Jun 2011
Quote:
Originally Posted by xArtx View Post
Wow, where on Earth do you get a test pattern like that?

& thanks
I've generated it myself. Interested in the parametrized version (for iOS), capable of generating checkerboards with square sizes of 2, 4, 6... etc. pixels? I can dig it up and paste it here.

(A non-parametrized, 2-pixel-only version is in the code I've originally linked to in the original article.)
Menneisyys2 is offline   0 Reply With Quote
Old May 22, 2013, 07:20 AM   #7
xArtx
macrumors 6502a
 
Join Date: Mar 2012
Quote:
Originally Posted by Menneisyys2 View Post
I've generated it myself. Interested in the parametrized version (for iOS), capable of generating checkerboards with square sizes of 2, 4, 6... etc. pixels? I can dig it up and paste it here.

(A non-parametrized, 2-pixel-only version is in the code I've originally linked to in the original article.)
Yes please

Thanks you very much
and for the article!

I've got 30 pin composite, component, VGA, and HDMI adapters, and also lightning to HDMI.
Since Apple were late with the lightning to VGA adapter,
I went and got a hardware HDMI to VGA converter.

Last edited by xArtx; May 22, 2013 at 07:45 AM.
xArtx is offline   0 Reply With Quote
Old May 22, 2013, 09:33 AM   #8
Menneisyys2
Thread Starter
macrumors 601
 
Join Date: Jun 2011
Quote:
Originally Posted by xArtx View Post
Yes please

Thanks you very much

I've uploaded that version to https://dl.dropboxusercontent.com/u/...generation.zip . Note that, to keep the size down, I've deleted all the ISO 12233 reschart PNG's from it - you'll need to copy them back to the bundle before running (any placeholder will do).

The code you'll want to check out is in CustomViewWithOverriddenDrawRect.drawRect. The main loop drawing the 2/4/6/8/10/12/14 checkerboards is as follows:
Code:
     for (int currPixelSize=1; currPixelSize<8; currPixelSize++)
     {
         X_BIAS = width*(currPixelSize-1);
         int startColumn = 0;
         CGContextRef context = UIGraphicsGetCurrentContext();
         for (int y=0; y<height; y=y+currPixelSize)
         {
             for (int z=0; z<currPixelSize; z++)
             {
                 for (int x=startColumn; x<width; x=x+currPixelSize*2) //  (y%2)
                 {
                     CGContextBeginPath (context);
                     CGContextMoveToPoint(context, x+X_BIAS, y+z+Y_BIAS);
                     CGContextAddLineToPoint(context, x + currPixelSize+ X_BIAS, y+z+Y_BIAS);
                     CGContextStrokePath(context);
                 }
                 
             }
             if (startColumn==currPixelSize) startColumn = 0;
             else startColumn = currPixelSize;
         }
         
     }
Quote:
and for the article!
You're welcome

Quote:
I've got 30 pin composite, component, VGA, and HDMI adapters, and also lightning to HDMI.
Since Apple were late with the lightning to VGA adapter,
I went and got a hardware HDMI to VGA converter.
I too have one - ordered from dx.com about a year ago. It's pretty OK; of course, the output IQ is somewhat worse than that of the input.

BTW, I've published several other TV out-related articles (the latest, on Keynote's TV output, today); if you search for the threads started by me here at MR, you'll find them all.
Menneisyys2 is offline   0 Reply With Quote
Old May 23, 2013, 12:38 AM   #9
xArtx
macrumors 6502a
 
Join Date: Mar 2012
Great, you can imagine, lightning to HDMI adapter, to hardware converter,
through to VGA, is a clumsy chain I'd like to pass the image through.

When I get the real Apple lightning to VGA adapter, it's going to be better,
and this hardware converter thing that doesn't work with DVD players,
and more expensive than the real Apple accessory will be useless!

Slightly off topic, I'd bet you'd like this thing I made yrs ago:
http://www.freewebs.com/defxev/VideoInv.htm
xArtx is offline   0 Reply With Quote
Old May 23, 2013, 09:24 AM   #10
Menneisyys2
Thread Starter
macrumors 601
 
Join Date: Jun 2011
Quote:
Originally Posted by xArtx View Post
Great, you can imagine, lightning to HDMI adapter, to hardware converter,
through to VGA, is a clumsy chain I'd like to pass the image through.

When I get the real Apple lightning to VGA adapter, it's going to be better,
and this hardware converter thing that doesn't work with DVD players,
and more expensive than the real Apple accessory will be useless!

Slightly off topic, I'd bet you'd like this thing I made yrs ago:
http://www.freewebs.com/defxev/VideoInv.htm
Btw, don't you have DVI-D input? Only VGA?
Menneisyys2 is offline   0 Reply With Quote
Old May 23, 2013, 08:37 PM   #11
xArtx
macrumors 6502a
 
Join Date: Mar 2012
Quote:
Originally Posted by Menneisyys2 View Post
Btw, don't you have DVI-D input? Only VGA?
My computer is using it.
If the computer is plugged in to the monitor DVI,
and something else plugged into VGA, the monitor shows the
picture for whatever is switched on.
xArtx is offline   0 Reply With Quote

Reply
MacRumors Forums > Apple Systems and Services > Programming > iPhone/iPad Programming

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -5. The time now is 04:09 PM.

Mac Rumors | Mac | iPhone | iPhone Game Reviews | iPhone Apps

Mobile Version | Fixed | Fluid | Fluid HD
Copyright 2002-2013, MacRumors.com, LLC