Apple's Lightning Digital AV Adapter is a Full-Fledged Computer

iSunrise said:

I´ve extracted (demuxed) the H.264 video track and AC3 audio track from MKV and repackaged (muxed) to M4V and added a second AAC audio track as the primary track (so the native Apple player can hw-decode it). The result is even worse than I remembered it. With iOS 6.1 currently on my iPad 3, I cannot make this file to play, with 5.1.1 I could at least make it play, even though it stuttered a lot. I even tested the MKV as well as the repackaged M4V in AirVideo with the video pass-through option and with the live transcoding option. The result is always the same, the file itself starts to play and the timecode advances, but I only get a black screen with audio. Can you please double-check and try to repackage the MKV with tools from OS X and see if you can make that file play without re-encoding the h.264 video?

Click to expand...

I've remuxed it with Subler (prolly the best OS X remuxer) into both an AAC-only and an AAC + AC3 video with default save params (m4v + no-64-bit video). On my 6.1.2 iPad 3, 4 and iPhone 5, apps directly using the HW decoder (including the stock Video apps itself, using iFunBox to directly transfer the video to it) show a black screen. "It's Playing" displays right at the beginning it's unable to play back the video using HW decoder and needs to fall back to SW decoding. That is, a no-go here either.

The problem MAY be something like malformed H.264 (e.g., the video doesn't start with a keyframe), which may cause problems like messed-up playback in the first 1-2 seconds (see THIS for an explanation). I'll run some further tests and report back.

Menneisyys2 said:

The problem MAY be something like malformed H.264 (e.g., the video doesn't start with a keyframe), which may cause problems like messed-up playback in the first 1-2 seconds (see THIS for an explanation). I'll run some further tests and report back.

Click to expand...

Tested -bsf h264_mp4toannexb on the direct Subler output (with AAC) - didn't produce a working file (actually, the file stopped working on desktop players and software decoders too). Simply optimizing in Subler didn't work either. Will continue reporting.

(I've somewhat edited / extended the original iFunBox article above.)

Menneisyys2 said:

b, the lag is, while somewhat larger (83ms (5 frames when playing back a 60 fps video) vs. 50 ms (3 frames)) than with an iPad 3 + (old) VGA / (old) HDMI adapter combo, is still considerably lower than the iPhone 5 + cabled, ideal ATV3 combo (150ms (9 frames)). That is, the lag is almost half of that of the standard wireless AirPlay, while "only" about 40% more than with the old adapters, on previous-gen iDevices (here, iPad 3). Whether this is sufficient for fast action games like Real Racing 2/3 - well, it's on the borderline, I'd say.

Click to expand...

Today, I've run some new benchmarks.

I've benchmarked the latency of the LG W2253V monitor I've made the tests videos. According to the article at http://www.prad.de/en/monitore/review/2009/review-lg-w2253tq-part12.html (the official specs of the monitor didn't have any data on input latency), its less-equipped (no HDMI input) brother, the W2253TQ, has an average lag of 9 ms.

Interestingly, in my first tests, I've seen far worse results (around 50 ms latency) when driving the monitor through the old VGA adapter from the iPad 3 using simple mirroring. Today, however, in the tests I've re-run, I've got much better results: the lag is practically unmeasurable with 60 fps test videos. (I've played the video at https://dl.dropbox.com/u/81986513/082012/sublerSubs/1080p60-counter - non-optimized.mp4 in GoodPlayer using its hardware decoder, entirely disabling the TV output so that the videos are played via simple screen mirroring.) However, the ATV3 AirPlay and the Lightning-HDMI latency figures have been pretty much the same as before:

- the former (ATV3 AirPlay, ATV3 driven via Ethernet; Wi-Fi Access Point is very close to iDevice) has shown about 133ms of lag (8 frames (latency) * 1 sec/60 frames = 0.133 sec) with both the iPhone 5 and iPad 3. The videos showing this for the iPhone 5 is at https://dl.dropbox.com/u/81986513/032013/0316-newHDMItests/ATV3-iPhone5-8frames.mp4 and the iPad 3 is at https://dl.dropbox.com/u/81986513/032013/0316-newHDMItests/ATV3-iPad3-8frames.mp4 .

- the latter (Lightning-HDMI + iPhone 5) has shown about 100ms latency (generally, a latency of six frames). The test video is at https://dl.dropbox.com/u/81986513/032013/0316-newHDMItests/LightningHDMI-iPhone5-6frames.mp4 .

- finally, the iPad 3 + (old) VGA adapter's test videos (not showing any kind of a lag):
a, with the LG W2253V LCD monitor, at https://dl.dropbox.com/u/81986513/032013/0316-newHDMItests/LCD-VGAout-ipad3.mp4
b, with the higher-end (Recommended mode: 1600 x 1200 @ 75 Hz ) CRT monitor IBM G97 Black 19" CRT Monitor (66344AN), at https://dl.dropbox.com/u/81986513/032013/0316-newHDMItests/CRT-VGAout-ipad3.mp4

BTW, yesterday, I've also run some external monitor tests – this time, with Flash-capable Web browsers. In the article https://forums.macrumors.com/threads/1557362/ I've published on this subject, you can easily see the lag introduced by the Lightning adapter on, say, the photo at https://dl.dropbox.com/u/81986513/032013/blutrol30/iphone5-photon-FS.jpeg (pay special attention to the iPhone, which shows the next scene's beginning frames superimposed to the old scene, while the - lagged - Lightning video still shows an earlier frame).

Hi Menneisyys2,

I wish there were more people like you out there willing to spend time in order to save others' time and money. Thank you.

I saw you have both RR2 and RR3 installed. In a post a few pages back you say that neither solutions are fit for fast games. My *only* use-case for the digital av adapter would be in order to play RR3 on the TV from my iPhone 5. Would it be possible to upload a short movie demonstrating the lag/playability or the lack of?

Thanks once more

fefence said:

Hi Menneisyys2,

I wish there were more people like you out there willing to spend time in order to save others' time and money. Thank you.

I saw you have both RR2 and RR3 installed. In a post a few pages back you say that neither solutions are fit for fast games. My *only* use-case for the digital av adapter would be in order to play RR3 on the TV from my iPhone 5. Would it be possible to upload a short movie demonstrating the lag/playability or the lack of?

Thanks once more

Click to expand...

Sure, will upload a video showing both the iPad 3 (old adapter) and the iPhone 5 (new one), hopefully as early as today.

fefence said:

Hi Menneisyys2,

I wish there were more people like you out there willing to spend time in order to save others' time and money. Thank you.

I saw you have both RR2 and RR3 installed. In a post a few pages back you say that neither solutions are fit for fast games. My *only* use-case for the digital av adapter would be in order to play RR3 on the TV from my iPhone 5. Would it be possible to upload a short movie demonstrating the lag/playability or the lack of?

Thanks once more

Click to expand...

Ready! RR3 doesn't support native (non-mirrored) TV output so the first three videos are all non-mirrored. This is why the iPad 3's output is so bad, I should add.

iPad3, VGA adapter: http://youtu.be/MweTBzIC6dw
iPhone5, ATV3, AirPlay: http://youtu.be/Los-kG8hd9I
iPhone5, Lightning-HDMI adapter: http://youtu.be/b2bnRMWC8hE

RR2, on the other hand, does support native TV output. This is why, for example, the intro video in RR2 is played back natively by the Lightning-HDMI adapter at full 1080p resolution (and not the reduced mirroring one) after starting the game - this is why video playback has no black bars around the active window, unlike when playing the game. (On the iPad 3, there're no black bars - it drives the monitor at Full HD.) Before starting and after exiting the game, I also play the 60 fps video (in mirrored mode, by explicitly disabling TV out as is shown at the beginning of the video - see the Settings dialog) in GoodPlayer. The latency is around 6 frames; that is, about 100ms.

At the end of the gameplay video, I quickly turn right and left several times to show you how fast mirroring is. Of course, I'm not that bad a player During the gameplay, I also switch on the OSD of the monitor to show you it's running in 1920*1080 mode, the mirrored output of the Lightning adapter being somewhat worse.

After the iPhone 5 video, I also present an iPad 3 one via the VGA adapter. As with the iPhone 5 video, I start and end with the 60 fps video test in mirrored mode so that you can see the latency (it's basically non-existent, as opposed to the Lightning adapter case). As it's not mirrored but native output, it's WAAAY better, both resolution, screen estate usage and stuttering (fps)-wise than the iPad 3-mirrored case (RR3, first video above).

RR2, iPhone5, Lightning-HDMI adapter, native (non-mirrored) mode: http://youtu.be/q8kyF5F-KgY

RR2, iPad 3, VGA adapter, native (non-mirrored) mode: http://youtu.be/kIVj-1zRA9E

EDIT: a side remark for jailbreakers: to evaluate the iPad 3 performance, I've chosen to entirely disable cabled mirroring on it. This can't be disabled by default in any way. As this hasn't been published yet, I took the liberty to write a quick writeup on it.

To disable the default mirroring over HDMI / VGA on the iPad 3, change "system / library / coreservices / springboard.app / J2aAP.plist / capabilities / display-mirroring" to NO. Note that, while the file N78aAP.plist also has the same set of capabilities, changing the same value in it is not noticed by the system. That is, don't modify it. (Original article describing enabling mirroring on the iPad 1, with a different plist filename: http://www.iphoneheat.com/2011/04/enable-video-mirroring-on-ipad-1-guide/ )

All this on a 64 GB iPad 3 3G purchased in Finland. Dunno whether iPads sold in other countries have different filenames - as was the case with my iPhone 5 video camera hack with AT&T users (see https://forums.macrumors.com/showthread.php?p=16801796 ).

Do you have a simple English translation? So is the HDMI adaptor a hack job or not?

Abazigal said:

Do you have a simple English translation? So is the HDMI adaptor a hack job or not?

Click to expand...

While not as good as the old VGA / HDMI adapters in any way (except for playing back native movie files, when it's identical to the old models), it's decent. You can safely buy it (unless you're an action gamer) and just disregard people stating it's useless.

Guys,

just posted some new effective resolution comparisons to https://forums.macrumors.com/showthread.php?p=17023301 for players natively supporting TV out.

As you can see, Lightning (in this case) has somewhat lower resolution than the old adapters but it's still better than that of AirPlay.

Great! That sure helped me feel better about spend 30 bucks on an adapter...

Starting with #221, we're discussing the Lightning-HDMI adapter at http://forums.toucharcade.com/showthread.php?t=173262&page=23

I've published a series of quality comparison screenshots here at MacRumors:

https://forums.macrumors.com/posts/17173581/

These shots are all grabbed with the Elgato Game Capture HD (which natively grabs HDMI), meaning they're far more reliable than previous shots made by a camera and a HDMI monitor.

Some updates

1, the Lightning adapter has its own About dialog under Settings > General > About, at the bottom of the main (middle) group. It's visible even after removing the adapter, unlike with the old RGB / composite cables (Settings > General > TV Out in the second (iPhone / iPt; below VPN / iTunes Wi-Fi Sync) / last (iPad; below Reset) group). Here's mine:



As you can see, my adapter's firmware is 6.1.0 and the hardware is 1.0.0. (No such dialogs with the old, “passive” 30-pin HDMI / VGA adapters, of course.)

2, I've shot some additional videos to check out whether it's indeed the iPad 3 that delivers far worse HDMI mirroring performance in fast-paced games than any other model. It does – every single other model based on A5 or A6 produces waaaaay faster mirroring speeds.

My benchmark videos are as follows. First, some explanation.

a, All direct grabs (non-camera videos) have been shot with the Elgato Game Capture HD in its “Preserve Source Format” mode, which, with all recordings, recorded 1080p30 footage, despite the original output being, in cases (for example, with iPhones in mirrored mode), of far lower resolution.

b, All dual-monitor camera shots have been shot with a Nikon P300 using its 720p60 mode. This, during playback, is played back with 30p; that is, with half the speed. It's an excellent tool to measure lag – the main reason of shooting dual-monitor shots, in the first place.

(Note that I haven't shot a demo video in every possible configuration.)

In all these cases, I was using the 30-pin HDMI adapter (A1388; that is, the first model – see http://www.ilounge.com/index.php/ba...l-av-adapter-gets-interesting-little-changes/ ) with the “old” (30-pin) A5+ iDevices (iPad 3, iPhone 4S) and the new Lightning – HDMI adapter (firmware version, as has already been stated, 6.1.0; HW: 1.0.0) on the iPhone 5 and the iPad 4. In some dual shot cases, I used the, otherwise, lag- and resolution-wise, identical VGA adapter.

Note that the lag of the Lightning-HDMI (and VGA) adapters have been widely discussed at both MacRumors ( https://forums.macrumors.com/threads/1550766/ ; see the posts starting with #428) and TouchArcade (http://forums.toucharcade.com/showthread.php?t=173262&page=23 ; dedicated posts starts at #221). It's in the 80-100 ms range, as opposed to the, in most cases, lag-less old 30-pin HDMI / VGA adapters. It's still much better than the at least 50% higher lag of the wireless AirPlay (I don't even mention the maximal 30 fps framerate, which is half of that of any cabled solution), which makes action / racing games almost impossible to play.

For the tests, I used three games:
- After Burner Climax by SEGA: a very fast-paced game without native TV out (that is, you need to rely on simple mirroring), which very easily shows the vastly reduced framerate of the iPad 3 on the external screen.
- Real Racing 2: both the iPhone and the HD version support native (non-mirrored) TV out resulting in far better performance on the iPad 3 and far better resolution on small-screen devices.
- Real Racing 3 (Universal): no native TV out support either, which means having to mirror, rendering the iPad 3 pretty useless and iPhones / iPod touches displaying pixelizated images.

After Burner Climax (mirrored), 1080p30 direct grabs:

iPad 3:
http://youtu.be/qmVQufnjs8g
As you can see, it's waaaay worse (plenty of dropped frames) than anything else A5-based.

iPad 4:
http://youtu.be/yppWpsVZjRo

While (as with the iPad 3) still having excellent resolution, it delivers waaaaaay better framerate. While the iPad 3 version was pretty useless, the iPad 4 is the opposite.

iPhone 4S:
http://youtu.be/KNLrjVPGsRA

iPad 2:
http://youtu.be/6YC-OKQ7aWU

Unlike the iPad 3, it's much better. Of course, its resolution isn't as good – but still far superior to that of the iPhone 4S.

After Burner Climax, 720p60 comparative shots played back at half the speed:

iPhone 4S:
http://youtu.be/MFlBQK7lnjI

iPhone 5:
http://www.youtube.com/watch?v=TalwoKz7bwo

The following recording has been recorded using the 30-pin VGA adapter and not the 30-pin HDMI one:

iPad 3:
http://www.youtube.com/watch?v=5pAa2CLJuik

Real Racing 2 (native non-mirrored output), 1080p30 direct grab

iPhone 4S:
http://youtu.be/PUUEkvWnBY4

Real Racing 2, 720p60 comparative shots played back at half the speed:

iPhone 4S:
http://youtu.be/enz7004BbHU

iPhone 5:
http://www.youtube.com/watch?v=q8kyF5F-KgY

The following recording has been recorded using the 30-pin VGA adapter and not the 30-pin HDMI one:

iPad 3:
http://www.youtube.com/watch?v=kIVj-1zRA9E

Real Racing 3 (mirrored), 1080p30 direct grab

iPad 4:
http://youtu.be/NDd6i4R5aEA

iPhone 4S:
http://youtu.be/AzbtHkgLVZs

Real Racing 3, 720p60 comparative shots played back at half the speed:

iPhone 4S:
http://youtu.be/EC7DQiqp530

iPhone 5:
http://www.youtube.com/watch?v=b2bnRMWC8hE

The following recording has been recorded using the 30-pin VGA adapter and not the 30-pin HDMI one:

iPad 3:
http://www.youtube.com/watch?v=MweTBzIC6dw

For comparison, here's the wireless AirPlay version of the same game, played on the iPhone 5, exhibiting quite bad lag:

http://www.youtube.com/watch?v=Los-kG8hd9I

So if I am getting this right. If I want to use mirroring often and play games on the tv I should get an iPad 2( don't care for iPhones) because the iPad 3 has to much frame drops and the iPad 4 has lag because of the lighting...?

Do you think apple can fix the lightning lag with an updates adapter soon or. Maybe we will have to upgrade to the newest I device wih will be announced soon?