Apple Opens Door to Hardware-Accelerated Decoding of H.264 Video in Flash

[tdream]

To all the fanboys. May I suggest you READ the actual article first before jumping once again to conclusions.

Apple Opens Door to Hardware-Accelerated Decoding of H.264 Video in Flash and Other Platforms

Apple Opens Door means the door was closed before by Apple not Adobe.

Thursday April 22, 2010 03:00 PM EST
Written by Eric Slivka

Michael Tsai reports (via Daring Fireball) that Apple has posted a new technical note describing how third-party developers can tap into hardware-accelerated decoding of H.264 video on compatible graphics cards.

The Video Decode Acceleration framework is a C programming interface providing low-level access to the H.264 decoding capabilities of compatible GPUs such as the NVIDIA GeForce 9400M, GeForce 320M or GeForce GT 330M. It is intended for use by advanced developers who specifically need hardware accelerated decode of video frames.
The move appears to provide Adobe with the means to implement hardware acceleration in its Flash Player, a feature that is utilized in Flash Player 10.1 for Windows to improve performance, but not included in the Mac OS X version due to Apple's refusal (until now) to allow third parties access to the required APIs for implementation.

The move by Apple allows Adobe to accelerate Flash.

Apple's refusal to allow access to APIs until now.

Flash performance on the Mac has been widely shown to be inferior to that on Windows, a deficiency that Adobe has noted it is working to address. Apple's offer of access to the tools necessary to implement hardware acceleration for video decoding appears to offer Adobe another avenue by which it can work to bring Flash performance on the Mac up to that on Windows.

Apple's offer not Adobe's. Flash performance on Windows is better because the PCs are generally higher spec than Macs, they allow access to APIs required, 90% of the world use Windows pcs, and PCs are expected to use the CPU and not cry about it.

 

[andrewfee]

Hm, I'm having a hard time deciding whether or not this is good news.

On one hand, it means that Apple is finally allowing other companies to implement H.264 decoding on the GPU, which will hopefully lead to much better video performance with flash and other applications. (and hopefully it won't take Adobe too long to implement)


On the other hand, Apple is only supporting this on three nVidia GPUs. There are many other nVidia GPUs that also support video decoding, as well as ATi and Intel graphics.

This means that, in addition to all the other applications triggering the nVidia 330M in the new MacBook Pros when it is not necessary, you will now have flash video switching over to it as well when the Intel graphics are more than capable for this. (if they can do Blu-ray decoding, they can more-or-less handle anything)


It's also disappointing that they are only enabling H.264 acceleration on the cards, and not MPEG2 or VC-1. I know Apple favours H.264, but the other codecs are still widely used.


Hopefully this is just the beginning for Apple, and they will gradually enable support for older nVidia graphics cards, as well as Intel and ATi. There's no reason to use the 330M in the MBPs for tasks that the Intel HD graphics are more than capable of handling. Wasn't that the whole point of the GPU switching?

Apple being Apple though, I wouldn't be surprised if everyone else is left behind, even though the cards are all capable of doing this.

 

[shadohh]

I am on the wagon with you.

This move from Apple is another nail in the coffin for FLASH. This is aimed at browsers like Firefox and Chrome to offer accelerated HTML5 video decoding.

Everyone who is saying flash works well on windows is right and wrong. Flash was horrible before 10.1. I believe the only reason Adobe is making the effort to improve flash with optimized coding and gpu accel is because of the pressure Apple put on them.

Ask yourself....Would flash 10.1 be good if HTML5/Apple wasnt around to punch Adobe in the kidneys?

It does not matter if flash dies or co-exists with HTML5. You can thank Apple for killing it or indirectly providing us with a better flash client.

Am I the only one that understand that the same thing can be applied to browsers like Safari, Chrome and Firefor for hardware-accelerated HTML5 video?

It's not just Flash that's important guys ;P

 

[Reed Rothchild]

Wow. Just wow.


Apple is the king of spin.They had everyone thinking that it was Adobe's fault that Flash sucked on OSX.


It's a bummer that Apple is forcing me to lose some love for them.

Not at all. It was well known that Apple was limiting performance of Flash by not exposing the API's. I and I'm sure many others have posted on this before whenever a Flash hater moaned about how slow the plugin was. Hopefully now we can get better Flash performance on our Macs and the haters can focus on other reasons to hate Flash outside of performance .

 

[nodezero]

It seems a very limited set of GPUs this works on right now. No ATi, even the Radeon 4850 from the latest iMacs? Only 3 GeForce types? Is this likely to expand? Not to seem ungrateful here.

 

[Mr. Retrofire]

And I think you're utterly clueless.

No, i use that hardware and i know how to use wikipedia.

http://en.wikipedia.org/wiki/MacBook_Pro#Technical_specifications_2

 

[iBug2]

You clearly don't know what Open CL is. Open CL is doing math calculation with the GPU, anything picture related as in 2d or 3d is ran in the video cards native mode not using the opencl. Windows has had 2d acceleration dated back to back in the NT days, like 1993. Apple used to have some in OS9 but since osx its been gone.

And you obviously have no idea that H.264 deblocking is completely a mathematical task, that's why it takes tons of CPU cycles to do it.

Hence, stream processors in the GPU can be used to do the same job via Open CL. There are already plans to use Open CL for the X.264 project, which will basically cover pretty much all the video players on OS X. But it's still in baby steps.

 

[iBug2]

OpenCL is not for video acceleration. it is for doing large calculations. General purpose calculation on the CPU. The acceleration adobe needs should be available but isn't because apple has cruddy video drivers. The same open gl games run like crap on OS x but on the same machine run smooth on windows. Feature like accelerated painted of overlays that cards have supported for quite some time just aren't available on apples drivers.

Again, nonsense. It has nothing to do with video drivers but with the H.264 acceleration API Apple just allowed. Open CL can be used to do the exact same job. Whenever you get confused in the future, think of this "GPU can pretty much do everything CPU can". That's how Nvidia Tesla Stream Computers work. They don't have any CPU inside, only GPU. But they can do everything a regular computer can.

You also have no clue about why games run faster on windows. Windows does not use Open GL in the first place (as default), it uses Direct3D for games. You can run certain games using Open GL, in which case, they will run a bit slower as well.

And game performance will never be equal on windows and OS X. Microsoft spends tons of money for Direct3D development, while Nvidia and ATI spend tons of money for driver development on Windows and Windows only. Because the demand is on Windows, not on OS X.

 

[iBug2]

To all the fanboys. May I suggest you READ the actual article first before jumping once again to conclusions.

Apple Opens Door means the door was closed before by Apple not Adobe.

The move by Apple allows Adobe to accelerate Flash.

Apple's refusal to allow access to APIs until now.

Apple's offer not Adobe's. Flash performance on Windows is better because the PCs are generally higher spec than Macs, they allow access to APIs required, 90% of the world use Windows pcs, and PCs are expected to use the CPU and not cry about it.

The problem is that most issues people have with flash have nothing to do with H.264 acceleration.

The only API missing for Adobe was now allowed by Apple. But that's only going to GPU accelerate a certain codec. It has nothing to do with the overall sloppiness of Flash, which is totally on Adobe's shoulders.

Linux is an open source OS, developers have access to everything. Yet Flash on Linux runs even worse than on OS X.

It's about Adobe not giving the effort to improve flash on low marketshare systems.

It is and always was Adobe's fault.

Not to mention the windows hardware acceleration of flash is "coming soon" on 10.1 as well. So currently, no OS has that. Not just OS X. Seriously, how lazy are you? This has been discussed ten times on this thread, you still post without reading anything.

 

[Mr. Retrofire]

Great! Now Apple has to allow Adobe access to its DirectX/GDI APIs so that vector animation won't suck eith- ...WAIT!

I think you mean the Shader Model support and the OpenGL APIs support in Mac OS X, because DirectX/GDI are Windows-based APIs.

 

[iBug2]

I think you mean the Shader Model support and the OpenGL APIs support in Mac OS X, because DirectX/GDI are Windows-based APIs.

I don't know how much that'd help. For all I know, Flash already uses some OpenGL since 2008 to accelerate certain functions on some of the content. (Legacy content don't get acceleration, so the content has to be written to take advantage of it as well)

 

[iSheep]

You can thank Apple for killing it or indirectly providing us with a better flash client.

You're pricelss!

 

[ranReloaded]

I think you mean the Shader Model support and the OpenGL APIs support in Mac OS X, because DirectX/GDI are Windows-based APIs.

I (jokingly) meant that again, Adobe expects to get away with a lazy port of their Windows codebase.

 

[iBug2]

So to sum things up (so certain people don't get confused or misinform others)

1. Apple has been quite slow to provide H.264 acceleration API's to third party developers, hence most of the player software on Mac still cannot accelerate H.264 through the GPU. This includes the likes of Mplayer, VLC, Plex, Flash, etc.
Windows have had that support for about a year now and Linux has partial support.



2. This has absolutely nothing to do with the terrible performance of Flash on Mac vs Windows, because there isn't H.264 acceleration on Flash for Windows right now either unless you are using 10.1 "beta". Same with Linux.
So Adobe has been quite late with incorporating H.264 acceleration into flash even though they could do it much earlier on Windows and Linux, where the API's have been available for a long time now.


3. This API Apple now provides won't speed up flash's vector animation, which most people have issues with to begin with. The sloppiness of websites made with flash won't change. That might change a bit with Adobe's recent efforts to improve flash, which initiated from the opportunity to port it to mobile platforms, which cannot handle flash at the moment due to phones having quite low CPU power compared to laptops/desktops. Hence the whole 10.1 development (the biggest update on history of flash). 10.1 will add vector acceleration on mobile phones, h.264 acceleration on mobile devices and desktop devices, and many other improvements.

4. For flash to perform as good as Windows on OS X, Adobe has to do much more than to incorporate Apple's new API. I have been testing 10.1 on my Mac and so far I'm not impressed. The CPU usage seem to be slightly less, but the general sloppiness of flash websites is still there.

 

[habika13]

Reply

This Apple tv supercharger product will be useful for every one and it is some thing new for us
===========================================================
apple iphone

 

[Anuba]

I hate flash because it messes up the internet experience. I hate websites done with flash. You spend 10 times more time to do the exact same thing you'd do on a regular HTML site because you need to watch all those small animations on every single click.

Right, and how many sites do that? Pretty much everyone uses Ajax for fancying things up beyond plain HTML. I only see Flash used for the main content on fashion sites and car sites, so unless you spend all day at calvinklein.com I'm not sure how you can run into so much Flash stuff.

Flash can be used for a plethora of things such as games and e-learning. It's great for things like interactive software tutorials that require a combo of voiceover + clickable interfaces + navigation/transport (not possible with video only, and not possible with current HTML5/Canvas tools), and it's also great for things like animated product trailers because you get pristine, uncompressed text and images using very little bandwidth. And it's scalable thanks to being vectorized. You can squeeze a 2-minute trailer into 2-3 MB with full image quality and good MP3 audio. If you were to export that content as an HD 1080p video without losing quality, it would be 150 MB like those HD movie trailers in QT format. End users are happy for only having to d/l 3 MB instead of 150, and content providers are VERY happy because bandwidth costs money, and the difference between 100,000 people downloading 3 MB vs. 150 MB, from the provider's POV, is 300 GB traffic vs 15 TB.

I'd also hate HTML5 if they used it to put the exact kind of ******** into websites.

They will.

I'd be celebrating if no website looked any more fancier than amazon.com. Web is supposed to be about efficiency. Getting things done.

Why not go back to Netscape 1.0 while you're at it? Center-aligned Times New Roman on grey background, blue and purple hyperlinks. It was great for "getting things done", only problem is you needed a bucket to puke in after looking at something that ugly.

 

[Headrush69]

OMG, some of you are so funny.

This is all Apple's fault because they never gave Adobe low level hardware access so they could HIDE/MASK the problem.

You people just don't get it.
Why should I need hardware acceleration just to watch a video when ALL other apps can handle it without?

 

[inlovewithi]

OMG, some of you are so funny.


You people just don't get it.
Why should I need hardware acceleration just to watch a video when ALL other apps can handle it without?

I don't think you understand how it all works. Things are not that simple. On windows when I play a flash video all the work is done on the CPU, when I play it of my computer the work is done on the CPU and GPU. Hardware acceleration simply means that it's going to play the video the same way it plays when you play it from your computer. In other words, you might not know this, but when I play a video of my computer, it's hardware accelerated. Without it, it would require as much CPU power as a flash video.

PS, I kept using Windows, and referring to myself playing it on Windows, because from what I'm hearing things might be different on OSX. Not sure if OSX uses just the CPU to play videos, from what I've read on this thread. Else I would have said, "When you play a video on your computer, it uses hardware acceleration..."

 

[Anuba]

OMG, some of you are so funny.


You people just don't get it.
Why should I need hardware acceleration just to watch a video when ALL other apps can handle it without?

When the GPU and CPU share the workload the video flows better, the computer remains more snappy and doesn't get too hot, so you avoid the fans revving up to jet engine mode.

If you think dumping all the workload on the CPU is a good idea, you ought to have loved Windows XP and every Windows version before it, since they utilized an ancient graphics engine that hogged the CPU while the powerful/expensive GPU was just sitting there twiddling its thumbs. It was horribly jerky and sluggish, and you'd often have to sit by and watch while the computer was painfully redrawing windows after maximizing or moving them.

Flash before and after hardware acceleration is like Mac desktop graphics before and after the Quartz engine. Flash has to draw stuff MacOS 9/WinXP style when it can't utilize hardware acceleration.

 

[IlluminatedSage]

I hope the fanboys take note of this. Flash sucks on OSX because of Apple. I've used it on Windows for years, and it runs perfectly. Hopefully this will go to alleviate some of the differences.

umm no, Adobe is widely known in developer circles to have skimped on their Mac development and software for Flash. sorry, but their version is very buggy and problematic.

that said, they are finally trying to get better, which is only a good thing

 

[NickK1066]

The ffmpeg accelleration code for H.264 is basically to provide the compressed frame to the API to decode. There's nothing stopping either a slow software implementation or OpenCL implementation being slotted in behind the API.

However as others have stated - this is only a decoder. It takes a compressed H.264 frame and decompresses it. It doesn't do anything todo with vector or drawing, so if Adobe don't use the Cocoa operations that are hardware accelerated then I'm not surprised that they're turning in a slow performance.

 

[CaptRoddick]

Read it again FLASH sucks because of Adobe "a deficiency that Adobe has noted it is working to address" get it right or go play with your Dell

ReallyOldGuys are cranky...

 

[diamond.g]

So to sum things up (so certain people don't get confused or misinform others)

1. Apple has been quite slow to provide H.264 acceleration API's to third party developers, hence most of the player software on Mac still cannot accelerate H.264 through the GPU. This includes the likes of Mplayer, VLC, Plex, Flash, etc.
Windows have had that support for about a year now and Linux has partial support.

I am confused, I thought Windows had DXVA support since XP? Where hardware acceleration is purely dependent on the hardware/drivers used. Is that not the case?

 

[gnasher729]

Bloody flash

The reason for this post: While I had the front page of MacRumors open, there was this Flash animation running. Takes between 60% and 90% of the CPU all the time. The size of that animation is 720 x 90 pixels. That is 3.125% of 1080p video. The animation has actually no movement in it at all; it would be perfect for h.264 because 90% of the image doesn't change from one frame to the next. It would work beautifully as an animated GIF with 90's technology.

And anyone tries to tell me that Flash uses 60% to 90% of my CPU for this tiny image _because nasty Apple isn't giving them hardware-accelerated h.264 video_?

And you obviously have no idea that H.264 deblocking is completely a mathematical task, that's why it takes tons of CPU cycles to do it.

Hence, stream processors in the GPU can be used to do the same job via Open CL. There are already plans to use Open CL for the X.264 project, which will basically cover pretty much all the video players on OS X. But it's still in baby steps.

Sorry mate, but you don't have a clue. Deblocking specifically is defined by the h.264 standard in an awful, awful way that makes it impossible to run in parallel over the whole image using general programming tools like Cuda or Open CL; it has to be done in a 16x16 pixel block four stripes vertical, followed by four stripes horizontal (or the other way round, can't remember), then proceeding to the next block. No shortcuts possible because they will lead to very visible artefacts. And its right in the middle of the performance critical path. No chance to use general hardware in any meaningful way. On the other hand, the task is trivial enough so that specialised hardware doing just h.264 deblocking and nothing else fits easily on a tenth of a square millimetre on a chip.

The only bit where OpenCL _might_ be useful is for brute force motion estimation when you are compressing, and that is useless for playback.

On one hand, it means that AIt's also disappointing that they are only enabling H.264 acceleration on the cards, and not MPEG2 or VC-1. I know Apple favours H.264, but the other codecs are still widely used.

I don't know about VC-1, but MPEG2 is so much simpler than h.264, it works just fine without any hardware support. Also these cards probably have _complete_ support for h.264, that means you plug in the compressed data stream at one end and the uncompressed frames come out on the other hand, without any CPU usage. I don't think anyone does that for MPEG-2 (it was too hard five years ago when MPEG-2 was relevant, and today when it could be done nobody bothers anymore).

 

[Alkiera]

Another vote for this not affecting the general performance problems with Flash on Macs. It's all about flash, not video. I tan take the same .flv video, play it in VLC, (or RealPlayer, for that matter. Think about that... REALPLAYER is more efficient than Flash?), and use like 10% of my RevA MBA's cpu... play that same video using Flash in Safari, and it uses 95% of CPU. The video isn't the problem, Flash being a CPU hog on MacOS is the problem.