Apple Previews New Door Detection, Apple Watch Mirroring, and Live Captions Accessibility Features

[kc9hzn]

As others have mentioned, Google has moved it to be on-device only and they're moving more and more of their ML stuff onto their device with the Tensor chips.

Remember Apple did in fact upload all Siri voice usage to their service as well, which got them in trouble: https://www.macrumors.com/how-to/delete-siri-audio-history-opt-out-audio-sharing/ Apple isn't fully innocent here.

Apple just like Google has been trying to move stuff to be on-device with each generation of their Silicon adding more dedicated hardware support for this (neural engine).

Also, Microsoft is the same; on-device only.



Source: https://blogs.windows.com/windowsex...-accessibility-features-coming-to-windows-11/

To be fair, not every Android device has the Tensor chip, and it’ll be a long time until most Android devices have AI performance comparable to it. Every current non-Mac Apple device has a neural network accelerator (including every supported iPhone and iPad), and every Apple Silicon Mac has one, too. It makes me wonder how well these accessibility features work on the current batch of low end Android phones.

 

[MisterSavage]

I have been waiting for Live Captions in FaceTime for years--ever since they first had the technology in the Clips app. I watched as Live Captions got added to Teams and then Zoom during the pandemic. I currently start up a Zoom call and just don't dial anyone to get live captions for meetings and conversations in the car. I'm hoping the live captions on FaceTime work even better than that.

I hope the feature is easy to turn off and on, as it sounds like it can caption any on-device audio. I don't need live captions on content that is already subtitled, for example. It also remains to be seen what the in-person interface looks like.

As a deaf user that doesn't use ASL, this will be a godsend.

There is a way you can get captions on FaceTime right now. Check out Navi:

‎Navi - Subtitles & Translation

‎Navi adds subtitles and Live Translation to your FaceTime calls through SharePlay. Enable subtitles and see them on top of the FaceTime video window. The app opens FaceTime up to people with hearing impairments and other disabilities that prevent them from engaging easily in a video call...

apps.apple.com

 

[exoticSpice]

iPads

To be fair, Android has this Live Captions feature already as well as Google Chrome. I had to rely on it on all platforms.

Microsoft announced and is testing Live Captions on Windows 11 insider builds for a few months now.

Apple is late as usual but I’m sure they will be the best implemented one as that is just them.

Regardless, everyone wins here. We need more accessibility support across the industry.

Yep we all benefit. I want Android to add Door Detection too.

 

[MikhailT]

To be fair, not every Android device has the Tensor chip, and it’ll be a long time until most Android devices have AI performance comparable to it. Every current non-Mac Apple device has a neural network accelerator (including every supported iPhone and iPad), and every Apple Silicon Mac has one, too. It makes me wonder how well these accessibility features work on the current batch of low end Android phones.

Yes however in this case, Live Caption doesn't require Tensor SoC at all. The Tensor SoC has Google's tuned hardware to accelerate ML modals and to be more efficient. However, the said ML modals can run on any SoC that has ML accelerators including SnapDragon, Exynos, etc. It works on all Android 10 or later devices and it works within Chrome on any platforms without any ML chips as well.

In fact, if Android developers use Google's MLKit SDK, many models can even run on iOS as well; https://developers.google.com/ml-kit/guides

 

[svish]

Live captions will be a very useful feature.

 

[kc9hzn]

Yes however in this case, Live Caption doesn't require Tensor SoC at all. The Tensor SoC has Google's tuned hardware to accelerate ML modals and to be more efficient. However, the said ML modals can run on any SoC that has ML accelerators including SnapDragon, Exynos, etc. It works on all Android 10 or later devices and it works within Chrome on any platforms without any ML chips as well.

In fact, if Android developers use Google's MLKit SDK, many models can even run on iOS as well; https://developers.google.com/ml-kit/guides

Is ML acceleration a requirement on all Android 10 devices? I’m just curious because I basically never have used a high end Android SoC, all just cheap MediaTek chips in fairly cheap Android devices that I’ve purchased for hobbyist purposes.

 

[alpi123]

I think the difference is that Google does all processing on their servers, Apple's implementation is on-device only and works offline. (not to mention your conversation stays private)

That's not true, Android's Live Captions are processed locally as well. And probably Windows 11's too.
It doesn't make sense to require internet since this is universal captioning, not just for content on the internet.

 

[Android_IOS_GUY]

As others have mentioned, Google has moved it to be on-device only and they're moving more and more of their ML stuff onto their device with the Tensor chips.

Remember Apple did in fact upload all Siri voice usage to their service as well, which got them in trouble: https://www.macrumors.com/how-to/delete-siri-audio-history-opt-out-audio-sharing/ Apple isn't fully innocent here.

Apple just like Google has been trying to move stuff to be on-device with each generation of their Silicon adding more dedicated hardware support for this (neural engine).

Also, Microsoft is the same; on-device only.



Source: https://blogs.windows.com/windowsex...-accessibility-features-coming-to-windows-11/

Thank you.