iOS 14 Features New OCR Capabilities That Will Convert Handwritten Text From Apple Pencil Into Typed Text

[MacRumors]

iOS 14 may include a new PencilKit feature that will allow you to handwrite text in any text input field using Apple Pencil, with the handwritten content then converted into standard text before it's sent. So, for example, you might be able to tap into a Messages text field with the Apple Pencil, hand write a message to someone, have it automatically converted to a more easily readable typewritten text, and then send it.

According to information obtained by MacRumors, the PencilKit feature appears to work with any text input field available in iOS, making it compatible with Messages, Notes, Reminders, Calendar, Mail, and more. A floating interface will pop up whenever a text field is tapped with the pencil, allowing for the written input.

Apple does not have a feature at this time that converts handwritten content into typed content, but the Notes app does have functionality that recognizes handwritten words and allows them to be searched.

It looks like third-party apps will also be able to access a PencilKit tool that will support some kind of new handwriting functionality, but the extent of it is unclear.

It is not known if this is a finalized feature that is going to make it into iOS 14. Rumors have indicated Apple is taking a new approach to iOS development that will allow certain features to be toggled off and sidelined if they are not ready for launch in order to cut down on bugs.

Along with the handwriting-to-text feature, Apple also appears to be working on a shape-drawing function that supports a "Magic Fill" feature, allowing users to draw a general shape with the Apple Pencil that's then filled in by iOS.

We'll be sharing more details on what we can expect to see in iOS 14, with our iOS 14 roundup serving as a hub for everything that's been found so far.

Article Link: iOS 14 Features New OCR Capabilities That Will Convert Handwritten Text From Apple Pencil Into Typed Text

 

[OriginalMacRat]

Newton is back baby!!

 

[nicho]

Great! I currently use Mazec, but it's not 100% ideal because at work I prefer a logitech crayon for scribbling observations/notes, so the "reject stuff not written by a pencil" settings don't work.

 

[cmaier]

If only my handwriting wasn’t complete ****

 

[canesalato]

Newton is back baby!!

has been back for a few years, on windows tablets. And the implementation is EXACTLY as described in the article. =)

 

[rhyzome]

Well it is about time!

(especially since this was in the Apple Watch forever and in Apple’s original tablet decades ago!)

 

[farewelwilliams]

Finally, the appropriate scenario for the use of the joke "Steve rolling in his grave"

 

[jmgregory1]

The more things change...

 

[alien3dx]

nebo takeover.. hehehe.. or copyright

Nebo

Where ideas take shape

www.nebo.app

 

[jpn]

this may well be the feature that, along with full trackpad support, is able to not just prompt me to finally think i can try to use an iPad for my work, but actually might be more useful than a keyboarded-MacBook.

 

[nicho]

nebo takeover.. hehehe.. or copyright

Nebo

Where ideas take shape

www.nebo.app

Is nebo able to be used anywhere in the OS now?

I’d say it’s more a competitor to/copy of mazec, a Japanese app that does pretty much literally what this is supposed to do. Interestingly, updates to Mazec have been very few and far between... perhaps their tech was bought under the radar.

 

[alien3dx]

Is nebo able to be used anywhere in the OS now?

I’d say it’s more a competitor to/copy of mazec, a Japanese app that does pretty much literally what this is supposed to do. Interestingly, updates to Mazec have been very few and far between... perhaps their tech was bought under the radar.

only in their app and it works perfectly. I'm using it with iPad mini 2019. I'm not sure even nebo can access other part of system to create a mini app . Let see either apple create a new one and have all bugs or buy out..

 

[cmaier]

Is nebo able to be used anywhere in the OS now?

I’d say it’s more a competitor to/copy of mazec, a Japanese app that does pretty much literally what this is supposed to do. Interestingly, updates to Mazec have been very few and far between... perhaps their tech was bought under the radar.

Nah. The sdk added some OCR stuff last year. This is the natural evolution.

 

[alien3dx]

Is nebo able to be used anywhere in the OS now?

I’d say it’s more a competitor to/copy of mazec, a Japanese app that does pretty much literally what this is supposed to do. Interestingly, updates to Mazec have been very few and far between... perhaps their tech was bought under the radar.

just saw in youtube, mazec also take algorithm from nebo @cmaier

 

[ThaGoochiestMane]

I'm curious to see what happens. Windows and OneNote have had this feature for years (the latter can even convert equations to type) but I remember having to concentrate for certain letters/numbers, as it couldn't recognize those in my penmanship.

Hopefully Apple's implementation will accommodate larger variations in handwriting, and given the wait I assume it will live up to their typical attention to detail.

 

[cmaier]

just saw in youtube, mazec also take algorithm from nebo @cmaier

Ok. My point is that apple doesn’t need it. They added a ton of machine learning sdks last year, including OCR. This is just more of the same, and tying it into the UITextEdit and other controls.

 

[alien3dx]

Ok. My point is that apple doesn’t need it. They added a ton of machine learning sdks last year, including OCR. This is just more of the same, and tying it into the UITextEdit and other controls.

i'm sure they need it, most developer would do by scratch but when salesman need something quick to show off. Buyout the fastest way.. rather waste time few year re-invent the wheel.

 

[cmaier]

i'm sure they need it, most developer would do by scratch but when salesman need something quick to show off. Buyout the fastest way.. rather waste time few year re-invent the wheel.

But they already have it. What, precisely, do they gain by buying what they already have? You trying to pump their stock or something?

 

[alien3dx]

But they already have it. What, precisely, do they gain by buying what they already have? You trying to pump their stock or something?

bugs , normal software company would push show of saying they have x y future but in the end how stable is it like now my waste of money buying camera adapter kit for usb purpose pen drive with ipad mini but only working on beta 4 ipad os not current stable version?

 

[konqerror]

Newton is back baby!!

has been back for a few years, on windows tablets. And the implementation is EXACTLY as described in the article. =)

As I understand it, Microsoft licensed their handwriting recognition from the same company that did the handwriting recognition on the Newton. It was originally bought for Windows CE/Pocket PC, and Microsoft later brought it to Windows when the Windows XP tablets came out.

 

[cmaier]

bugs , normal software company would push show of saying they have x y future but in the end how stable is it like now my waste of money buying camera adapter kit for usb purpose pen drive with ipad mini but only working on beta 4 ipad os not current stable version?

1) how about some punctuation? I have no idea what you are saying.
2) https://developer.apple.com/documentation/vision. They already spend massive amounts on this sort of problem. They aren’t going to buy somebody else’s solution.
3) are you Nebo’s mom or something? Your statements are very weird.

 

[nicho]

Nah. The sdk added some OCR stuff last year. This is the natural evolution.

Either way... it can't come quickly enough for me. Native text entry beats 3rd party solutions every time.

 

[cmaier]

Either way... it can't come quickly enough for me. Native text entry beats 3rd party solutions every time.

Yep. More importantly will be the ability of third parties to use it in places other than text boxes. Would be lovely to have smart ink in lots of places.

 

[alien3dx]

1) how about some punctuation? I have no idea what you are saying.
2) https://developer.apple.com/documentation/vision. They already spend massive amounts on this sort of problem. They aren’t going to buy somebody else’s solution.
3) are you Nebo’s mom or something? Your statements are very weird.

err. what. good luck with just pure link..

 

[cmaier]

err. what. good luck with just pure link..

The link is to the text recognition SDK’s they already have, today.