Apple Product Managers Discuss Cool New iPadOS 18 Features

[MacRumors]

With iPadOS 18, Apple added some useful iPad-specific features in the Notes app, including Smart Script and Math Notes. Both of these additions are designed to work with the Apple Pencil, and they're some of the neater new features on Apple's tablets. Apple product manager Ty Jordan and engineering manager Jenny Chen recently did an interview with Chris Lawley to discuss the work that went into the iPadOS 18 updates.

Smart Script is one of the ways that Apple is using machine learning in iPadOS 18. It refines handwritten text, making your handwriting look neater without losing the personal flare that makes it your handwriting rather than generic writing. Smart Script learns your handwriting style, which Jordan said makes it as useful as typed text.

You'll notice that as soon as you start writing words, it refines your handwriting. You'll notice that it spell checks, so if you make a spelling mistake, it'll be underlined. It can also do some really cool things with reflowing your text, so if you tap and hold your Pencil, you can move words around, and you'll see that it reflows in your paragraph so you can get some more space.

You can even paste typed text as your own handwritten text right in the middle of a paragraph, so a lot of things that we take for granted with typed text, you can now do with handwritten text.

Smart Script learns continually and while it is designed to work quickly, it will get better over time as you write more, especially for features such as paste as handwriting. Recognition does not persist from note to note, however. Apple wanted the feature to feel personal and not like a font, which is why it refines dynamically with each note.

It's very contextual. So, for example, if you were to go to a previous paragraph where you had some band handwriting, we're going to match the handwriting that was refined there and not sort of try to replicate what you've had in other notes.

As for Math Notes, Jordan said that Apple took the time to "really reimagine" the calculator feature could be designed for the iPad's interface. If you're unfamiliar with the feature, Math Notes on iPad lets you write out equations with the Apple Pencil, with the Notes app solving the equation automatically when an equals sign is added.

It sort of works like magic. It combines the natural input of a pencil with the amazing on-device machine learning models that Jenny and her teams work on. You just write math like on a piece of paper and like magic, it just gives you the answer. It's so simple and intuitive.

Math Notes is able to recognize variables and the numbers associated with them. So if you define variables like price and area in one part of the note, and then write "price x area," Math Notes is able to provide an answer using the numbers specified earlier. Jordan explains:

I'll do my best shot and go back to high school. So a variable is just a named value, right? So you probably know a lot of them from school, like X and Y, and you could declare a variable with that [...] but you can do this with any word that you want.

I've personally loved using it for natural experiences, where I might declare like drinks equal something and food equals something else, and then I can just say that plus that and divided by the number of people, and I get an answer. So it just creates this very natural way to work with your math.

Math Notes is intelligent enough to understand the difference between separate variables in a note, so if you have multiple math problems with the same variable like X, it won't get confused. You can have graphs added automatically in Math Notes if an equation can be grafted, which Chen says uses the work that Apple has previously done for the Swift Charts API.

If you're interested in more context on the new Notes features in iPadOS 18, the full video is well worth watching. Both Math Notes and Smart Script are available in the iPadOS 18 beta, which developers and public beta testers can download. The new features will launch in the fall.

Article Link: Apple Product Managers Discuss Cool New iPadOS 18 Features

 

[Apple Knowledge Navigator]

Math Notes is one of the few genuinely ‘only Apple’ experiences that I’ve seen in a long time. It really does add a natural and human element to something that ordinarily doesn’t always fit right in a workflow, which is switching back and forth with a calculator. Good job!

 

[now i see it]

but I thought the whole point of bringing personal computers to the masses was so we didn’t have to use math
(Kidding)

 

[contacos]

I am personally glad I didn’t need much math since graduating and I’ll leave it at that. Thank you 😅

 

[MikeSmoke]

It would be nice to see some of this spill over to music notation software.

 

[Simmias]

I owned both Apple Pencil 1 and Apple Pencil 2 and never found a use for them. Now that I’ve upgraded to an M4 iPad Pro, I have no intention of buying another pencil. These new features look cool, but don’t change my mind on that.

As far as handwriting, typing is just a better and faster method of text input. As far as making diagrams or charts, this is easier and neater using shapes in Keynote or PPT. As far as math, apps like Soulver limit the need for handwritten equations.

I’m happy for visual artists or people who still like the idea of handwritten notes for nostalgic or aesthetic reasons, but I personally struggle to see the utility of Apple Pencil outside a few small niches. And Steve Job‘s comment on the downsides of styluses (“you have to get it and put it away…yuck”) still apply. I often lost my pencil when it detached from my iPad and spent a long time rummaging in bags and drawers to find it.

 

[Someyoungguy]

Best clipboard ever! Computer replacement? Nope.

 

[terminator-jq]

There's a few cool features but at the end of the day, we are just over 3 years into iPads having desktop class M chips... The current iPad Pro literally introduced the M4 generation. Seeing the slow growth of iPad OS was disappointing after the introduction of the M1 iPad. It was even more annoying after the M2 iPad Pro (not to mention the M1 making it to the Air). At this point the [Hardware Power : OS Capabilities] ratio has gotten ridiculous.

iPad OS needs a serious overhaul. They need to unlink iPad OS from iOS in a bigger way (especially since Vision OS is based on iPad OS which impacts yet another system). With as much talent as Apple has, I really don't understand how they haven't found more ways to make the iPad more of a computer and less of a giant iPhone. We don't need MAC OS on iPad but we DO need something that gets us halfway there. A few things I'd like to see:

1. iPad needs to have the same freedom Mac OS has of being able to download and install software both from the App Store as well as from 3rd party sources.

2. We need to see more Pro apps from Apple. There's no reason Xcode shouldn't have an iPad version. iPad pros now cost $999 for the base model! Easily in laptop territory. For those prices, people need the ability to get real work done and make that investment back.

3. A longshot but it would be awesome if Apple had a "Rosetta" style system for M chip iPads that allowed you to run MAC OS software that was built for AS. Obviously most software would only be usable with a connected keyboard & mouse. Again, complete long shot but it sure would make the iPad more of a complete computing solution. A program like Blender will almost certainly never get a dedicated iPad version but performance wise, iPads with M1 or newer are capable.

These types of updates would also make the power difference between iPad models more crucial. Currently, there's very few apps that actually push the M4 enough to notice a difference over the M2.

 

[dandy1117]

I owned both Apple Pencil 1 and Apple Pencil 2 and never found a use for them. Now that I’ve upgraded to an M4 iPad Pro, I have no intention of buying another pencil. These new features look cool, but don’t change my mind on that.

As far as handwriting, typing is just a better and faster method of text input. As far as making diagrams or charts, this is easier and neater using shapes in Keynote or PPT. As far as math, apps like Soulver limit the need for handwritten equations.

I’m happy for visual artists or people who still like the idea of handwritten notes for nostalgic or aesthetic reasons, but I personally struggle to see the utility of Apple Pencil outside a few small niches. And Steve Job‘s comment on the downsides of styluses (“you have to get it and put it away…yuck”) still apply. I often lost my pencil when it detached from my iPad and spent a long time rummaging in bags and drawers to find it.

Let me summarize your comment using AI: "There isn't significant value in using the Apple Pencil for my purposes."

I understand your perspective, but I believe you may be overlooking the numerous specialized applications for iPad users beyond nostalgic visual artists, where handwriting input with a pencil is much more versatile and effective than using a keyboard. This includes tasks such as annotating scanned documents, creating architectural illustrations, illustrating patent diagrams, and many other uses.

 

[fontman]

OK OK let's get back to work on this product and get it out the door is clean as possible stop talking and start doing the ghost of Steve Jobs would say

 

[zachz]

I owned both Apple Pencil 1 and Apple Pencil 2 and never found a use for them. Now that I’ve upgraded to an M4 iPad Pro, I have no intention of buying another pencil. These new features look cool, but don’t change my mind on that.

As far as handwriting, typing is just a better and faster method of text input. As far as making diagrams or charts, this is easier and neater using shapes in Keynote or PPT. As far as math, apps like Soulver limit the need for handwritten equations.

I’m happy for visual artists or people who still like the idea of handwritten notes for nostalgic or aesthetic reasons, but I personally struggle to see the utility of Apple Pencil outside a few small niches. And Steve Job‘s comment on the downsides of styluses (“you have to get it and put it away…yuck”) still apply. I often lost my pencil when it detached from my iPad and spent a long time rummaging in bags and drawers to find it.

As someone with 0 artistic ability and above average typing speed, the Apple Pencil is great for handwriting notes not for aesthetic purposes (mine looked like garbage, worse than on paper actually) and not for nostalgic purposes (I’m too young to get nostalgic over writing), but for having a digital notebook to organize written notes. While in school I’ve tried typing notes but it’s far too easy to just start transcribing what is being said, whereas handwriting helped me understand what was being said and retain the information.

 

[bradman83]

There's a few cool features but at the end of the day, we are just over 3 years into iPads having desktop class M chips... The current iPad Pro literally introduced the M4 generation. Seeing the slow growth of iPad OS was disappointing after the introduction of the M1 iPad. It was even more annoying after the M2 iPad Pro (not to mention the M1 making it to the Air). At this point the [Hardware Power : OS Capabilities] ratio has gotten ridiculous.

I've said this many times before it but bears repeating - there is absolutely nothing special about Apple putting the M-series in the iPad lineup. The base M chip is essentially the same chip that the X variants used to be; the M1 is scaled up from the A14 almost exactly how the A12X/Z was from the A12. It's not worth Apple's while to tape out an entirely separate X variant A-chip to strip out some of the I/O and Intel memory compatibility goodies they added with that revision.

Put another way, it's not so much that Apple put desktop class chips into a tablet as it is that they scaled their tablet chips to be desktop class.

1. iPad needs to have the same freedom Mac OS has of being able to download and install software both from the App Store as well as from 3rd party sources.

Apple's current management will never allow this. Apple wants their cut of every software purchase made on their non-Mac platforms. Services revenue, which includes App Store purchases, is a huge part of Apple's revenue stream. Heck if Apple could lock down macOS to only App Store apps and not lose massive amounts of developer support or attract regulatory attention they probably would. They would have to be forced, kicking and screaming, by regulators to allow for sideloading.

3. A longshot but it would be awesome if Apple had a "Rosetta" style system for M chip iPads that allowed you to run MAC OS software that was built for AS.

Technically this is what Catalyst is for, the back end of an app can be built once with separate interfaces tailored for macOS versus iPad (or even the same interface). macOS and iPadOS share significant amounts of their codebase despite how different the OS's are superficially. Given the touch-first nature of iPadOS I don't see Apple giving too much priority to software that requires a keyboard and pointing device.

 

[stevemiller]

in fairness i'm reacting more to the headline than the content here - but "product managers discuss..." just gives the exact middle-management-driven-product-design energy that has made me tap out on apple stuff lately.

 

[koelsh]

Not to be too cynical (and I'm also reacting more to the headline than the content) but does this smack of desperation to anyone else? Like they see that iPadOS doesn't have the general sentiment adn momentum the Mac has built up in recent years so they feel the need to hype up what little things they have done.

"We're deliberately ignoring the things people need and want iPad to do but check out this other way we made a thing to do kid's schoolwork for them."

 

[gumbyhw]

Getting closer. Math notes needs a way to grab the result of a calculation and use it in another one: circle the result and draw an arrow. Then they should buy mathmatica and integrate it!

As for the handwriting: I take all my notes in handwriting in Notes. Much faster than typing (and I'm a heavy "typer" on the mac -- emacs user). But one of the advantages of handwriting is that you can write in different sections of the page, or write in mixed landscape / portrait, or draw a diagram and write text next to things. Notes still has too rigid a view of where text goes.

Also Notes search and organization is terrible. Not bad, not inadequate, but actually terrible.

But everthing discussed here is a step in the right direction.

 

[gusping]

Cool features are all well and good, but how about some useful features that move the iPad on from being a consumption device. Anyone....?

 

[videosoul]

It would be nice to see some of this spill over to music notation software.

That would be HUGE.

 

[Octavius8]

As a demo is nice. As a truly useful feature for most of us, I doubt it.
We need an easier way to organize and find notes and info in them. And to create documents with charts, tables, text, drawings, etc. like a webpage, but for taking notes.

 

[Apple_Robert]

Cool features are all well and good, but how about some useful features that move the iPad on from being a consumption device. Anyone....?

For me and many others, the iPad is a computer that is used to consume content as well get work done. Just because one can’t do all one expects on an iPad, that doesn’t make it a device solely for content consumption. When many here talk about using their MBP or MBA to do the same kinds of things as many others do on the iPad, the MBA and MBP devices do not get labeled as consumption devices as is the case with the iPad. There is a lot of selective bias on MR.

 

[Apple_Robert]

I would have liked to have seen the GUI ability to turn off (and reboot) the iPad, instead of having to press buttons. Said feature comes in handy for me on a Mac, especially when using Screens or JumpDesktop. If I am away from home and the weather is really bad, it would be nice to be able to do a remote shut down.

 

[Dave-Z]

Apple took the time to "really reimagine" the calculator feature could be designed for the iPad's interface

"Reimagine"? Really? I think that's a bit of a stretch.

 

[anthogag]

Smart Script and Math Notes are very good.

When I write in Notes I see my writing change. My hand writing is good and Smart Script makes it smoother. Notes keeps getting better and better. It would be nice if Notes had layers like Procreate.

Math Notes is beautiful to use for calculations in Notes. It is really great to be able to use the Pencil and write-out math stuff.

It is true, hand writing is better for learning compared to typing on a keyboard.

There are studies that showed using a keyboard and typing is for zombies. Basically, typing on a keyboard is brainless and very unsatisfying for exercising creativity.

 

[constantsnags!]

I owned both Apple Pencil 1 and Apple Pencil 2 and never found a use for them. Now that I’ve upgraded to an M4 iPad Pro, I have no intention of buying another pencil. These new features look cool, but don’t change my mind on that.

As far as handwriting, typing is just a better and faster method of text input. As far as making diagrams or charts, this is easier and neater using shapes in Keynote or PPT. As far as math, apps like Soulver limit the need for handwritten equations.

I’m happy for visual artists or people who still like the idea of handwritten notes for nostalgic or aesthetic reasons, but I personally struggle to see the utility of Apple Pencil outside a few small niches. And Steve Job‘s comment on the downsides of styluses (“you have to get it and put it away…yuck”) still apply. I often lost my pencil when it detached from my iPad and spent a long time rummaging in bags and drawers to find it.

Some people retain and process information far more effectively when they hand-write it - myself being one of them. Typing might be a better input method for you, or in particular use cases...but it is by no means universally "better".

 

[meaty]

There's a few cool features but at the end of the day, we are just over 3 years into iPads having desktop class M chips... The current iPad Pro literally introduced the M4 generation. Seeing the slow growth of iPad OS was disappointing after the introduction of the M1 iPad. It was even more annoying after the M2 iPad Pro (not to mention the M1 making it to the Air). At this point the [Hardware Power : OS Capabilities] ratio has gotten ridiculous.

iPad OS needs a serious overhaul. They need to unlink iPad OS from iOS in a bigger way (especially since Vision OS is based on iPad OS which impacts yet another system). With as much talent as Apple has, I really don't understand how they haven't found more ways to make the iPad more of a computer and less of a giant iPhone. We don't need MAC OS on iPad but we DO need something that gets us halfway there. A few things I'd like to see:

1. iPad needs to have the same freedom Mac OS has of being able to download and install software both from the App Store as well as from 3rd party sources.

2. We need to see more Pro apps from Apple. There's no reason Xcode shouldn't have an iPad version. iPad pros now cost $999 for the base model! Easily in laptop territory. For those prices, people need the ability to get real work done and make that investment back.

3. A longshot but it would be awesome if Apple had a "Rosetta" style system for M chip iPads that allowed you to run MAC OS software that was built for AS. Obviously most software would only be usable with a connected keyboard & mouse. Again, complete long shot but it sure would make the iPad more of a complete computing solution. A program like Blender will almost certainly never get a dedicated iPad version but performance wise, iPads with M1 or newer are capable.

These types of updates would also make the power difference between iPad models more crucial. Currently, there's very few apps that actually push the M4 enough to notice a difference over th

There's a few cool features but at the end of the day, we are just over 3 years into iPads having desktop class M chips... The current iPad Pro literally introduced the M4 generation. Seeing the slow growth of iPad OS was disappointing after the introduction of the M1 iPad. It was even more annoying after the M2 iPad Pro (not to mention the M1 making it to the Air). At this point the [Hardware Power : OS Capabilities] ratio has gotten ridiculous.

iPad OS needs a serious overhaul. They need to unlink iPad OS from iOS in a bigger way (especially since Vision OS is based on iPad OS which impacts yet another system). With as much talent as Apple has, I really don't understand how they haven't found more ways to make the iPad more of a computer and less of a giant iPhone. We don't need MAC OS on iPad but we DO need something that gets us halfway there. A few things I'd like to see:

1. iPad needs to have the same freedom Mac OS has of being able to download and install software both from the App Store as well as from 3rd party sources.

2. We need to see more Pro apps from Apple. There's no reason Xcode shouldn't have an iPad version. iPad pros now cost $999 for the base model! Easily in laptop territory. For those prices, people need the ability to get real work done and make that investment back.

3. A longshot but it would be awesome if Apple had a "Rosetta" style system for M chip iPads that allowed you to run MAC OS software that was built for AS. Obviously most software would only be usable with a connected keyboard & mouse. Again, complete long shot but it sure would make the iPad more of a complete computing solution. A program like Blender will almost certainly never get a dedicated iPad version but performance wise, iPads with M1 or newer are capable.

These types of updates would also make the power difference between iPad models more crucial. Currently, there's very few apps that actually push the M4 enough to notice a difference over the M2.

You’re missing the point. It’s about selling Macs and iPads.

 

[Rradcircless]

So she’s the person responsible for the absolutely atrocious caddy and markup experience. They need a huge overhaul.