Craig Federighi Says iPad's Cursor Experience Inspired by Apple TV

[MacRumors]

Apple software engineering chief Craig Federighi recently spoke with TechCrunch's Matthew Panzarino about the iPad's new cursor functionality, introduced alongside trackpad and mouse support in iPadOS 13.4.

Federighi said the iPad's new cursor experience was inspired by the Apple TV:

"There was a process to figure out exactly how various elements would work together," Federighi says. "We knew we wanted a very touch-centric cursor that was not conveying an unnecessary level of precision. We knew we had a focus experience similar to Apple TV that we could take advantage of in a delightful way. We knew that when dealing with text we wanted to provide a greater sense of feedback."

And:

"Part of what I love so much about what's happened with iPadOS is the way that we've drawn from so many sources. The experience draws from our work on tvOS, from years of work on the Mac, and from the origins of iPhone X and early iPad, creating something new that feels really natural for iPad."

When using a trackpad, the cursor displays as a circle on the screen, popping up only when you have a finger on the trackpad. The circle then morphs into various other shapes when interacting with various elements of iPadOS:

"So we came up with a circle that elegantly transforms to accomplish the task at hand. For example, it morphs to become the focus around a button, or to hop over to another button, or it morphs into something more precise when that makes sense, like the I-beam for text selection."

And:

"We set out to design the cursor in a way that retains the touch-first experience without fundamentally changing the UI," Federighi says. "So customers who may never use a trackpad with their iPad won't have to learn something new, while making it great for those who may switch back and forth between touch and trackpad."

MacRumors recently went hands-on with the cursor experience, providing a closer look at various trackpad gestures and more.

Subscribe to MacRumors on YouTube for more videos​

iPadOS 13.4 introduced the new cursor experience and trackpad support on all iPad Pro models, the iPad Air 2 and later, the fifth-generation iPad and later, and the iPad mini 4 and later. Keyboards with built-in trackpads released so far include Apple's Magic Keyboard, Logitech's Combo, and Brydge's Pro+.

Article Link: Craig Federighi Says iPad's Cursor Experience Inspired by Apple TV

 

[Joseph C]

It’s really great for the first stab, but iPadOS 14 hopefully will bring further refinements, as right click behaviour, selection and hover states aren’t as consistent as they should be.

In Reminders, I can’t select multiple Reminders with it. In Music, hovering over albums doesn’t give the Parallax effect. Date pickers and other elements need to be refined for trackpad/mouse and keyboard input. In Photos, right clicking a Photo doesn’t give much context for example to hide a photo I still have to go into the Share sheet window. Right clicking on selected text brings up a different type of limited old-style menu to what you get when, say, right clicking a song in Music or an App on the home screen.

There are more examples but I can’t think of them off the top of my head. Hopefully many more UI refinements are coming which will make the experience more consistent.

 

[thadoggfather]

Is the cursor on 13.4 really that good? I have MM2 as well

I haven't updated my Pro from 13.3 yet, I plan to eventually but.

 

[mi7chy]

Looks a lot like the Fire TV cursor.

 

[Aluminum213]

Is the cursor on 13.4 really that good? I have MM2 as well

I haven't updated my Pro from 13.3 yet, I plan to eventually but.

That good? It works like any cursor you’ve ever used, not sure what you’re asking

 

[thadoggfather]

That good? It works like any cursor you’ve ever used, not sure what you’re asking

Well they made it distinguished from the way it was pre-13.4, on iOS 13, and made a big fuss about it.

so that's what I'm asking about. Comments like these aren't helpful at all

 

[SAIRUS]

So Craig found out he has a team that already had a solution and reused it in another place...Steve would be pleased.

Okay before this turns into a Steve vs Tim post, I think the new solution is rather elegant, and a little harmony between the OSes isn’t a bad thing. tvOS has a great solution, and a little animation goes a long way to signify to the user something has happened. The original accessibility cursor prior to iOS 13.4 felt like a hit test with a cursor, and you had to react to a possible sucesss. Now an iteration later, seems like they got the solution fleshed out and we have proactive indicators via animations to achieve success.

So this post is about the attention to detail that finally made it out to the user. Now for my subtle Steve bias. Wish this feature came out like this. Now my Tim bias. Hopefully they iterate on this and bring this polish earlier than just put something out for the sake of it. I could see getting the functionality out to increase accessibility out as soon as possible is a good thing, and all for it, but coming out with polish is going to make users happy and make the company image look better.

 

[Even Longer]

On an iPad even a cursor has experience... Good times to be alive.

 

[D.T.]

It’s really great for the first stab, but iPadOS 14 hopefully will bring further refinements, as right click behaviour, selection and hover states aren’t as consistent as they should be.

In Reminders, I can’t select multiple Reminders with it. In Music, hovering over albums doesn’t give the Parallax effect. Date pickers and other elements need to be refined for trackpad/mouse and keyboard input. In Photos, right clicking a Photo doesn’t give much context for example to hide a photo I still have to go into the Share sheet window. Right clicking on selected text brings up a different type of limited old-style menu to what you get when, say, right clicking a song in Music or an App on the home screen.

There are more examples but I can’t think of them off the top of my head. Hopefully many more UI refinements are coming which will make the experience more consistent.

Terrific post. I agree, I think it's a solid foundation, it works well as an input alternative, and they've created a nice "cursor UX language" (for lack of better terms), so now with that in places, I'm hoping to see them extend it, including many of the suggestions above.

 

[Aluminum213]

Well they made it distinguished from the way it was pre-13.4, on iOS 13, and made a big fuss about it.

so that's what I'm asking about. Comments like these aren't helpful at all

It works like a cursor like you’ve used on a laptop and desktop

Not sure what the confusion is

 

[Feyl]

I’m still wondering why the first gen Magic Mouse can’t scroll on iPadOS. Somebody ask Craig about it.

 

[ikir]

It’s really great for the first stab, but iPadOS 14 hopefully will bring further refinements, as right click behaviour, selection and hover states aren’t as consistent as they should be.

Right click is already there.

 

[nvmls]

Wow Craig! Lots of innovation. Not really.

 

[Joseph C]

Right click is already there.

I wasn’t saying it wasn’t there; I was saying the right click behaviour is not consistent.

 

[urnotl33t]

tvOS has a great solution, and a little animation goes a long way to signify to the user something has happened.

Well, to be fair, my Siri remote is at a constant health risk of me throwing it off my balcony to the rocks below, because I despite that piece of trash. Can I disable that completely-useless PiP option that constantly gets in the way when I try to rewind or fast forward? How about making the rewind/FF actually operational and be consistent? Too often, I get the "(10)" in the circle but clicking does nothing... ugh. Then a slightest twitch makes the giant PIP option highlighted (which is barely noticeable, so I can't tell), then BAM... my enjoyment of the content is trash and I'm missing parts of the episode. Grrr.

I'm infinitely-close to throwing out that remote and just using my iPhone instead from now on. I *hate* that remote. I found my old ATV 2/3 stick but it didn't seem to work on my ATV4K. :-/

 

[bobob]

On an iPad even a cursor has experience... Good times to be alive.

Must they only ever expect to live a cursored life?

 

[pup]

My understanding is that the TVos was originally supposed to be more like the new iPad os in that the cursor was supposed to be visible and not snap instantly between selections. They'd do us all a favor by reverting to that behavior and it might make the Siri remote just a slight bit less annoying.

 

[jamezr]

Too bad they couldn't get inspired from something like well a mouse cursor

 

[mazz0]

Craig Federighi Says iPad's Cursor Experience Inspired by Apple TV

Well yeah - did anybody not notice that?

I’ve turned that feature off (Settings -> Accessibility -> Pointer Control -> Pointer Animations). I’m actually fine with the animations, it’s the cursor disappearing while they animate that’s the problem, it means you’re not 100% sure where it’ll appear when it comes out of the other side, which spoils the smooth flowing feeling of moving the cursor. It’s hard to describe, it sounds minor, but it’s very offputting to me. I hate the Apple TV interface too, but that’s because it’s so hard to swipe the right number of times at once. Delightful my arse.

Incidentaly, anyone else frustrated by the trackpad and keyboard settings being arbitrarily divided between General and Accessibility? Jeez, it’s like Windows 10!

 

[DeepIn2U]

TechCrunch ... quite late for this interview. Why talk about the iPadOS experience with the new Magic Case now after all the interviews, reviews and statements last month? Stop allowing Craig to grandstand, give us current details for newly released products such as the 13" MBP 2020 model - ask deep questions into that! Touch Bar why the insistence vs negative feedback from developers. Why the return of the Escape key only? Does he, Apple, foresee the Touch Bar going away like butterfly keyboard?

Ridiculous $700US pricing for optional wheels of the Mac Pro - I don't think Apple has EVER done anything as ridiculous as this in it's history.

 

[danmart]

Is the cursor on 13.4 really that good? I have MM2 as well

I haven't updated my Pro from 13.3 yet, I plan to eventually but.

Well, prior to this latest version of iPad OS the cursor was really intended as an accessibility aid, and all it gave you was a big, round cursor about the size of your fingertip.

The new version is intended more for general use as a choice rather than a necessity and so tried to add functionality. It includes a whole bunch of non-pointer stuff like multi-finger swipes to move between apps and so on. It also includes a level of visual language that it didn't previously have, and this is what Craig is referring to here. It doesn't directly effect how you use the pointer but it does enhance the experience like any good visual language does. I have to say that I was wondering for some time why they didn't use the ATV concept with pointers on iOS devices. It was the natural next step once the 'new' ATV interface was created using a touchpad.

It also brings more 'regular' features to the pointer on the iPad, like context sensitive behaviours.

So, in some ways it just brings 'regular' pointer behaviours to the iPad, but the iPad previously lacked those. In other ways it brings a new visual language to the pointer but that is a marginal benefit.

BTW, I found it lacklustre with the Magic Mouse 2; it's better with the Magic Trackpad 2.

 

[MarkB786]

Or maybe inspired by Samsung, who has had this functionality for its superior Tab for some time.

 

[PortoMavericks]

Kudos to Federighi and his team, because it works so much better than I expected.

Now, it's time for a help to make Affinity Photo, Photoshop and LumiaFusion compatible with all pointer features. Also, please, whatever you do, make sure you make extended display a reality on iOS 14.

And if it's physically possible to have USB Out and Video Out on the same Lightning dongle, make it happen Apple.

 

[rp2011]

The MK for the ipad Pro does cost a lot of money, but i have no regrets. My iPad pro was already my main computer and now I don’t even think about using my desktop computer unless i absolutely need to work on something major.

 

[The Game 161]

Well they made it distinguished from the way it was pre-13.4, on iOS 13, and made a big fuss about it.

so that's what I'm asking about. Comments like these aren't helpful at all

It works as great as it does on a MacBook