Third-Party Devs Will Be Able to Access iPadOS Apple Pencil Latency Improvements for Art Apps

Discussion in 'MacRumors.com News Discussion' started by MacRumors, Jun 21, 2019.

  1. MacRumors macrumors bot

    MacRumors

    Joined:
    Apr 12, 2001
    #1
    [​IMG]


    Apple in iPadOS introduced some performance improvements between the iPad Pro and the Apple Pencil, cutting latency from 20ms to 9ms with the new software.

    Third-party developers who make apps that use the Apple Pencil will also be able to take advantage of some of these latency improvements, Apple software development chief Craig Federighi confirmed last week.

    [​IMG]

    Federighi shared the information in a response to an email sent by Artstudio Pro developer Cladio Juliano, who tweeted what Federighi had to say last week. The info was highlighted today in a tweet by developer Steve Troughton-Smith.

    In the email, Federighi explains that third-party developers have had access to predicted touches via UIKit since iOS 9, and with iOS 13, developers will receive the "latest and greatest" touch prediction advancements in minimizing PencilKit drawing latency.

    Federighi explains just how Apple introduced the latency improvements, and he points out that there's a small gap of 4ms that developers won't have access to at the current time because Apple didn't have a way to safely expose the capability to developers. From Federighi's email:
    For developers, the WWDC sessions Federighi suggests include PencilKit, Adopting Predicted Touches, and Metal Performance Optimization.

    In a nutshell, the information shared by Federighi confirms that third-party apps that take advantage of the Apple Pencil will be getting some of the same latency improvements that we'll be seeing when using the Apple Pencil within native functions like Markup.

    The Apple Pencil latency improvements are built into iPadOS, the version of iOS 13 that is designed to run on the iPad. All of Apple's current iPads support the Apple Pencil. iPad Pro models work with the Apple Pencil 2, while the 6th-generation iPad, iPad mini, and iPad Air work with the original Apple Pencil.

    Article Link: Third-Party Devs Will Be Able to Access iPadOS Apple Pencil Latency Improvements for Art Apps
     
  2. EvilEvil macrumors 6502a

    EvilEvil

    Joined:
    Jan 8, 2007
    Location:
    New York City
    #2
    How nice of Apple. You would think they would limit functionality improvements to their own apps.
     
  3. Cosmosent macrumors 6502a

    Joined:
    Apr 20, 2016
    Location:
    La Jolla, CA
    #3
    Thanks for the Heads Up on the Metal WWDC session !
     
  4. thisisnotmyname macrumors 68000

    thisisnotmyname

    Joined:
    Oct 22, 2014
    Location:
    known but velocity indeterminate
    #4
    I would have expected that was automatically exposed from the OS. I'm a bit surprised that they had to explicitly make some of those capabilities available to third party developers.
     
  5. NickName99 macrumors 6502

    NickName99

    Joined:
    Nov 8, 2018
    #5
    I love that he gets into such detail. That’s interesting about the 4ms improvement they got using something they apparently can’t expose as a public method without some risk.

    Now I’m curious about “mid-frame event processing”, but googling it hasn’t immediately got me anything.
     
  6. izyreal macrumors regular

    Joined:
    Sep 26, 2012
    Location:
    USA
    #6
    I am glad that Apple is going down this route. I recently had the chance to use a Pixel Book for an extended time, and the high latency of the pen in third-party apps was a distinct problem.
     
  7. TSE macrumors 68030

    TSE

    Joined:
    Jun 25, 2007
    Location:
    St. Paul, Minnesota
    #7
    It's cool even the original Apple Pencil is getting an improvement as well. Very awesome Apple. :)
     
  8. Burnsey macrumors 6502a

    Joined:
    Jul 1, 2007
    Location:
    Canada
    #8
    Does anyone know if the latency improvements include the 1st gen iPad pros without the 120hz display? Or is it limited to the newest iPad Pros and apple pencil?
     
  9. RightMACatU macrumors 65816

    RightMACatU

    Joined:
    Jul 12, 2012
    Location:
    192.168.1.1
    #10
    While on the topic of latency... where are the refurbished iPad Pro?
     
  10. djeeyore25 macrumors 6502

    djeeyore25

    Joined:
    Dec 23, 2014
    Location:
    New York City
    #11
    I’m hoping it’s both. The latency with the original Apple Pencil on my 1st gen iPad Pro is terrible while using Pages. I also have a new iPad Pro and Apple Pencil 2 and have no issues while working on the same document in Pages.
     
  11. Cayden macrumors 6502a

    Cayden

    Joined:
    Jul 10, 2014
    Location:
    Utah
    #12
    Now I’m not sure so take this with a grain of salt, but as an engineer I’m inclined to believe “mid-frame event processing” means they are updating some pixel information (likely just the pixels associated with the pencil) in between frame updates in which all pixel information is updated and displayed. In other words, in between hardware detections of the pencil location, software would update where it presicts the pencil to be on the next update, and it can start looking for the pencil there instead of looking arbitrarily, mean the location can (usually) be found quicker. What I’m not sure about is if these pixels are actually being updated mid-frame or if the processing is simply keeping this information stored until the next frame is ready to update. I can’t see how the pixels could be updated mid-frame unless they had an individual refresh rate, so I’m inclined to believe the second case. If it’s the second case, it would make sense why Apple doesn’t want to give developers access to this, as this could quickly lead to timing errors between the software and hardware interrupts, such that it would only work within Apple’s framework and not an arbitrary code framework.
     
  12. cmaier macrumors G5

    Joined:
    Jul 25, 2007
    Location:
    California
    #13
    It is, if you use the appropriate control. But developer’s may want to integrate it into their own canvas or controls. In which case it is harder to expose it since things you do in your own code can interfere with the ability of the pen code to get the cycles it needs from the GPU and CPU.
     
  13. henrikhelmers macrumors newbie

    henrikhelmers

    Joined:
    Nov 22, 2017
    #14
    I think that makes sense. It is probably used for more accurate prediction, and it is probably extremely timing sensitive. Another commenter asked why this isn't just baked into the existing APIs, and the answer is probably that Apple will rewrite it to do just that for next year.
     
  14. thisisnotmyname macrumors 68000

    thisisnotmyname

    Joined:
    Oct 22, 2014
    Location:
    known but velocity indeterminate
    #15
    That makes a lot more sense, thank you.
     
  15. satchmo macrumors 68020

    Joined:
    Aug 6, 2008
    Location:
    Canada
    #16
    I hope this improved latency isn’t limited to just iPad Pros but also the 6th Gen iPad.
     
  16. nexusrule macrumors 6502

    Joined:
    Aug 11, 2012
    #17
    I think you don’t know how development works. When you start creating code you can’t always abstract it in a way that’s usable by third party devs through an API. What Federighi meant is right now the code that allow for that part of delay reduction is split between different of Apple software technologies. To be made safely accessible to others devs it needs to be abstracted, made indipendent, because private frameworks can’t be exposed for security reasons. You build these frameworks after you have the working feature, it’s simply impossible to abstract a solution that doesn’t exist. And this sort of work can require a massive rewrite of some parts of the relevant underlying technologies, and it requires time.
     
  17. mdriftmeyer macrumors 68030

    mdriftmeyer

    Joined:
    Feb 2, 2004
    Location:
    Pacific Northwest
    #18
    Private vs. Public Frameworks. Apple bangs on them until they are confident enough these APIs are for general consumption, without compromising any underpinning APIs still private.
     
  18. NickName99 macrumors 6502

    NickName99

    Joined:
    Nov 8, 2018
    #19
    I actually am a software engineer, but not in anything like this tech, much more mundane stuff. I like your predictive pencil location idea, it seems highly likely and would account for improved latency. I bet you’re also right about keeping the information stored until the next update. As long as they can ensure that the stored information of where the pencil was and where it’s predicted to be next is kept inaccessible (security concerns), I bet they’ll eventually expose this in the public API.
     
  19. jjudson macrumors regular

    Joined:
    Sep 20, 2017
    Location:
    North Carolina
    #20
    I’m watching this new Pencil functionality with anticipation. I’m hoping that we’ll be able to use the iPad as an editing tablet for Photoshop and Lightroom — similar to the Wacom tablets now, but better.
     
  20. hancuriang macrumors member

    Joined:
    Jan 17, 2019
    #21
    They had to, they dont have a drawing app. Markup is ****. However it’s nice they included the original apple pencil. Apple has changed lately for the better with ipados and this....
     
  21. travelsheep macrumors 6502

    travelsheep

    Joined:
    May 30, 2013
    #22
    Please bring Apple Pencil support to the iPhone. I use ProCreate on iPhone and both with the finger and any Wacom Stylus it's terrible.
     
  22. mihike macrumors member

    Joined:
    Mar 30, 2016
    #23
    Awesome, a great product gets even better. Would be nice if this emphasis on driving out latency and lags would be applied elsewhere in iOS. And for that matter on most non-Apple products, too, seems like accepting laggy performance is just the norm on so many products.
     
  23. Fixey macrumors regular

    Fixey

    Joined:
    May 16, 2017
    #24
    It has been mentioned that even the 1gen pencil is getting improved so yes 1st gen iPad Pro as newer ipad pros use the 2gen
     
  24. totalgiraffe macrumors newbie

    totalgiraffe

    Joined:
    Jun 22, 2019

Share This Page

26 June 21, 2019