Members of Apple's PrimeSense Team Patent Method of Interacting With Mac Using Hand Gestures

MacRumors

macrumors bot
Original poster
Apr 12, 2001
7,458
8,522



Apple today was granted a patent originally filed in August 2016, describing a method in which users would be able to control a Mac computer -- and potentially other devices -- using a "non-tactile three dimensional (3D) user interface" (via Patently Apple). The patent's inventor credits go in part to Amir Hoffnung and Jonathan Pokrass, two current Apple employees who joined the company from PrimeSense after Apple acquired it November 2013.

Some of PrimeSense's tech, which was originally used in Microsoft's Kinect devices on Xbox platforms, now resides in the front-facing TrueDepth camera of the iPhone X, and the new patent hints at a potential future where this technology expands in function to Macs as well. Instead of recognizing faces, Apple's patent describes a Mac that recognizes a "gesture by a hand," allowing users to interact with the computer without tactile inputs like a keyboard, mouse, or trackpad.



The patent includes a variety of gestures that users would use to control the 3D user interface, including what are called "push," (figure 2) "wave," (figure 3) and "up" (figure 5) interactions, which are all grouped into a category of "focus gestures." According to the patent, some of these could be used to perform basic app interactions, like scrolling through a menu, as well as change the state of the system from locked to unlocked.
Gestures described herein include focus gestures and unlock gestures. A focus gesture enables the user to engage (i.e., take control of) an inactive non-tactile 3D user interface. An unlock gesture enables the user to engage a locked non-tactile 3D user interface, as pressing a specific sequence of keys unlocks a locked cellular phone. In some embodiments, the non-tactile 3D user interface conveys visual feedback to the user performing the focus and the unlock gestures.

Examples of unlock gestures include an "up" gesture (e.g., raising hand 30 a specified distance), a sequence of two sequential wave gestures, and a sequence of two sequential push gestures, as described in detail hereinbelow.
Apple has previously been granted patents related to 3D sensing and gesture controls, but the company has yet to release a product that takes advantage of these features. Some of these previous patents included gesture controls on iPads and iPhones as well as on Magic Keyboards, which would allow users to gain access to virtual buttons and potentially streamline certain elements of the user interface.


Of course, with Face ID already launched on iPhone X, it's predicted that Apple will look into adding the facial recognition software into future iMacs and MacBooks before it focuses on a new unlocking system. It's also unclear whether the new 3D user interface patent would include both pieces of technology -- 3D gestures and Face ID -- to further augment a Mac's unlocking process.

As with any patent, the technology in question might not make it to an Apple product in the near future, if at all, but it is an interesting glimpse into what Apple might be planning to do with PrimeSense's technology down the line.

Article Link: Members of Apple's PrimeSense Team Patent Method of Interacting With Mac Using Hand Gestures
 
  • Like
Reactions: DeepIn2U

bbeagle

macrumors 68040
Oct 19, 2010
3,339
2,453
Buffalo, NY
I would definitely use this for switching spaces on my desktop, or scrolling. That's pretty much the only things I use my trackpad for. I use the mouse for everything else.

It would seem odd at first. Like I'm an orchestra conductor in front of my computer. Sorta cool. Sorta odd.
 

PastaPrimav

macrumors 6502a
Nov 6, 2017
800
1,292
I don't see these things coming to fruition. The iterations of watchOS and all the failed touchscreen desktops/laptops on the market have already taught us that raising your hands is not a workable UI. Apple has shaped their entire watchOS experience around minimizing the amount of time you spend with your wrist raised, because it is critical to the UX. Every second is a lifetime.

If this research leads anywhere, it will be quick, casual gestures that you perform once in a while (turning something on or off). There is simply NO application for extended hand gestures in the air. None whatsoever.
 
  • Like
Reactions: Morgenland and Mc0

OldSchoolMacGuy

Suspended
Jul 10, 2008
4,202
8,906
Remember, Apple has thousands of patents and most never ever see real-life use.

This doesn't mean they're likely to put it into an upcoming product. In fact, most of the time Apple doesn't give away upcoming changes by filing a patent far in advance.
 

PaddyTB

macrumors newbie
Dec 3, 2013
16
25
I don't see these things coming to fruition. The iterations of watchOS and all the failed touchscreen desktops/laptops on the market have already taught us that raising your hands is not a workable UI. Apple has shaped their entire watchOS experience around minimizing the amount of time you spend with your wrist raised, because it is critical to the UX. Every second is a lifetime.

If this research leads anywhere, it will be quick, casual gestures that you perform once in a while (turning something on or off). There is simply NO application for extended hand gestures in the air. None whatsoever.
Tell that to Marcel Marceau
 

Neepman

macrumors 6502
Jul 31, 2008
349
340
You know what you can do with these gestures....
God. Things like this make me giad I will be dead soon enough .
 

keysofanxiety

macrumors G3
Nov 23, 2011
9,472
24,228
Why is everybody thinking this would be a new UI like MS's touchscreen/mouse & keyboard bastardisation?

It could just be for accessibility features; many would find this useful and there are already existing gestures such as these, but require external hardware to work.
 
  • Like
Reactions: sofila

840quadra

Moderator
Staff member
Feb 1, 2005
8,091
3,384
Twin Cities Minnesota
I see great value in this as accessibility as well as new ways to be able to operate these devices without touch.

Loving the iPhone X for allowing me to operate it without touch, been a great kitchen companion in this regard. Offering more options for control outside of SIRI would be awesome!

It should recognize American Sign Language, which would give you a gestural keyboard that is already well worked out, and some people already know it.
I would love to see this, it would make adoption of sign language much larger. I can imagine that it would be a great learning opportunity as well!
 
  • Like
Reactions: airmanchairman

ChrisCW11

macrumors 65816
Jul 21, 2011
1,039
1,417
Oh yeah, this is WAY more usable then touching the screen.

Also I am pretty sure Microsoft patented the **** out of gesture controls with Kinect and their newer Hololens product, so good look there Apple.

There is absolutely no innovation here, just copying someone else's idea with some minor change in intent. Sure maybe Apple can might patent a specific "gesture" that wasn't covered by Microsoft or many others, but the original innovation to use body movements to manipulate on screen objects was already innovated by Microsoft, and Nintendo, and Sony and a slew of other smaller companies more then a decade ago.

Apple is the penultimate patent troll crawling through patents to find some loophole or gap not covered so they can jump on it. Apple just wades into a market and rips everyone off years later and then creates a revisionist history duping fanbois in believing that they invented all this in the first place. It would be nice if Apple took their 250 billion in profits sitting in a foreign bank and actually came to market first with ANY kind of innovation not already pioneered by someone else first. Apple is a follower, period.

Apples ONLY innovation is their logo.
 
  • Like
Reactions: mi7chy and swagi

bernuli

macrumors 6502a
Oct 10, 2011
531
300
That is great, but all I really want is a vertical scroll bar on the side of my MacBook Pro screen to easily scroll documents.
 
  • Like
Reactions: NMBob

asleep

macrumors 68040
Sep 26, 2007
3,454
1,390


Because we want a bunch of lasers in our eyes and tracking our every move?

Think what Google analytics and Amazon can do with that...

We see you're reaching for a pen... how about a uni-ball Vision Rollerball Pens, Fine Point (0.7mm), Black, 12 Count?

Would you like some SweetLeaf Natural Stevia Sweetener to go in you tea?

Oops, looks like you spilled your coffee. How about some Bounty Select-a-Size Paper Towels, the quicker picker upper!

We're noticing a few unruly nose-hairs, we've ordered you a Fancii Professional Nose & Ear Hair Trimmer with LED Light, Water Resistant, Stainless Steel Blades, and Battery Power!
 

Lepton

macrumors 6502a
Apr 13, 2002
821
251
Cold Spring Harbor, NY
I don't see these things coming to fruition. The iterations of watchOS and all the failed touchscreen desktops/laptops on the market have already taught us that raising your hands is not a workable UI. Apple has shaped their entire watchOS experience around minimizing the amount of time you spend with your wrist raised, because it is critical to the UX. Every second is a lifetime.

If this research leads anywhere, it will be quick, casual gestures that you perform once in a while (turning something on or off). There is simply NO application for extended hand gestures in the air. None whatsoever.
Yes, and Apple knows this. They said many years ago that having a touch screen on an iMac or laptop is no good because your arm gets tired reaching for it all the time. So with this knowledge, I feel it would be as you say, a way to give quick commands on occasion.
 

artfossil

macrumors 65816
Oct 5, 2015
1,124
1,153
Florida
I think this is fabulous.

Especially for all of us who have different learning preferences (kinestheic learner here) and for all who are differently abled.
 
  • Like
Reactions: airmanchairman

Tec972

macrumors regular
Aug 19, 2010
216
137
LOL the picture. Looks like an Xbox 1 connect. "Can't innovate my ass" my ass. :rolleyes: