Become a MacRumors Supporter for $25/year with no ads, private forums, and more!

eBay Builds Tech That Uses iPhone X TrueDepth Camera and ARKit to Navigate Apps With Head Motion

MacRumors

macrumors bot
Original poster
Apr 12, 2001
50,441
11,827



eBay has created a new technology it's calling "HeadGaze," which tracks the user's head movement through the iPhone X's TrueDepth camera and ARKit so that they can navigate around eBay without touching the screen. The technology was created by Muratcan Cicek, an eBay intern with motor impairments who was looking for a way to shop online independently.


The eBay team built a model that tracks the user's head using 3D information from ARKit, creating a virtual cursor that follows the head's motion in every direction. Then, designated buttons on the screen can be activated depending on how long the cursor has been in one spot. These buttons can perform actions like scrolling down, moving to another page, selecting a product to purchase, and more, all without touching the screen.

Cicek says that the technology's modular code design will let developers "easily integrate" HeadGaze's features into existing or future apps:
This year as part of my internship project at eBay, my team and I developed HeadGaze, a reusable technology library that tracks head movement on your iPhone X and starting today, the technology is available via open source on GitHub.com. The first of its kind, this technology uses Apple ARKit and the iPhone X camera to track your head motion so you can navigate your phone easily without using your hands.

HeadGaze enables you to scroll and interact on your phone with only subtle head movements. Think of all the ways that this could be brought to life. Tired of trying to scroll through a recipe on your phone screen with greasy fingers while cooking? Too messy to follow the how-to manual on your cell phone while you're tinkering with the car engine under the hood? Too cold to remove your gloves to use your phone?
eBay has developed an app called "HeadSwipe" as a way to test the HeadGaze technology. HeadSwipe is focused on browsing and buying items in eBay's deals section, and can be navigated entirely through head motions on iPhone X. Both HeadGaze and HeadSwipe are available open source on GitHub.


Next, the team is also looking at technology that tracks eye movements, and potentially fusing the two experiences together in future apps.

Article Link: eBay Builds Tech That Uses iPhone X TrueDepth Camera and ARKit to Navigate Apps With Head Motion
 

Kabeyun

macrumors 68030
Mar 27, 2004
2,818
5,289
Eastern USA
For anyone with mobility problems, this is a really useful tool. I would have expected it to come from Leap or Apple itself, not eBay. But I guess that shows how Apple's "Kit" frameworks are enabling some interesting innovations.
Love comments like this.

This innovation, aside from those with disabilities, may be a real boon as Apple seems to have gone all in with phones too big for my thumb to reach across. (I’m looking at you, iPhone SE, as you sadly fade to the horizon.)
 
  • Like
Reactions: Solomani
Comment

racerhomie

macrumors 6502
Aug 14, 2015
342
618
Love comments like this.

This innovation, aside from those with disabilities, may be a real boon as Apple seems to have gone all in with phones too big for my thumb to reach across. (I’m looking at you, iPhone SE, as you sadly fade to the horizon.)
You do realize the iPhone SE & 5S work perfectly , right?
 
Comment

MrJeffreyGee

macrumors regular
Apr 20, 2009
126
100



eBay has created a new technology it's calling "HeadGaze," which tracks the user's head movement through the iPhone X's TrueDepth camera and ARKit so that they can navigate around eBay without touching the screen. The technology was created by Muratcan Cicek, an eBay intern with motor impairments who was looking for a way to shop online independently.


The eBay team built a model that tracks the user's head using 3D information from ARKit, creating a virtual cursor that follows the head's motion in every direction. Then, designated buttons on the screen can be activated depending on how long the cursor has been in one spot. These buttons can perform actions like scrolling down, moving to another page, selecting a product to purchase, and more, all without touching the screen.

Cicek says that the technology's modular code design will let developers "easily integrate" HeadGaze's features into existing or future apps:
eBay has developed an app called "HeadSwipe" as a way to test the HeadGaze technology. HeadSwipe is focused on browsing and buying items in eBay's deals section, and can be navigated entirely through head motions on iPhone X. Both HeadGaze and HeadSwipe are available open source on GitHub.


Next, the team is also looking at technology that tracks eye movements, and potentially fusing the two experiences together in future apps.

Article Link: eBay Builds Tech That Uses iPhone X TrueDepth Camera and ARKit to Navigate Apps With Head Motion


eBay made this!?! Well that turned my head.

Definitely a good thing.
 
Comment

EndersShadow

macrumors newbie
Sep 7, 2016
26
21
Venice
This will be helpful for not just accessability but hand wankers, game movement controls, and advertisers will use this to see what you focus on the longest.
 
Comment

billybobfred

macrumors newbie
Jul 20, 2018
17
44
I would love to see tech that can track your eye movement. Imagine if your phone or computer could scroll and navigate based on what your eyes were focused on. That would be much more useful than using head movement.
 
Comment

Kabeyun

macrumors 68030
Mar 27, 2004
2,818
5,289
Eastern USA
You do realize the iPhone SE & 5S work perfectly , right?
Indeed I do, although not perfectly. At some point the A9 processor will struggle with current software and the experience will become annoyingly laggy. (That’s why I dropped my 6, and no, an acceptable answer for me is not to just ignore software updates.) Might happen with iOS 12, might not. An SE I buy now might run well today, but that’s today.
 
Comment

Bin Cook

Suspended
Jun 16, 2018
383
780
Nobody post a one word play on the name will they, the mods will mess their knickers.
 
Comment

Adithya007

macrumors regular
Apr 16, 2015
178
484
I remember Apple having their own version of this system called SwitchControl in their accessibility settings menu for years now. They're almost the same I suppose.
 
Comment

NT1440

macrumors G5
May 18, 2008
12,512
15,853
For anyone with mobility problems, this is a really useful tool. I would have expected it to come from Leap or Apple itself, not eBay. But I guess that shows how Apple's "Kit" frameworks are enabling some interesting innovations.
I’d be shocked if FaceID accessibility features aren’t well under development inside Apple for when it makes its way to the Mac.
 
  • Like
Reactions: Solomani
Comment

Justanotherfanboy

macrumors 6502a
Jul 3, 2018
813
1,348
My aunt uses a 5S . Trust me , it’s more than fine. It runs smoothly.

I’ve always had the latest iPhones at launch since iPhone 4, but recently had my X stolen & had no insurance...
I bought a 6S to tide me over until the 6.1” LCD iPhone is released & I’ll tell ya: other than a few of the betas hosing the battery life for a week or two- sure, I absolutely miss the bigger screen & FaceID, but performance-wise... I have zero complaints using a 3 year old phone; even having JUST come from the X.
Apple’s devices are simply astounding in their longevity!
 
Comment

Solomani

macrumors 601
Sep 25, 2012
4,011
8,263
Slapfish, North Carolina

Kang: "Dammit Chang! Stop looking at the pink Watch band! I told you, we are
not buying a pink band!"

Chang: "No fair! You always get to choose the watch bands. Why can't I choose this time?!"


1*EkJeSPXIuG5_i4teqATqdw.jpeg
 
Comment

racerhomie

macrumors 6502
Aug 14, 2015
342
618
I’ve always had the latest iPhones at launch since iPhone 4, but recently had my X stolen & had no insurance...
I bought a 6S to tide me over until the 6.1” LCD iPhone is released & I’ll tell ya: other than a few of the betas hosing the battery life for a week or two- sure, I absolutely miss the bigger screen & FaceID, but performance-wise... I have zero complaints using a 3 year old phone; even having JUST come from the X.
Apple’s devices are simply astounding in their longevity!
Did you have iCloud & FaceID enabled?
 
Comment
Register on MacRumors! This sidebar will go away, and you'll see fewer ads.