PDA

View Full Version : 'i3D' Demo App Shows Off Head-Tracking 3D on iOS Devices




MacRumors
May 23, 2011, 08:40 AM
http://cdn.macrumors.com/im/macrumorsthreadlogo.gif (http://www.macrumors.com/2011/05/23/i3d-demo-app-shows-off-head-tracking-3d-on-ios-devices/)


bBQQEcfkHoE

Last month, French researchers Jeremie Francone and Laurence Nigay posted a video (http://www.macrumors.com/2011/04/11/head-tracking-allows-glasses-free-3d-on-ipad/) showing off a method for using head tracking by a device's camera to simulate three-dimensional scenes for the iPad, a concept in which Apple has expressed some interest (http://www.macrumors.com/2009/12/17/apple-working-on-3d-hyper-reality-displays/).

http://cdn.macrumors.com/article-new/2011/05/i3d_targets.jpg

As noted by MacStories (http://www.macstories.net/news/ipad-2-head-tracking-glasses-free-3d-app-now-available/), a new demo app from Francone and Nigay known as i3D (http://appshopper.com/entertainment/i3d) has appeared in the App Store, giving users the opportunity to test out the technology on their own devices.

i3D is a free universal app running on the iPad 2, iPhone 4, and fourth-generation iPod touch. It contains a sample video image displaying a bounding box as it tracks the user's head location, as well as five demos showing the simulated 3D perspective that can be generated using the tracking.

Article Link: 'i3D' Demo App Shows Off Head-Tracking 3D on iOS Devices (http://www.macrumors.com/2011/05/23/i3d-demo-app-shows-off-head-tracking-3d-on-ios-devices/)



motwera
May 23, 2011, 08:44 AM
mm Cool hope the next idevices get this integrated already

Trius
May 23, 2011, 08:47 AM
Cool concept... but it doesn't work very well yet..

ArtOfWarfare
May 23, 2011, 08:47 AM
I wonder if a free API for this will be released... I can think of some cool features it could be used for in a game...

dlewis23
May 23, 2011, 08:49 AM
Cool concept... but it doesn't work very well yet..

The problem is as soon as you get somewhat out of view of the camera it doesn't work right. It's a cool concept but in the real world probably wont go anywhere.

Satori
May 23, 2011, 08:51 AM
Just downloaded this for my iP4. Some demos work better than others - and overall it is a bit jerky. I don't think the head tracking is quite right yet. Very promising though -it will be so cool when it's perfected.

JilzVT
May 23, 2011, 08:53 AM
Good start - but we're still several years away.

adztaylor
May 23, 2011, 08:55 AM
Cool concept... but it doesn't work very well yet..

Agreed.

VTNick7
May 23, 2011, 08:55 AM
I just want to know how it works when no one is touching the ipad.. I can understand the technology with the accelerometer in place.

ghostface147
May 23, 2011, 08:56 AM
Hmm, not as cool as I thought it would be.

nwcs
May 23, 2011, 08:56 AM
It is a bit smoother in the video than in real life but I think these researchers deserve a lot of credit. It's quite good for a proof of concept research project.

EricStriffler
May 23, 2011, 08:57 AM
I've been so excited to try out this head tracking 3D, and after playing with this app for a few minutes I'm so underwhelmed. It doesn't feel anywhere near as cool as I expected it to. Maybe it will be better with some more development, but I'm definitely let down.

eawmp1
May 23, 2011, 08:58 AM
It's a cool concept but in the real world probably wont go anywhere.

Famous last words.

guzhogi
May 23, 2011, 08:58 AM
Just downloaded it and I have to agree - very cool stuff, but still needs a lot of work.


I just want to know how it works when no one is touching the ipad.. I can understand the technology with the accelerometer in place.

Actually, it doesn't use the accelerometer. According to the article, it uses the front camera to track where your head is in relation. Because of that, there's no need to touch the iPad.

FSMBP
May 23, 2011, 09:09 AM
I just want to know how it works when no one is touching the ipad.. I can understand the technology with the accelerometer in place.

Thanks for the concern, chief, but read the article/watch the video. It's not a "fake vid."

AppleMacDudeG4
May 23, 2011, 09:09 AM
This is basically a demo to show as a proof of concept. It will only get better with time but it was still fun to play with. Given the fact that Apple has shown some interest, perhaps there will be a future product that will support this better.

samcraig
May 23, 2011, 09:09 AM
Cool in concert. Not nearly ready for prime time obviously.

The demo was "neat" - but kludgy at best

Small White Car
May 23, 2011, 09:17 AM
I think the current problem is that it seems to rely on ONLY the video camera. I think this accounts for a lot of the jerkiness and lag.

I suspect this technology will get much better once it starts using more iPhone sensors in tandem. Imagine if it worked by accelerometer and gyroscope and only used the camera to re-calibrate every half second or so. Then it won't be as big a problem when the camera loses you for a second.

I think they'll get there. Apps like MotionX-Dice already use a basic form of this trick using only the accelerometer. Once they make this better you'll see it spread to more and more apps.

Frosticus
May 23, 2011, 09:20 AM
Not bad for a proof of concept.

After trying it on my iPad 2 and iPhone 4, it seemed to work better and feel more natural on the iPhone.

Just an afterthought.. although the suggestion of 3D using this technique is cool, is it really that useful on a device which (in my case at least) is always held in the same position relative to the face? Would be interesting to see how something like this works on a desktop. Does a similar thing exist for Mac using the iSight camera..?

dcbjones
May 23, 2011, 09:28 AM
Very cool! :)

*LTD*
May 23, 2011, 09:35 AM
I've been so excited to try out this head tracking 3D, and after playing with this app for a few minutes I'm so underwhelmed. It doesn't feel anywhere near as cool as I expected it to. Maybe it will be better with some more development, but I'm definitely let down.

It's experimental at this point.

lesiu
May 23, 2011, 09:39 AM
Will be nice to manipulate some excel presentations :cool:
What is the music from ??

GadgetAddict
May 23, 2011, 09:39 AM
Great concept and a decent first attempt. If in the future iDevices get wide angle front facing cameras this could really be a great feature. Till then, congrats to the makers!

chadley_chad
May 23, 2011, 09:41 AM
Sure, it needs a small amount of work but WOW, unlike most others I was very impressed!

Looking forward to this being introduced into iOS and apps real soon!

jclardy
May 23, 2011, 09:44 AM
This is a cool demo but I don't see the tech going anywhere.

The problem is when you use a device like iPad or iPhone is that you would have to be aware of it all the time. If you have your ipad on a table and go to touch the screen your camera angle will change because your hand will probably block the camera.

And of course the iPad front facing camera is not very wide angle so once you move out of bounds it can no longer track you.

It might be a neat trick for games or a little bit useful in a 3d modeling app (to peek around the side I guess...still not that useful.) but I don't see it being used in anything other than that.

The extra processing power required for head tracking plus switching to a 3d interface wouldn't be worth the small benefit that the "3d" would allow.

IMO 3D will remain a gimmick until we can interact with our computers in 3 dimensions (Mouse: 2d, trackpad: 2d, touchscreen: 2d). That and we have displays that allow multiple people to see the effect without glasses.

CFreymarc
May 23, 2011, 09:44 AM
The problem is as soon as you get somewhat out of view of the camera it doesn't work right. It's a cool concept but in the real world probably wont go anywhere.

Yes and no. Whenever introducing a new technology, it needs to be in a form where there is little risk to the end user. This is what a lot of the game market is all about. I can easily see this technology go into a game such as many wooden maze games on the iPhone / iPad already. From that, you have a public with a more familiar technology and you can make production / enterprise apps from it.

pcogan
May 23, 2011, 09:52 AM
I suspect this technology will get much better once it starts using more iPhone sensors in tandem. Imagine if it worked by accelerometer and gyroscope and only used the camera to re-calibrate every half second or so. Then it won't be as big a problem when the camera loses you for a second.

In the demo the 3d works both while tilting the device and with the device stationary (while the cameraman moves around). With your suggestion, only the first ability is retained.

I think it's pretty cool, and will get better as algorithms and hardware improve.

MBMBaMdotcom
May 23, 2011, 09:59 AM
wow i thought it was going to be good, but when downloading the app, i had a sneaking suspicion it wouldn't work right. it barely works guys, at all on my ipad 2

irun5k
May 23, 2011, 10:01 AM
Overall a pretty impressive prototype, considering the available hardware was not engineered with this as a use case. The initial demo that tracks your head with a red rectangle demonstrates that they're doing a pretty solid job with the tracking part.

I think the stuttering and hesitation on the subsequent demos might have more do with the graphics processing & rendering than the actual tracking algorithms.

Note: I tested on an iPhone 4.

ciTiger
May 23, 2011, 10:05 AM
I see Apple using something like this on at least a few selected apps...
But If indeed this is used we won't see it for a year at least...

Chris Blount
May 23, 2011, 10:06 AM
Neat concept but really isn't any good without an actual 3D screen.

Popeye206
May 23, 2011, 10:06 AM
Hmm, not as cool as I thought it would be.

Agreed. The video was smoother than the App. Also, does not seem to work well in low light.

Perfected more, this could be cool.

Dagless
May 23, 2011, 10:12 AM
It's interesting, for sure. But you're still just getting 1 image per eye so it doesn't really appeal to me.

There are some regular DS games that use this technique too.

Arkious
May 23, 2011, 10:17 AM
Sure, it needs a small amount of work but WOW, unlike most others I was very impressed!

Looking forward to this being introduced into iOS and apps real soon!

much the same, its a shame apple are slow to act on things like this though, android will nab it before apple start working on it. however when they finally release it, it will probably own the face off androids lol

Arkious
May 23, 2011, 10:20 AM
I just want to know how it works when no one is touching the ipad.. I can understand the technology with the accelerometer in place.

it tracks your face using the camera

Astro7x
May 23, 2011, 10:40 AM
I think that head tracking on a handheld device is not necessary. My initial instinct when I loaded the App was to move the device itself. Then when I realized "hey, it's tracking my head!" I noticed it just wasn't as natural to move my head than it was to tilt the device. While I think it's great that the technology is there, I'd think this would be much easier and accurate to pull off with the accelerometer and gyroscope.

I would imagine that head tracking would make more sense in a environment when using a TV, where the screen is stationary. Though I've heard bad reviews of how head tracking was implemented in The Fight on the PS3 using the PS Move, the technology is still early too.

johncarync
May 23, 2011, 10:42 AM
The performance improved greatly when I positioned myself so that the camera saw only my head and a blank wall six feet behind me.

This also doubles as a pretty good hand warmer :)

DocNYz
May 23, 2011, 10:54 AM
It's experimental at this point.

Yeah for POC I think it worked pretty well. It worked better once I held my iPhone 4 a few feet from my face rather than fairly close, and I think the movement could be exaggerated a little bit more, but overall it does as advertised and paves the way for some really cool possibilities.

Steelers7510
May 23, 2011, 11:25 AM
Just downloaded on my iP4 and I don't really think it's that great. It's a good start though.

ivladster
May 23, 2011, 12:12 PM
This works only if one person is using the app. When you playing a game with someone next to you, the whole thing doesn't work.

louis Fashion
May 23, 2011, 01:13 PM
Damn. First they track my body - now they track my head.

WestonHarvey1
May 23, 2011, 01:38 PM
Tried it on the iPhone, haven't tried it on the iPad yet. It's awfully low-res and notchy... I didn't really get much effect out of it. The YouTube video, though, looks so good I swear at times it is tricking my brain into thinking it is stereoscopic, and it's not even my own head being tracked.

So this demo is a little rough but I think the concept is still good. I'd like to see them add in/out as well. When my head moves closer or farther the view should compensate, like I'm walking up to or away from a window.

SandynJosh
May 23, 2011, 03:47 PM
What is the music from ??

A piano. ;)

spooky2k
May 23, 2011, 03:54 PM
Neat concept but really isn't any good without an actual 3D screen.

The point is that this is 3D without a 3D screen. It's a very cost effective way of implementing 3D on tech that everyone already has. It's a great proof of concept. Combine head tracking with stereoscopic 3D and a 'lenticular lens' type wall and the 3D will be as good as any, but without the glasses and will work no matter where you sit.

I hope this is developed further. There's huge potential in this.

Trojita
May 23, 2011, 03:55 PM
Is this the same thing Johnny Lee came up with for the Wii three years ago? The picture of the Demo looks exactly the same.

spooky2k
May 23, 2011, 04:03 PM
Is this the same thing Johnny Lee came up with for the Wii three years ago? The picture of the Demo looks exactly the same.

It's building on what Johnny Lee did. He used a Wii sensor bar attached to a camera and a laptop. This uses a front-facing cam. Nice work.

Trojita
May 23, 2011, 04:37 PM
It's building on what Johnny Lee did. He used a Wii sensor bar attached to a camera and a laptop. This uses a front-facing cam. Nice work.

That's cool. You mean "Nice work" as in getting it to work with a front-facing camera is nice work right?

mingoglia
May 23, 2011, 04:51 PM
This is a cool demo but I don't see the tech going anywhere.

The problem is when you use a device like iPad or iPhone is that you would have to be aware of it all the time. If you have your ipad on a table and go to touch the screen your camera angle will change because your hand will probably block the camera.

And of course the iPad front facing camera is not very wide angle so once you move out of bounds it can no longer track you.

It might be a neat trick for games or a little bit useful in a 3d modeling app (to peek around the side I guess...still not that useful.) but I don't see it being used in anything other than that.

The extra processing power required for head tracking plus switching to a 3d interface wouldn't be worth the small benefit that the "3d" would allow.

IMO 3D will remain a gimmick until we can interact with our computers in 3 dimensions (Mouse: 2d, trackpad: 2d, touchscreen: 2d). That and we have displays that allow multiple people to see the effect without glasses.

I somewhat disagree. Right now this technology was using the existing camera as a proof of concept. As others have said, future versions would likely take into advantage the other hardware also in the device to help with tracking. In addition, if implemented, I'm sure the hurdles of blocking the camera will also be addressed. For example, perhaps they put multiple cameras or implement a camera behind the actual display (there's been some rumors about the camera moved here in the past).

Mike

ghostlyorb
May 23, 2011, 08:20 PM
Not bad!

SuperCachetes
May 23, 2011, 09:26 PM
Loads of potential IMHO. I'm not sure it's "3D" per se, but rather "artificial perspective." With front and rear cameras churning simultaneously this could be amazing in augmented reality apps.

Percybut
May 24, 2011, 12:16 AM
The idea is originally from a Carnegie Mellon University student.
see video posted in 2007 http://www.youtube.com/watch?v=Jd3-eiid-Uw

They even used the same demo! And they didn't acknowledge the student who came up with the idea.

JerzeyLegend
May 24, 2011, 08:28 AM
I'm pretty sure this is a recycled topic. I have already read about this.

I know because this is the only site where I get my mac news.

Alvi
May 24, 2011, 11:25 AM
There has been an app form OSX 10.5 for a while with the same demos which never got updated to work with Leopard/Snow Leopard

rcappo
May 24, 2011, 04:15 PM
I bet they could get 3D movies to play using this tech.

There are other potentials that developers will find uses for. Don't underestimate the developers.

And to all the complainers that this isn't 'perfection', development isn't pretty, it takes a while to make it better and work the bugs out.

mateus
May 24, 2011, 04:32 PM
There has been an app form OSX 10.5 for a while with the same demos which never got updated to work with Leopard/Snow Leopard

Are you referring to this?... p://www.chrisharrison.net/projects/headtracking/index.html (http://www.chrisharrison.net/projects/headtracking/index.html)

iTracker's pretty cool too (http://www.eyetwig.com/?doc=150009) - control your mouse with head tracking.

caspersoong
May 27, 2011, 07:47 AM
Just downloaded. Really awesome but laggy on iPod touch, as expected.