Video-In and Magnetometers Could Introduce Interesting iPhone App Possibilites

[audioteknika]

Wait.. A magnetometer? Just give me a nuclear powered battery and I'll be impressed..

iPhone G3 with 2 hours of battery life, comming to an apple store near you.. Yawn..

 

[DELLsFan]

Now that BG apps are the only things left for people to complain about (which is also technically possible. Apple has to work harder to prevent it from happening, for battery life reasons), once Apple enables those when battery technology allows it, what will people start complaining about?

I am putting my money on "I can't get my work done unless Apple sells it in a flashy pink color with a picture of a giraffe engraved on the back"...

In all seriousness, though, more than background processes (which again, are a technical usability issue) Apple needs to improve inter-app communications, and notifications. The notification system with the popup dialogs is not very scalable, and will become a mess with push notifications.

Good question, point, and assertion.

I think there will come a point where one of two things will happen:

1. Apple just won't or can't practically innovate too much further to improve upon the phone's hardware - really a computer - due to the form factor limits and battery restrictions.

2. The form factor (ie. size) of the phone will have to increase - making it more Star Trek PADD-like or small tablet-like to accommodate all the desired bells and whistles and battery demands. The iPhone will be reworked - and transform into some new product smaller than a netbook, but larger than the current iPhone (think mini-Kindle) that will still be able to make and receive phone calls (video and audio).

I think both 1 and 2 are possible ... with Apple simply releasing some sort of iPhone Deluxe model that is larger for 2 ... while keeping the original iPhone size form factor (iPhone classic) for 1.

 

[NightFox]

Wow, just hope we're not setting ourselves up for a big disappointment - let's remember that potential and what we actually get on the next iPhone may not be the same.

Here's hoping though!

 

[X38]

Using the existing video out capabilities and the third party hardware access being added in the new SDK along with these new compass and video input functions, one could add something like this http://www.vuzix.com/iwear/products_wrap920av.html
and things might get very interesting indeed.

 

[miketcool]

3D magnetometer + Android + SkyMap app = amazing!

Basically the app just gets your GPS location so it loads the correct map. Then your screen shows the overlay of constellations for where you are looking. The possibilities really open up with these kind of interfaces. You can just move around and get the star information for any part of the sky, including those below horizon.

 

[7egend]

How did these guys make these if the new iPhone is yet to be released

the SDK doesn't support any of this, but you can code hardware to do anything.

---

I like how the first App is set to recognize the barcode to generate a certain animal and then allow it to move across the "scene" pretty slick really.

And the last one, I have seen this done before but not using a built in magnetometer or the touch controls. The pop-up affect is always great, cause everyone loves a good pop-up book.

 

[bokdol]

there is a japanese virtual doll that you can buy for your computer that you can interact with in the very same manner. a little more perverted but they have had it out for a bit now.

http://techblips.dailyradar.com/video/japanese_webcam_doll_offers_augmented_reality/

 

[iPhoneNYC]

Wow. Imagine the potential. And I understand the thing makes phone calls as well.

 

[diamond.g]

Would be nice - but there are issues there that make it a harder problem than it sounds.


How would you do it?
How would BG processes help with the scalability?

Personally, from having used, as well as developed for, mobile platforms that allow third party background process, I'm really loving the iPhone's lack of them and think the Notification Service system is the right solution. There might still be 0.01% of apps (figure plucked from the air in true forum posting fashion) that would benefit from having them, but I think it's a case of "the needs of the many".
Of course Apple can still screw it up with a MobileMe-esque rollout - but I'm hoping they were serious when they said the delay was while they redesigned it to scale enough to meet demand.

Honestly the only apps that would really use background processes would be internet radio type apps (think ipod) and skype like apps (think phone). Everything else could do without. The only fear I guess for the push notification feature is that everyone will want to use popups when a badge notification is probably more elegant. The only thing Apple needs to worry about is push notifications being a single point of failure (I seem to recall them not saying if they will guarantee service availability) see RIM...

 

[Lesser Evets]

It's quite amazing what can fit into a tiny iPhone.

Though nice programming and interesting at a glance, these games or applications shown aren't particularly useful or engaging.

I can't wait to see what Apple does with its new hardware.

 

[iPaf]

If this actually becomes reality in the new iPhone, I'm officially old.

When you are so mind blown by technology like that, is when you know you're no longer a member of the current generation.

Hey, I'm 20 and I'M blown away that these possibilities!

Don't worry, you're probably not that old

 

[twoodcc]

wow. these things sound great! can't wait for the new iPhone!

 

[Truz21]

The Dog coming out of the printed card isn't really anything new. Sony has a whole card playing game based around this with their PS3 and the EyeToy.

In fact you can produce something similar right now with a webcam and a printer, thanks to General Electric. Check it out:

http://ge.ecomagination.com/smartgrid/#/augmented_reality

 

[crossifixio]

damn this is good stuff. Apple is always one step ahead even when they are two steps back!

 

[*LTD*]

damn this is good stuff. Apple is always one step ahead even when they are two steps back!

Well consider this:

Apple's iPhone interface is about two years old now.

Other phones that are current, or that are due for release in the immediate future (aside from Pre vapourware), don't even come close. After 2 YEARS! And Apple is just widening and widening the gap.

It' sad to see hoe half-asleep the rest of the industry was at the time of the iPhone's release, and even now, two years later.

 

[megfilmworks]

In and of itself, a 3-axis magnetometer can only detect the roll, pitch and yaw angles of the device relative to the Earth's magnetic field lines.

It can also detect direction of the magnetic field. Hence where north is.

 

[Cander]

3D magnetometer + Android + SkyMap app = amazing!

Basically the app just gets your GPS location so it loads the correct map. Then your screen shows the overlay of constellations for where you are looking. The possibilities really open up with these kind of interfaces. You can just move around and get the star information for any part of the sky, including those below horizon.

Now that would be awesome.

 

[kspreen]

Hey all,

As the developer behind ARf I'm excited to see this discussion I want to clear up a few things though...

First, as a few people have mentioned, this isn't new technology. The only new thing here is putting it on the iPhone. Sony has been doing this with the EyeToy, and the new FLARToolKit has really opened up desktop augmented reality (using your computer's webcam).

However, what we're excited about here is handheld Augmented Reality (AR). The phone becomes a window to another world, which you can explore much more freely than you could with a fixed camera. Again, there are other small devices that will let you do this, but the iPhone is something of the "Holy Grail" in the mobile world at the moment. Putting these apps on the iPhone would be a great way of getting more people to think in terms of AR.

The video you see here uses an older version of the SDK. There's no reason these apps shouldn't exist now, except that the functionality required is in private rather than public frameworks. We can only hope that Apple intends to make any new video capabilities part of the public SDK, so that developers can finally explore this exciting class of applications.

Thanks for all your feedback! Hopefully you'll be seeing more apps like this later this summer

 

[zombitronic]

A few of us have been discussing augmented reality on these forums for awhile. This is really amazing stuff and the technology of the future. Comments about yesterday's article about the new camera and magnetometer went into detail about this as well as 3-axis magnetometers. There are also a lot more links to videos showing off this technology on the iPhone and other devices. Recently, I started a thread detailing examples of GPS-based navigational AR. Check it out if you're interested. I really love this technology and love discussing it.

We won't see the true can't-tell-virtual-from-reality version of AR until we have the technology to implement augmented data in relation to all of it's real world surroundings. Think of an imaginary virtual friend. You point your camera at a location and you see a virtual person. Creepy. Anyway, as you move around the virtual person, keeping your camera pointed at them, you see the virtual person from all angles. It looks, through the display, as if there's really a person there. This would work great through a pair of goggles with see-through displays.

For this to work this well, GPS and the magnetometer would need to be precise enough to know the exact location and orientation of your device down to an inch, maybe even less. I'm not sure that GPS is there, yet.

In addition, if you put your hand up in front of your virtual friend, ideally, your hand should block them. Without technology to sense and judge distances of near range objects, however, your hand and everything else in the foreground would be blocked out by the augmented image. Technology such as a stereo camera (dual cameras to see two images and judge depth perception) and LIDAR (lasers that scan and determine range) would need to be implemented to properly display augmented data behind objects in the foreground, in relation to the augmented image's intended position. Unfortunately, LIDAR is bulky and expensive and stereo camera hardware and software isn't very consumer ready, yet.

Regardless, this gives you an exciting look at what the world may look like, or at least how we may look at the world in the future.

 

[zombitronic]

Hey all,

As the developer behind ARf I'm excited to see this discussion I want to clear up a few things though...

First, as a few people have mentioned, this isn't new technology. The only new thing here is putting it on the iPhone. Sony has been doing this with the EyeToy, and the new FLARToolKit has really opened up desktop augmented reality (using your computer's webcam).

However, what we're excited about here is handheld Augmented Reality (AR). The phone becomes a window to another world, which you can explore much more freely than you could with a fixed camera. Again, there are other small devices that will let you do this, but the iPhone is something of the "Holy Grail" in the mobile world at the moment. Putting these apps on the iPhone would be a great way of getting more people to think in terms of AR.

The video you see here uses an older version of the SDK. There's no reason these apps shouldn't exist now, except that the functionality required is in private rather than public frameworks. We can only hope that Apple intends to make any new video capabilities part of the public SDK, so that developers can finally explore this exciting class of applications.

Thanks for all your feedback! Hopefully you'll be seeing more apps like this later this summer

Hey kspreen. I love your work! I'd love to get into the development of software like this. Imagine the possibilities... augmented advertising with virtual billboards, augmented navigation and virtual tour guides, augmented social networking real-world overlays, ... and I'm sure the adult content industry would get in on this.

 

[Stately]

Can't wait for WWDC ! It's gonna be phenominal.

 

[miketcool]

Now that would be awesome.

IS awesome.

SkyMap

 

[kas23]

This would be great to use with google street view.

Another application/function that would be totally possible is the ability to search Google Streetview.I know this would likely be entirely redundant, but it would be cool if we could take a photo(s) of a street and it would search the Streetview database and show exactly where you are via Streetview. Or, maybe Google could construct Streetview for areas that are not currently available via people's uploaded photos (a la photosynth). Or, allow people to upload photos that Google could incorporate into Streetview to show streets during different seasons or at different time periods (besides the static time snapshot we get now). The magnetometer would make these images much more precise and accurate. Some of these capabilities would revolutionize the ability to search for real estate.

 

[qubex]

There's a serious mistake in the post

A triple-axis magnetometer does not allow the determination of position, it only allows the determination of orientation.

Determining position is a much more complicated affair that requires (for example) a known point of origin and then continued integration of an inertial frame, as is the case with inertial guidance systems.

Or GPS.

Which raises the issue: if determning position were all that a triple-axis magnetometer is useful for, why would they bother to include redundant capability already covered by the GPS?

 

[guzhogi]

The dog video looks kinda cool. It would've been funny (in a perverted way) if there was a way to make the dog lift his leg (if you know what I mean). Hey, Apple allowed iFart.