Advanced Multitouch Gesturing Patents

[MacRumors]

PCJoint.com points to ongoing research by Apple into multi-touch technologies. The most recent patent applications reveal techniques to recognize both palm and finger contacts in order to react appropriately.

PCJoint summarizes the possibilities:

- Need to move a cursor across the screen? Just slide your finger and it goes.
- Want to enter text? Just start typing and the text is there. You even get the touch feedback as if really pressing the keys.
- Need to erase some text? Use the backspace or just slide the finger backwards
- Have to write or draw something? Just pretend you hold a stylus or pen in your hand and start writing/drawing.
- If that is still not good enough, pick up the real stylus and start writing with it.
- All these cool touch gestures that Apple has implemented in iPhone and is exploring further? You know, scrolling, zooming, rotating & moving objects, etc; They are here too
- 3D object manipulation? Select a few 3D objects on the screen and use all 5 fingers on both hands to play with them.
- Want to take a break to relax and afraid to put your hands on the touch surface? Don’t worry Multi-touch 2.0 thingie knows that and does not pay attention to your resting palms or fingers

This should come as no surprise to those who have been following Apple's patent applications as well as their previous acquisition of Fingerworks. Previous patent applications have also suggested the possibility of one surface acting for many different functions (such as typing and gestures).

Article Link

[LillieDesigns]

I think there is a world of things we can do with multi-touch, but Apple just has to make sure it stays intuitive. I don't want to have to learn sign language and 300 gestures to get my iPod to do what I am telling it to do. Hopefully some firmware updates come out and we can get some cool new features on our new Apple toys.

[ryanmcd02]

Apple DS? Input and output screen?

[viper002060]

we're definitely gonna see a big change in the way in which one will interact w/ a computer pretty soon. seriously, how much longer can they drag out the mouse/keyboard set-up?

[Wayfarer]

¡Que guay! Eso es muy interesante.

[Merkuryy]

What I hope the most is to get these features through a firmware update in Leopard, because Multitouch is really a "secret feature" that everyone will be happy to own.

On a bigger screen , you'll be able to have less mistakes. My iPod Touch is great, but still made some mistakes some time. Another surprise is how can Apple really handle with texting, this is a delicate work after all

[Thertel88]

42 minutes

[plumbingandtech]

and the rumored big iphone size thingie makes sense now.

GIMME!

[Pandaboots]

I forsee greater things in the future (near future) for this than just iPhone/iPod use. Imagine a glass (and aluminum, of course) surface that becomes everything we need to manipulate a computer and a media center. Virtual keyboard surface, virtual mouse surface and a virtual remote. The technolgy is already here and in use in the iPhone/iTouch....Apple, it's time to revolutionize the industry (again) with touch sensitive virtual input devices. Physical keyboards, mice and remotes are so 2007...It will be fun to watch other companies caught with their pants down again. While everyone wastes their time trying to copy Apple's iPod, iPhone, iTunes Store, and OS, Apple will once again shock everyone with their creativeness, foresight and innovation.

[celloman]

wow so awesome!

[treysmay]

I hate to say this But I think the other half of the market, ie. Microsoft and friends have a more promising function for the distant future. While apple has a short terms plan. is only covering X and Y, while the big M is putting big money in the 'Z'

[dvince2]

What would be cool (and more realistic for the short term), is if some of this is brought into the touchpads on laptops and/or new mice for desktops.
For instance...if it can recognize a "stylus grab" (essentally 3 fingers close together), use this for highlighting text. a "pinch" motion for copy, and reverse for paste.
That would let them test the waters a bit before they completly redesign a standard over 30 years old of how we interact with a computer

[applefan69]

Wirelessly posted (Mozilla/5.0 (iPod; U; CPU like Mac OS X; en) AppleWebKit/420.1 (KHTML, like Gecko) Version/3.0 Mobile/3A110a Safari/419.3)

I'm willing to bet within 3 years we'll be seeing a touch screen MacBook or iMac.

Who know maybe Mac OS 11 will be touch-only OS. Knowing apple they could do AMAZING things if they made an OS for computer with multi-touch.

One thing troubles me though, that whole article just screamed "microsoft surface" I mean sure apple is the first to make multi-touch commerical. So its tricky to say who Is copying who.

[digitalbiker]

Originally Posted by Merkuryy

What I hope the most is to get these features through a firmware update in Leopard, because Multitouch is really a "secret feature" that everyone will be happy to own.

On a bigger screen , you'll be able to have less mistakes. My iPod Touch is great, but still made some mistakes some time. Another surprise is how can Apple really handle with texting, this is a delicate work after all

Multi-touch is primarily designed to be used for small devices where a keyboard and mouse are not practical. It is not really intended to replace a full size keyboard and mouse in a desktop set-up.

It would be very impractical to be reaching 20 or 30 inches across your desktop rubbing your fingers all over your 30" cinema display. Talk about repetitive stress syndrome!

About the only good use for multi-touch in a desktop environment would be for a new tablet like device (similar to Wacom's Cintiq) or a display table like microsoft demoed this year.

Otherwise think, ipod, iphone, pda, or notebook trackpad.

[danerh]

They'll be scrapping the big cat names soon for something a little more like...Mac OS Touch... I give them a couple of years at least before we see any macTOUCH's though... I hope I'm wrong, but I'd say its still quite a way off, enough time for them to develop the new operating system they'd need to run it all.

BUT HOW MAD WILL IT BE WHEN THEY DO!?

On the topic though... microsofts 'surface' technology scaled to a regular operating system would also be pretty sweet... and is no doubt just around the corner as well... which brings up some issues....

It could become quite difficult to move from one machine to another if they go about things differently. For a mac user today with little experience on a windows machine, most tasks are still obvious and more or less the same when using a windows machine. Resizing windows, poining and clicking, selecting text, right clicking etc are all universal. As are many keyboard shortcuts and; typing is a no-brainer.
If we move to touch set-ups even the simple things could become difficult no matter how intuitive they are when switching between platforms.

I wonder if there will be any formal standards formed or whether they will just evolve...

[fawlty]

Originally Posted by treysmay

I hate to say this But I think the other half of the market, ie. Microsoft and friends have a more promising function for the distant future. While apple has a short terms plan. is only covering X and Y, while the big M is putting big money in the 'Z'

ya mean Zune?

[applefan69]

Wirelessly posted (Mozilla/5.0 (iPod; U; CPU like Mac OS X; en) AppleWebKit/420.1 (KHTML, like Gecko) Version/3.0 Mobile/3A110a Safari/419.3)

Originally Posted by fawlty

ya mean Zune?

no he's talking about the axis's. Quote frankly he's being retarded though. I aint seen a 3d touch interface from microsoft

Now motion sensing is in a way 3d touch interface, so maybe nintendo deserves credit for making that commercial.

Cant help but wonder what would happen if nintendo and apple merged, both company's seem rather similiar in most aspects.

[digitalbiker]

Originally Posted by applefan69

Wirelessly posted (Mozilla/5.0 (iPod; U; CPU like Mac OS X; en) AppleWebKit/420.1 (KHTML, like Gecko) Version/3.0 Mobile/3A110a Safari/419.3)
Quote frankly he's being retarded though. I aint seen a 3d touch interface from microsoft

You need to get around a little more. Microsoft debuted Microsoft Surface before Apple officially released the iphone.
http://www.microsoft.com/surface

[ChrisA]

Originally Posted by Pandaboots

...Imagine a glass (and aluminum, of course) surface that becomes everything we need to manipulate a computer and a media center. Virtual keyboard surface, virtual mouse surface and a virtual remote.

Maybe something like this...
http://www.wacom.com/cintiq/

One other idea might be to use the webcam to watch the user's hands. So you would not have to touch the screen. We all can start learning ASL.

[ckurowic]

Originally Posted by viper002060

we're definitely gonna see a big change in the way in which one will interact w/ a computer pretty soon. seriously, how much longer can they drag out the mouse/keyboard set-up?

It depends on what your most comfortable with I suppose. How many people are really going to want to move their entire arm to make a pointer move? I don't see it catching on very fast. Why would I want to give up my easy to move mouse to lift my entire arm to move a pointer like in Minority Report?

Last time I checked, most people wouldn't want to use their hand to move a pointer across a 20+ inch screen. Lame. Multi-touch should be for small devices only.

I don't see this as something most users will want. Virtual mouse and keyboard? Um....okay. Last time I checked human beings work best with tactile input devices. Good luck typing on a virtual keyboard where you have zero feedback as to what button you have pressed. I'm skeptical to say the least.

Originally Posted by ChrisA

Maybe something like this...
http://www.wacom.com/cintiq/

One other idea might be to use the webcam to watch the user's hands. So you would not have to touch the screen. We all can start learning ASL.

So why would I want to lift my whole arm to move the mouse? Not practical. Never will be.

[applefan69]

Wirelessly posted (Mozilla/5.0 (iPod; U; CPU like Mac OS X; en) AppleWebKit/420.1 (KHTML, like Gecko) Version/3.0 Mobile/3A110a Safari/419.3)

Originally Posted by ckurowic

So why would I want to lift my whole arm to move the mouse? Not practical. Never will be.

Wouldnt be for a desktop. But it could amazing on a laptop. For starters the size of the laptop could be like half an inch thinner due to no keyboard



Plus laptops generally only have a max of a 17" screen which people rest on their laps. This means you woyldnt really have to move your arm much.



Plus I'm sure there's other advantages in which I'm missing.

[GregA]

Originally Posted by digitalbiker

Microsoft debuted Microsoft Surface before Apple officially released the iphone.
http://www.microsoft.com/surface

Huh?
Microsoft demoed Surface months after Apple demoed iPhone.
Microsoft released Surface... when? (have they?)... certainly after Apple released iPhone.

Neither Surface nor iPhone are 3D interfaces though, which is what you're replying to.... Anyway.. I'm not sure what your point is.

(edit:.. oh, I guess when they're talking about working on different axes ('axis's') you could consider the iPhone to do some of that with the accelerometer... but I really wouldn't call that a 3d interface).

[SiliconAddict]

Damn it. You can't patent some of this crap! I mean really. Moving your finger backwards to erase. Good lord this has been don on PDA's for about a decade. Apple is going patent insane.

Originally Posted by GregA

Huh?
Microsoft demoed Surface months after Apple demoed iPhone.
Microsoft released Surface... when? (have they?)... certainly after Apple released iPhone.

Neither Surface nor iPhone are 3D interfaces though, which is what you're replying to.... Anyway.. I'm not sure what your point is.

(edit:.. oh, I guess when they're talking about working on different axes ('axis's') you could consider the iPhone to do some of that with the accelerometer... but I really wouldn't call that a 3d interface).

The iPhone is to Surface as a barcode reader in a PDA is to the technology in general. Apple's implementation is "cute". Its not trying to invent an industry. To put it more bluntly. Microsoft is taking risks with their tech with a long term plan. Apple is doing NOTHING with the touch implementation in the iPhone other then being gimmicky.

Oh and as for Surface and when it was being developed. Sorry but I've been hearing rumors about this since 2003 when we had reps from MS doing then mobile experience tour for the Pocket PC. I use to lead the Pocket PC user group here in MN and I can tell you that there was rumblings about a gesture based interface back then. At the time it was thought that it was going to ship with the Tablet PC edition of Vista to allow you to interface with the tablet without a pen. This obviously didn't happen. MS HAS been working on this a while. The only reason why they choose now to demo it was because of Apple and Mac fanbois who would immediately bitch that they are ripping Apple off even though the two have NOTHING in common other then using your finger and hand. It was launched early and wasn't ready for prime time simply because of Apple. But frankly Apple's implementation is NOTHING. You do realize that coverflow was purchased by Apple. It didn't originate from them. Right? Really. People do need to end this copy crap. Apple copies MS. MS copies Apple. Apple gets a metric crap ton of technologies from open source. And it goes on and on and on.

[GregA]

Originally Posted by ckurowic

It depends on what your most comfortable with I suppose. How many people are really going to want to move their entire arm to make a pointer move? I don't see it catching on very fast. Why would I want to give up my easy to move mouse to lift my entire arm to move a pointer like in Minority Report?

A very important question. In fact, if you think of real life, you reach out and touch things, walk across a room and pick something up. BUT watch a fantasy or magic show, and we all want to make the tiniest gestures with our fingers and have what we want float across to us.

In the computer world... the mouse conveys our magic tiny gesture to make big movements on the screen. In that sense, why would you ever want to lift your arm to touch the screen... it's so much quicker and easier to grab the mouse (and you don't have to clean the screen afterwards).

And yet, I believe touch is a huge part of our near future. I'm not exactly sure why... let's see.... hmmm!

Of course, the touch is more natural than the mouse, for anyone untrained.

I think the main thing is that the mouse can only ever point at one thing at a time. A finger can also only point at one thing at a time (and a mouse on Windows can have 2 types of click!)... but what happens when you can use multiple fingers or even 2 hands? Basically, it enables a whole raft of new interface methods.

Note that the best way of making touch useless is making it work like a mouse. You can still only click one thing, and it's harder to do that!

What Apple (& MS) do with Multitouch will be intriguing.

[MikeDTyke]

Originally Posted by digitalbiker

You need to get around a little more. Microsoft debuted Microsoft Surface before Apple officially released the iphone.
http://www.microsoft.com/surface

Yeah, and Microsoft hasn't actually released a surface device yet. Last we heard Balmer said he was hoping for something by the end of this year but most likely early next year.

I think it's also important to point out that the 'coffee table' isn't a consumer device, at least not for the foreseeable future. It's going to turn up in casino's. hotels and fancy winebars. Whereas apple's 1st multi-touch system has been on the market for the last 6 months and combined sales with the iPod touch has probably beaten 2 million sales.

There's a lot of huff an puff about who's done what first, but if you ain't shipped a product then its just vaporware.

M.