Apple Exploring MultiTouch Interface Gestures

[MacRumors]

If there was any doubt where some at Apple think the direction of the future is, a new patent application describes the progression of the computer user interface over the years:

Many attempts have been made over the years to improve the way users interact with computers. In the beginning, cards or tapes with punched holes were used for user input. Punch cards gave way to terminals with alphanumeric keyboards and text displays, which evolved into the modern keyboard, mouse and graphical-display based graphical user interfaces. Many expect that the use of multi-finger, touch-sensitive user interfaces ("multi-touch interfaces"), such as those described in the references incorporated above, will become widely adopted for interacting with computers and other electronic devices, allowing computer input to become even more straightforward and intuitive.

This particular patent is credited to John Elias who was one of the founders of Fingerworks, a company who had done much work in the field of touch-interfaces. Apple acquired Fingerworks in 2005.

Elias goes on to describe how multi-touch gestures can have a broad vocabulary and invoke a number of actions. The patent application describes how to implement dictionary of gestures ("chords") which are made up of a combination of fingers, thumbs, and/or other hand parts. The dictionary would be presented to the user for teaching purposes but could also allow the user to assign meanings to particular gestures.

An extremely broad vocabulary is described in that Elias explains that each user's hands can execute twenty-five combinations based on five independent fingers. In the end, over 300 possible combinations are expected per hand.

Examples given include "Thumb & 1 Finger" combinations producing on action:





and "Thumb & 2 Finger" combinations producing different actions:





This patent application was filed January 3, 2007 and was just published today. Apple's iPhone presently utilizes a multi-touch interface, but only implements a limited number of gestures, but there have been hopes that Apple would expand this interface to iPods and Displays

Article Link

[lazyrighteye]

300 Gestures per hand?!?
Dag.

So... could Apple, via a Software Update, enable more Gestures on iPhones?

Regardless, it will be interesting to see how Apple will implement this tech in their (more traditional, aka: non-iPhone) computer offerings. Why do I get the feeling the next iMac is where Apple will debut such things?

Hmmm, interesting times...

[idea_hamster]

Looks neat -- but how do you do 212 and 213 with only two fingers?

Are they really talking about the "pinch" (and "reverse-pinch" if that's a word) that are currently used on the iPhone for zooming? Maybe -- just bad icons.

[pavelbure]

seems easier to just use a mouse.

[wattage]

Of course it seems really cool, but that looks confusing, surely Apple will maintain simplicity with their further application of this.

[FF_productions]

This will be interesting to see. It looks confusing.

[RZetlin]

Apple should team up with Nintendo to work on this technology.

After all the Nintendo has the Wii with its motion controller.

[Grimace]

you'd need a whole new type of computer. ultra-portable tablet?

Fingerworks...

Best
Acquisition
Ever!

[TheAnswer]

Originally Posted by FF_productions

This will be interesting to see. It looks confusing.

Yeah...looks harder to memorize than all the gestures of RPS-101.

[arn]

Originally Posted by idea_hamster

Looks neat -- but how do you do 212 and 213 with only two fingers?

Are they really talking about the "pinch" (and "reverse-pinch" if that's a word) that are currently used on the iPhone for zooming? Maybe -- just bad icons.

Yeah, the icons are pretty generic and are the same for thumb + 1 finger, thumb + 2 finger, thumb + 3 finger etc...

so I think that just represents "pinch" and "reverse-pinch". So each "arrow" doesn't necessarily corrspond to a finger/thumb.

arn

[Small White Car]

Originally Posted by pavelbure

seems easier to just use a mouse.

You know how old folks all seem to be techno-phobic fuddy duddies who don't understand anything more complicated than a TV remote? It's not that they're stupid. The world just moved faster than they could keep up.

Well, this is the begining of the end for us, my friend.

Yes, I agree that a mouse seems easier. But I bet my 5 month old daughter is going to know all these hand motions by the time she's 4.

Give it another 2 decades and she'll be flying space shuttles with her toes while you and I and all the rest of the 'mouse users' will be sleeping in boxes under the freeway overpass.

[twoodcc]

Originally Posted by FF_productions

This will be interesting to see. It looks confusing.

yeah, it does a little bit. very interesting. maybe we'll find out more on Tuesday?

[mozmac]

These remind me of the Graffiti icons from my old Palm. Apple will definitely make them simpler and more intuitive. I think it's great that they're making this they're "next big thing" because it will give people a naturally human interaction with their computers instead of requiring them to learn how to use special input devices. I want to go down on record as liking the keyboard more than the mouse. The mouse has certain functions to perform, but people rely too heavily on it. You can get so much more done in a short period of time using keyboard shortcuts.

[mozmac]

Originally Posted by Grimace

you'd need a whole new type of computer. ultra-portable tablet?

Fingerworks...

Best
Acquisition
Ever!

Grimace...

Best
McDonalds character
Ever!

[Tankgunk]

I have to agree that the gestures look hard to remember, but playing around with my fingers (and thumbs) on the computer desk makes me think that this could be somthing I could live with, and might even find to be fun. However, I'd still like to have a keyboard and mouse on my computer desk. Not everything is better with multitouch.

[rbroady]

seems like something that would be added to a touchmouse on a laptop, im not sure how you would integrate this onto something such as an imac

[ImAlwaysRight]

I wonder instead of touch computer display if they will put a small trackpad-like "display" on the keyboard or something in the future -- maybe a small iPhone sized touch device that can be placed to the right or left of the computer, replacing the mouse. As has been said many times on this forum, I can't imagine using an iMac or desktop Mac with a touch screen display.

[MacJediDude]

The future of this will be fun to take part in .... all iPhone users are beta testers for this new cool technology

[arn]

The thing to remember is that multitouch gestures are not untested technology.




http://www.fingerworks.com/gesture_guide_web.html
http://www.fingerworks.com/userguides.html

Fingerworks had a very loyal following.

Our multi-finger gestures are the quickest way to issue command shortcuts, move the text cursor and more! Here's why:

All MultiTouch gestures can be performed anywhere on the surface, wherever your hand happens to be. This avoids wasteful hand repositioning to awkwardly reach for multiple keys in a hotkey sequence, or to move the mouse cursor onto a toolbar button.
MultiTouch gestures are distinguished by finger combination and a simple motion direction as the gesture starts, so the command assigned to the gesture issues immediately.
In contrast, symbolic gestures drawn with stylus on tablet cannot be recognized until you finish drawing the symbol. Drawing complex symbols takes awhile, which delays command issuance and slows you down.
What makes our multi-finger gestures healthier than the alternatives?

Because our touch sensing technology is zero-force, MultiTouch gestures are essentially effortless. Just as easy as swiping your fingers through the air! In contrast, holding multiple keys for a shortcut can get tiring!
For each command, you use a slightly different motion and finger set. This way you're less likely to overuse any one finger muscle, as can happen with repeated clicking of mechanical mouse buttons.
You won't tire your hand gripping a mouse or stylus. Just rest your hand on our touch surface whenever you like!
A single, continuous gesture motion can do the same job as repetitive keypresses and holds. For commands like Undo or Next App that sometimes need repeating, the command issues repeatedly the farther you move if you start the gesture slowly. But if you only want a command to issue once, start with a quick, crisp motion.
Reversing direction usually produces a complementary command like Redo or Prev App. Such intuitively paired motions are easier to memorize than separate shortcut keys for every command.

arn

[sartinsauce]

Originally Posted by Small White Car

You know how old folks all seem to be techno-phobic fuddy duddies who don't understand anything more complicated than a TV remote? It's not that they're stupid. The world just moved faster than they could keep up.

Well, this is the begining of the end for us, my friend.

Yes, I agree that a mouse seems easier. But I bet my 5 month old daughter is going to know all these hand motions by the time she's 4.

Give it another 2 decades and she'll be flying space shuttles with her toes while you and I and all the rest of the 'mouse users' will be sleeping in boxes under the freeway overpass.

I pretty much had the same reaction. It's a whole new language to me, and one that seems tricky to learn. However, children learning it as they grow up will find it natural. Whole new interfaces can be built on the basis of this language and users will find it intuitive, because they've know this language their whole life.

[salvatorereda]

I always thought that using a mouse is like using one finger for everything you do, eating, drinking, writing. The mouse never made sense to me. If you use all ten fingers then your able to utilize a computer as you would something in real life.

Look at the interface for the 3d software program MAYA. They started to utilize two hands and multi-button mouses back in 1996. Then Maya came up with the star system for quick commands without going to the top menu bar all the time. Its still brilliant.

Apple will incorporate this new technology into its mouse pads/laptops and possible tablets. I can also see apple coming out with a tablet/monitor such as WACOMS CINTIQ.

Its about time we use all ten fingers when we use a computer.

[TheSlush]

So will Apple's multitouch patents ever conflict with Microsoft's Surface Computing patents? Microsoft's Surface utilizes cameras rather than touch-sensitivity, but it's still a method of using hand gestures as a computer interface.


Sent from my iPhone

[letaalio]

Ok, this is really cool

Think about it and try them out on your touchpads, there isn't enough room for (most) of them (and I don't even have that big fingers) which means there must be an iMac (maybe not the next one per se) or a different kind of device whos screen is capable of detecting them!

[zombitronic]

I'm not saying that I expect this in the upcoming iMacs, but with iMacs being all about the built in display, unless Apple develops a Mac Tablet, the iMac seems like a logical debut for this technology in the desktop form factor.

The other alternative would be selling a touch capable Apple Display for non-iMacs, or as mentioned, an external desktop trackpad. Touching the "objects" on screen seems to be the more natural form of input, for example, the iPhone's interface.

[TheSlush]

Originally Posted by FF_productions

This will be interesting to see. It looks confusing.

But leave it to Apple to make it simple and easy for anyone to use.



Sent from my iPhone