Apple's Multi-Touch Designer Describes His Inspiration, More to Come?

[MacRumors]

http://www.macrumors.com/images/macrumorsthreadlogo.gif (http://www.macrumors.com/2009/02/18/apples-multi-touch-designer-describes-his-inspiration-more-to-come/)

Wayne Westerman and John Elias are the two engineers who are responsible for much of Apple's multi-touch technology found in the iPhone and notebook trackpads. Westerman and Elias originally founded a company called Fingerworks (http://www.fingerworks.com) in 1998 and developed a number of multi-touch input devices including keyboards and touch-pads. Apple then acquired Fingerworks in 2005 and Westerman has been a senior engineer at Apple since. Many of the multi-touch patents coming out of Apple have since credited Westerman as the lead inventor.

In a University of Delaware news article (http://www.udel.edu/udaily/2009/oct/citations102108.html) from October of 2008, Westerman gives some rare comments about his inspiration when first designing these multi-touch interfaces.

Westerman had apparently suffered from a "stubborn bout" of tendonitis which was later relieved by the use of this touch-sensitive technology:"I had an ergonomic problem and I paired it with a motivation," Westerman said of the early inspiration. "I'd always felt that playing the piano was so much more graceful and expressive than using a computer keyboard, and I thought how great it would be if I pulled some of that expression from the piano to the computer experience."While Fingerworks' input accessories did receive critical praise during their lifetime, they never achieved significant mainstream success before the acquisition. Apple has been more cautious by slowly introducing multi-touch into their Macs. At present, multi-touch technology is limited to notebook trackpads and the iPhone itself.

Apple, however, is continuing to hire more multi-touch engineers. Their latest job listing (http://jobs.apple.com/index.ajs?BID=1&method=mExternal.showJob&RID=31780&CurrentPage=1) is seeking a "Gesture Algorithms Engineer" -- a position that likely didn't exist even a year ago. This potential hire would become part of the engineering team responsible for creating Apple's "next-generation input devices and displays". Apple has been hiring (http://www.macrumors.com/2008/05/13/apple-hiring-for-advanced-multitouch-displays/) Multi-Touch specialists for almost a year now, and previous patent applications (http://www.macrumors.com/2007/08/02/apple-exploring-multitouch-interface-gestures/) show the possibilities of what could be accomplished with advanced gesture recognition.


Article Link: Apple's Multi-Touch Designer Describes His Inspiration, More to Come? (http://www.macrumors.com/2009/02/18/apples-multi-touch-designer-describes-his-inspiration-more-to-come/)

[yoyo5280]

Multi Touch Macs sometime soon?


Somehow I don't think thats actually a good idea.

[Frits]

Hope they aint gonna to make a multitouch keyboard or some crap like that. :rolleyes:

[talkingfuture]

His comments about the ergonomics are how I feel about using a Mouse at the moment. I'd really like to see a large multi touch surface for my iMac. It would just feel much more natural.

[ihabime]

I wish Apple would re-release fingerworks iGesture pad (http://www.fingerworks.com/), that thing was fantastic, you could ditch your mouse and just use a pad that recognized gestures from up to five fingers.

It was pricey, $200-300, but they were a small outfit, I'm sure with the volume that Apple would sell it could be cheaper.

[koobcamuk]

Hope they aint gonna to make a multitouch keyboard or some crap like that. :rolleyes:

I would like a multitouch keyboard... a secondary panel that can be manipulated by touch.

It's very hard to envisage though...

[dannyboi83]

I can just imagine the usefulness of a large multi-touch pad. It would work great with an iMac and can be used to replace the mouse and keyboard.

I'm up for that, let me know when its out :)

[bobbleheadbob]

I wonder if multitouch specialists are better in bed? :confused::eek::o;)

[dannyboi83]

I wonder if multitouch specialists are better in bed? :confused::eek::o;)


One would think so ;)

[JackSYi]

Awesome article. Where is that multi-touch mouse?

[nicksoper]

I have an external keyboard and laptop stand at the office, but I often just use the laptop on its own because of the multi touch gestures on the track pad.

Also having to switch between two hands on the keyboard, to the mouse all the time is laborious. It's much more graceful on the laptop with the trackpad.

Good on Apple for phasing all this functionality into the products, it gives slow people like me a chance to figure it all out.

[Frits]

I would like a multitouch keyboard... a secondary panel that can be manipulated by touch.

It's very hard to envisage though...

What would be good about a touch sensitive keyboard? I mean, what would be the difference? I personally like the crisp feel of real buttons beneath my fingers when I type. Also, it would be harder to spot typo's if you didn't actually feel the button being pressed.

Something as a replacement for the mouse would be nice though. I sometimes find myself typing short lines, like searches, with just one hand, because it's just faster than getting my right hand off the mouse. Maybe the numpad could be replaced with a multi touch surface, that would be way more easily reachable. It still isn't perfect though.

[Otaviano]

I want a tablet!

I want photoshop where I can draw directly on the screen, and manipulate things by touch. I want an iTunes alternative where I can touch my music as it's playing. Scratch on any mp3 like it was a vinyl record. Pitch shift, by moving my finger in a circle faster and slower. Mix on screen, admittingly the crossfader will be hardest to replicate with precision. People say there are not enough mainstream uses for a touch screen tablet, but to me the possibilities are endless. Give me a touch screen tablet, I'm selling my MacBook Pro (bought an iMac on sale) and won't purchase another laptop. I want a tablet.

[gzvinmen]

I don't think thats actually a good idea. :)
I wish Apple would re-release fingerworks iGesture pad, that thing was fantastic, you could ditch your mouse and just use a pad that recognized gestures from up to five fingers.

[grapes911]

I can't wait until they have an entire keyboard that is multitouch. My Fingerworks pad is starting to go and I need a replacement.

[Stately]

Tablet time. About 4 inches in height aaaand about, 5 inches in width, with some really nice processing power, full OS and enormous battery life. I would do some serious designing with that thing on the go. I could get my documents typed up and edit them. Everything on the fly. My gosh . . and then think if they tagged on phone capability? Uh oh . . A behemoth of a product. :cool:

[RichardI]

Every time this multi-touch stuff comes up I get ticked off. Am I the only one who has no interest in learning a whole new sign language? Does everyone out there see this as a step forward - an improvement? I don't. Why not just have a keyboard pop up on the screen that you can then use by touch? This "multi-touch" stuff just seems like a step backwards to me. Lemme see, was that 2 fingers or 3? Swipe left, or right - oops!:rolleyes:

Rich :cool:

[cnorth3]

Every time this multi-touch stuff comes up I get ticked off. Am I the only one who has no interest in learning a whole new sign language? Does everyone out there see this as a step forward - an improvement? I don't. Why not just have a keyboard pop up on the screen that you can then use by touch? This "multi-touch" stuff just seems like a step backwards to me. Lemme see, was that 2 fingers or 3? Swipe left, or right - oops!:rolleyes:

Rich :cool:

On the other hand, should we ignore technologies that potentially change the I/O paradigm for the better simply because we are used to a keyboard? I'm old enough :o to remember when people argued that the mouse was a worthless waste of time.

[notjustjay]

"Input devices" we already knew about. It's the "displays" part of the job description that intrigues me.

[Lepton]

Sony has a little keyboard (http://www.adaptivepath.com/blog/2008/11/12/ps3-controller-adds-touchpad-interaction/) that clips on the PlayStation 3 game controller, for chatting and such. The reason I mention it is that the little device has a trackpad mode - touch a button to go into trackpad mode, and the keyboard turns into a touch sensitive trackpad! Move your finger lightly over the keys and the PS3 mouse cursor (used in the browser etc.) moves.

Wouldn't this be interesting to see on a real computer device? Gesture right on the keyboard itself...

[GQB]

I want a tablet!

I want photoshop where I can draw directly on the screen, and manipulate things by touch. I want an iTunes alternative where I can touch my music as it's playing. Scratch on any mp3 like it was a vinyl record. Pitch shift, by moving my finger in a circle faster and slower. Mix on screen, admittingly the crossfader will be hardest to replicate with precision. People say there are not enough mainstream uses for a touch screen tablet, but to me the possibilities are endless. Give me a touch screen tablet, I'm selling my MacBook Pro (bought an iMac on sale) and won't purchase another laptop. I want a tablet.

I agree about the tactile possibilities for a tablet.
Where I think people miss the boat is crying for a touch screen on a notebook or desktop. There's a use for vertical touch at wall mounted kiosk displays, but as a normal way of interacting with a desktop its nuts. Ergonomically, your arms can take no more than a couple of minutes of reaching out to a vertical screen.
I think people got way too taken in by the Tom Cruise interface in Minority Report.

[GQB]

Every time this multi-touch stuff comes up I get ticked off. Am I the only one who has no interest in learning a whole new sign language? Does everyone out there see this as a step forward - an improvement? I don't. Why not just have a keyboard pop up on the screen that you can then use by touch? This "multi-touch" stuff just seems like a step backwards to me. Lemme see, was that 2 fingers or 3? Swipe left, or right - oops!:rolleyes:

Rich :cool:

Whether or not you personally see the use for you of a new interface (yeah, its a pain to learn new things), why would you be 'ticked off'? Seems a waste of good anger.

[Doctor Q]

"I had an ergonomic problem and I paired it with a motivation," Westerman said of the early inspiration. "I'd always felt that playing the piano was so much more graceful and expressive than using a computer keyboard, and I thought how great it would be if I pulled some of that expression from the piano to the computer experience."
Four more lessons from pianos:

* We use both hands for playing the piano or typing text, but our mice and trackpads are designed for one hand, while our brains are perfectly capable of coordinating two hand movements.

* For the most natural style, keyboards should be wide. Maybe not 8 octaves worth, but for the sake of our laptop sizes we are cramping our hand movements unnaturally.

* Like a piano, or a car, computers could benefit from foot input, maybe pedals. Our feet are idle under the desk while our fingers try to handle all of our interface actions. For example, a foot control that acts like a modifier key would greatly increase the gestures we could make.

* The best piano playing takes practice. So does fast typing. We shouldn't be afraid of new techniques just because we can't master them instantly.

[illegallydead]

I agree about the tactile possibilities for a tablet.
Where I think people miss the boat is crying for a touch screen on a notebook or desktop. There's a use for vertical touch at wall mounted kiosk displays, but as a normal way of interacting with a desktop its nuts. Ergonomically, your arms can take no more than a couple of minutes of reaching out to a vertical screen.
I think people got way too taken in by the Tom Cruise interface in Minority Report.

Precisely. In order for something like this to succeed, I see it requiring a huge shift in the design of computers. Think Microsoft Surface (http://www.microsoft.com/SURFACE/index.html). That is a good implementation of a touch interface in a manner that is build solely for multi-touch. You cannot simply take an existing iMac of Macbook, slap a huge touch sensitive space on it, and call it good.
Oh, and call me old fashioned, but I see serious typists staying with the current paradigm of computers, that is, PHYSICAL keyboard, with normal monitor, etc. For these folks, the multi-touch would be limited to something we are talking about, like a large multi-touch pad in place of a mouse or in place of the numpad...

Every time this multi-touch stuff comes up I get ticked off. Am I the only one who has no interest in learning a whole new sign language? Does everyone out there see this as a step forward - an improvement? I don't. Why not just have a keyboard pop up on the screen that you can then use by touch? This "multi-touch" stuff just seems like a step backwards to me. Lemme see, was that 2 fingers or 3? Swipe left, or right - oops!:rolleyes:

Rich :cool:

The "whole new sign language" is exactly what multi-touch designers DON'T want. This sort of U.I. is meant to feel natural, that is, anyone can come up and immediately start manipulating, say, a set of photos on screen. This is why I am arguing for a sort of direct manipulation like the example of Microsoft Surface, etc...

[Virgil-TB2]

... The best piano playing takes practice. So does fast typing. We shouldn't be afraid of new techniques just because we can't master them instantly.This may be true, but it's also the reason why only some tiny fraction of 1% of humanity knows how to play the piano. Hell, only one out of every five or so of the "techies" I meet even knows how to type. Most just use two or three fingers.

The ideal world where people learn to compute in as complicated and expressive a way as they use a piano is a few hundred years off me thinks. You need a whole better race of humans for such Utopian ideas to really work. ;)

[wobudong]

...Merely speeding up the sync function, for example; speeding up the (Me)Mail function and the associated "services" that take too long. Perhaps more servers for the $99/year fee would be a nice touch. I'm not opposed to multi-touch functions, but I sure would like to see some basic improvements made.
I guess that's the prime reason I just cancelled my $99 automatic annual fee 'service' to Apple, tossed Safari and now use Firefox and Camino as a secondary browser. More and more, Apple seems to storm into the future, leaving basics behind.

[Benguitar]

Multi Touch Macs sometime soon?


Somehow I don't think thats actually a good idea.

I think it could be a good idea, as long as they configure the "typing" situation out.

For instance the "MacBook Wheel" looks like it will be a HUGE pain to type with. If Apple were to make a "MacBook touch" I think that it would be best if you had more than one option for a keyboard,


Touch Area where you wish to type and virtual keyboard comes up like featured on the iPhone/iPod touch
Extrenal touch screen keyboard *as seen here*
USB/BlueTooth Apple Keyboards


http://s229.photobucket.com/albums/ee180/Benguitar2/?action=view&current=Sequence-21709-1031PM.flv

[thisguyukno]

This may be true, but it's also the reason why only some tiny fraction of 1% of humanity knows how to play the piano. Hell, only one out of every five or so of the "techies" I meet even knows how to type. Most just use two or three fingers.

The ideal world where people learn to compute in as complicated and expressive a way as they use a piano is a few hundred years off me thinks. You need a whole better race of humans for such Utopian ideas to really work. ;)

It's all about productivity and options. All multi-touch gestures have the option to turn off. You don't have to use it. But for those of us who do, and who feel that this will make us xx% faster in our daily lives, who want to learn new skills that will improve our computer experience -- we'll be that much more ahead of the competition. With Darwinism comes a better human race.

[powers74]

All I know is that ever since I got my Intuos2 off eBay a couple weeks ago I barely use my wireless Logitech.

[JoeDMD]

I agree about the tactile possibilities for a tablet.
Where I think people miss the boat is crying for a touch screen on a notebook or desktop. There's a use for vertical touch at wall mounted kiosk displays, but as a normal way of interacting with a desktop its nuts. Ergonomically, your arms can take no more than a couple of minutes of reaching out to a vertical screen.
I think people got way too taken in by the Tom Cruise interface in Minority Report.

Ever use a Wii?

[Saladinos]

I'm still not sure that Multi-touch has any place amongst the typical desktop paradigm we're used to (mouse and keyboard). Apple will have to redefine the desktop if they want multi-touch to be in it IMO. I have no idea what that will look like, but that's why they have all these full-time engineers dreaming it up.

It works on the mobile because there's no standard input paradigm. There are all kinds of models with rollerballs, joystiqs, buttons...etc. It's also immature enough of a market that people are accepting of changing the input paradigm. It also makes sense on mobiles where surface space is of such a premium. Nobody really got it right before - companies experimented with all kinds of screen and button sizes.

It looks like Fingerworks was a fantastic acquisition though. Great forward-thinking on Apple's part. Those guys were really pioneers of touch technology. Now they have some of the people most experienced with the technology adding to their products (and patent portfolio) as well as the acquired IP from FW. Touch is gaining enough momentum that Apple really could redefine the desktop, but this time they're smart enough to recognise it and protect their IP.

I would like to see Apple create a plain multi-touch panel with a display underneath. Just create the board, and let software developers think of clever things to do with it. You could do all kinds of things - create a keyboard, piano, game pad, eReader, whatever. Like a mini MS Surface, controlled by your computer so it can be thin and portable like the BT keyboard. Maybe with RFID technology. The size I'm thinking of is about the height of a DVD case and the width of the wireless alu keyboard, with about the same thickness. Possibly communicating via WiFi or some UWB technology. Built-in battery, recharged by docking one side to a strip that is fixed on your desk to raise it for keyboard use.

[twoodcc]

great, can't wait to see what comes next! hopefully a mactablet of some kind

[Turmoil]

I like what multi-touch I've used so far. It's amazing how addictive it becomes. Two button scroll. Oh yeah.

But most important; wouldn't it be great to be a "Gesture Algorithms Engineer" - sounds like you'd flind new ways to flip people off for a living!

[kas23]

Multitouch is great. But, what I want us cut and paste for the iPhone. They hire this very creative smart engineer, buy up all his patents, but Apple can't think of a few simple gestures to implement c/p. Oh, I must be dreaming, I guess c/p would produce a new revenue stream.

[kdarling]

I agree about the tactile possibilities for a tablet.

Of course, there have been touch tablets available for many years. With the exception of vertical and enterprise apps, they haven't done too well. It's really tough to tap out real work on a touchscreen vs. using a keyboard and mouse.

Ergonomically, your arms can take no more than a couple of minutes of reaching out to a vertical screen.
I think people got way too taken in by the Tom Cruise interface in Minority Report.

Indeed, Cruise had to take lots of breaks during those scenes, because his arms would get tired.

Touch has its place, but not everywhere. Same goes for voice.

[iPhoneJoe]

a multi-touch Mac Tablet would be great

[AUFan]

I could see a TouchTrackPad, similar to the one currently in MBP's, to replace the mouse for most ordinary computing tasks. Opening/Closing, switching between documents etc. In theory, it would also speed up other tasks, like rotating pictures, if you could just make a contextual hand gesture vs. having to find and click a button. Coupled with a regular keyboard, it'd be a step towards integrating multi-touch into "normal" desktop computing.

[BTW]

Hope they aint gonna to make a multitouch keyboard or some crap like that. :rolleyes:

Actually that would be kind of cool. Think of it like sign language. Instead of spelling your words you'd gesture them on a keyboard sized pad.

I could see Apple ditching the physical keyboard for a touch sensitive display that would be software driven and could have infinite keyboard layouts. Would I want one? Maybe not right away since it is all in what you're used to. I'm used to tactile feedback. The younger generation may not care so much.

[iMaggot]

I can't wait until they have an entire keyboard that is multitouch. My Fingerworks pad is starting to go and I need a replacement.

A multi-touch keyboard ?, are you mad lol.

[grapes911]

A multi-touch keyboard ?, are you mad lol.

Nope. Been using a Fingerworks one for a while now. It really needs to be replaced, but since Apple bought Fingerworks they don't sell them anymore. Apple has to release one at some point.