Apple's Multi-Touch Designer Describes His Inspiration, More to Come?

[MacRumors]

Wayne Westerman and John Elias are the two engineers who are responsible for much of Apple's multi-touch technology found in the iPhone and notebook trackpads. Westerman and Elias originally founded a company called Fingerworks in 1998 and developed a number of multi-touch input devices including keyboards and touch-pads. Apple then acquired Fingerworks in 2005 and Westerman has been a senior engineer at Apple since. Many of the multi-touch patents coming out of Apple have since credited Westerman as the lead inventor.

In a University of Delaware news article from October of 2008, Westerman gives some rare comments about his inspiration when first designing these multi-touch interfaces.

Westerman had apparently suffered from a "stubborn bout" of tendonitis which was later relieved by the use of this touch-sensitive technology:

"I had an ergonomic problem and I paired it with a motivation," Westerman said of the early inspiration. "I'd always felt that playing the piano was so much more graceful and expressive than using a computer keyboard, and I thought how great it would be if I pulled some of that expression from the piano to the computer experience."

While Fingerworks' input accessories did receive critical praise during their lifetime, they never achieved significant mainstream success before the acquisition. Apple has been more cautious by slowly introducing multi-touch into their Macs. At present, multi-touch technology is limited to notebook trackpads and the iPhone itself.

Apple, however, is continuing to hire more multi-touch engineers. Their latest job listing is seeking a "Gesture Algorithms Engineer" -- a position that likely didn't exist even a year ago. This potential hire would become part of the engineering team responsible for creating Apple's "next-generation input devices and displays". Apple has been hiring Multi-Touch specialists for almost a year now, and previous patent applications show the possibilities of what could be accomplished with advanced gesture recognition.


Article Link: Apple's Multi-Touch Designer Describes His Inspiration, More to Come?

[yoyo5280]

Multi Touch Macs sometime soon?


Somehow I don't think thats actually a good idea.

[Frits]

Hope they aint gonna to make a multitouch keyboard or some crap like that.

[talkingfuture]

His comments about the ergonomics are how I feel about using a Mouse at the moment. I'd really like to see a large multi touch surface for my iMac. It would just feel much more natural.

[ihabime]

I wish Apple would re-release fingerworks iGesture pad, that thing was fantastic, you could ditch your mouse and just use a pad that recognized gestures from up to five fingers.

It was pricey, $200-300, but they were a small outfit, I'm sure with the volume that Apple would sell it could be cheaper.

[koobcamuk]

Originally Posted by Frits

Hope they aint gonna to make a multitouch keyboard or some crap like that.

I would like a multitouch keyboard... a secondary panel that can be manipulated by touch.

It's very hard to envisage though...

[dannyboi83]

I can just imagine the usefulness of a large multi-touch pad. It would work great with an iMac and can be used to replace the mouse and keyboard.

I'm up for that, let me know when its out

[bobbleheadbob]

I wonder if multitouch specialists are better in bed?

[dannyboi83]

Originally Posted by bobbleheadbob

I wonder if multitouch specialists are better in bed?

One would think so

[JackSYi]

Awesome article. Where is that multi-touch mouse?

[nicksoper]

I have an external keyboard and laptop stand at the office, but I often just use the laptop on its own because of the multi touch gestures on the track pad.

Also having to switch between two hands on the keyboard, to the mouse all the time is laborious. It's much more graceful on the laptop with the trackpad.

Good on Apple for phasing all this functionality into the products, it gives slow people like me a chance to figure it all out.

[Frits]

Originally Posted by koobcamuk

I would like a multitouch keyboard... a secondary panel that can be manipulated by touch.

It's very hard to envisage though...

What would be good about a touch sensitive keyboard? I mean, what would be the difference? I personally like the crisp feel of real buttons beneath my fingers when I type. Also, it would be harder to spot typo's if you didn't actually feel the button being pressed.

Something as a replacement for the mouse would be nice though. I sometimes find myself typing short lines, like searches, with just one hand, because it's just faster than getting my right hand off the mouse. Maybe the numpad could be replaced with a multi touch surface, that would be way more easily reachable. It still isn't perfect though.

[Otaviano]

I want a tablet!

I want photoshop where I can draw directly on the screen, and manipulate things by touch. I want an iTunes alternative where I can touch my music as it's playing. Scratch on any mp3 like it was a vinyl record. Pitch shift, by moving my finger in a circle faster and slower. Mix on screen, admittingly the crossfader will be hardest to replicate with precision. People say there are not enough mainstream uses for a touch screen tablet, but to me the possibilities are endless. Give me a touch screen tablet, I'm selling my MacBook Pro (bought an iMac on sale) and won't purchase another laptop. I want a tablet.

[gzvinmen]

I don't think thats actually a good idea.
I wish Apple would re-release fingerworks iGesture pad, that thing was fantastic, you could ditch your mouse and just use a pad that recognized gestures from up to five fingers.

[grapes911]

I can't wait until they have an entire keyboard that is multitouch. My Fingerworks pad is starting to go and I need a replacement.

[Stately]

Tablet time. About 4 inches in height aaaand about, 5 inches in width, with some really nice processing power, full OS and enormous battery life. I would do some serious designing with that thing on the go. I could get my documents typed up and edit them. Everything on the fly. My gosh . . and then think if they tagged on phone capability? Uh oh . . A behemoth of a product.

[RichardI]

Every time this multi-touch stuff comes up I get ticked off. Am I the only one who has no interest in learning a whole new sign language? Does everyone out there see this as a step forward - an improvement? I don't. Why not just have a keyboard pop up on the screen that you can then use by touch? This "multi-touch" stuff just seems like a step backwards to me. Lemme see, was that 2 fingers or 3? Swipe left, or right - oops!

Rich

[cnorth3]

Originally Posted by RichardI

Every time this multi-touch stuff comes up I get ticked off. Am I the only one who has no interest in learning a whole new sign language? Does everyone out there see this as a step forward - an improvement? I don't. Why not just have a keyboard pop up on the screen that you can then use by touch? This "multi-touch" stuff just seems like a step backwards to me. Lemme see, was that 2 fingers or 3? Swipe left, or right - oops!

Rich

On the other hand, should we ignore technologies that potentially change the I/O paradigm for the better simply because we are used to a keyboard? I'm old enough to remember when people argued that the mouse was a worthless waste of time.

[notjustjay]

"Input devices" we already knew about. It's the "displays" part of the job description that intrigues me.

[Lepton]

Sony has a little keyboard that clips on the PlayStation 3 game controller, for chatting and such. The reason I mention it is that the little device has a trackpad mode - touch a button to go into trackpad mode, and the keyboard turns into a touch sensitive trackpad! Move your finger lightly over the keys and the PS3 mouse cursor (used in the browser etc.) moves.

Wouldn't this be interesting to see on a real computer device? Gesture right on the keyboard itself...

[GQB]

Originally Posted by Otaviano

I want a tablet!

I want photoshop where I can draw directly on the screen, and manipulate things by touch. I want an iTunes alternative where I can touch my music as it's playing. Scratch on any mp3 like it was a vinyl record. Pitch shift, by moving my finger in a circle faster and slower. Mix on screen, admittingly the crossfader will be hardest to replicate with precision. People say there are not enough mainstream uses for a touch screen tablet, but to me the possibilities are endless. Give me a touch screen tablet, I'm selling my MacBook Pro (bought an iMac on sale) and won't purchase another laptop. I want a tablet.

I agree about the tactile possibilities for a tablet.
Where I think people miss the boat is crying for a touch screen on a notebook or desktop. There's a use for vertical touch at wall mounted kiosk displays, but as a normal way of interacting with a desktop its nuts. Ergonomically, your arms can take no more than a couple of minutes of reaching out to a vertical screen.
I think people got way too taken in by the Tom Cruise interface in Minority Report.

[GQB]

Originally Posted by RichardI

Every time this multi-touch stuff comes up I get ticked off. Am I the only one who has no interest in learning a whole new sign language? Does everyone out there see this as a step forward - an improvement? I don't. Why not just have a keyboard pop up on the screen that you can then use by touch? This "multi-touch" stuff just seems like a step backwards to me. Lemme see, was that 2 fingers or 3? Swipe left, or right - oops!

Rich

Whether or not you personally see the use for you of a new interface (yeah, its a pain to learn new things), why would you be 'ticked off'? Seems a waste of good anger.

[Doctor Q]

"I had an ergonomic problem and I paired it with a motivation," Westerman said of the early inspiration. "I'd always felt that playing the piano was so much more graceful and expressive than using a computer keyboard, and I thought how great it would be if I pulled some of that expression from the piano to the computer experience."

Four more lessons from pianos:

* We use both hands for playing the piano or typing text, but our mice and trackpads are designed for one hand, while our brains are perfectly capable of coordinating two hand movements.

* For the most natural style, keyboards should be wide. Maybe not 8 octaves worth, but for the sake of our laptop sizes we are cramping our hand movements unnaturally.

* Like a piano, or a car, computers could benefit from foot input, maybe pedals. Our feet are idle under the desk while our fingers try to handle all of our interface actions. For example, a foot control that acts like a modifier key would greatly increase the gestures we could make.

* The best piano playing takes practice. So does fast typing. We shouldn't be afraid of new techniques just because we can't master them instantly.

[illegallydead]

Originally Posted by GQB

I agree about the tactile possibilities for a tablet.
Where I think people miss the boat is crying for a touch screen on a notebook or desktop. There's a use for vertical touch at wall mounted kiosk displays, but as a normal way of interacting with a desktop its nuts. Ergonomically, your arms can take no more than a couple of minutes of reaching out to a vertical screen.
I think people got way too taken in by the Tom Cruise interface in Minority Report.

Precisely. In order for something like this to succeed, I see it requiring a huge shift in the design of computers. Think Microsoft Surface. That is a good implementation of a touch interface in a manner that is build solely for multi-touch. You cannot simply take an existing iMac of Macbook, slap a huge touch sensitive space on it, and call it good.
Oh, and call me old fashioned, but I see serious typists staying with the current paradigm of computers, that is, PHYSICAL keyboard, with normal monitor, etc. For these folks, the multi-touch would be limited to something we are talking about, like a large multi-touch pad in place of a mouse or in place of the numpad...

Originally Posted by RichardI

Every time this multi-touch stuff comes up I get ticked off. Am I the only one who has no interest in learning a whole new sign language? Does everyone out there see this as a step forward - an improvement? I don't. Why not just have a keyboard pop up on the screen that you can then use by touch? This "multi-touch" stuff just seems like a step backwards to me. Lemme see, was that 2 fingers or 3? Swipe left, or right - oops!

Rich

The "whole new sign language" is exactly what multi-touch designers DON'T want. This sort of U.I. is meant to feel natural, that is, anyone can come up and immediately start manipulating, say, a set of photos on screen. This is why I am arguing for a sort of direct manipulation like the example of Microsoft Surface, etc...

[Virgil-TB2]

Originally Posted by Doctor Q

... The best piano playing takes practice. So does fast typing. We shouldn't be afraid of new techniques just because we can't master them instantly.

This may be true, but it's also the reason why only some tiny fraction of 1% of humanity knows how to play the piano. Hell, only one out of every five or so of the "techies" I meet even knows how to type. Most just use two or three fingers.

The ideal world where people learn to compute in as complicated and expressive a way as they use a piano is a few hundred years off me thinks. You need a whole better race of humans for such Utopian ideas to really work.