Got a tip for us?
Let us know
Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.
Apple Researching Physical Keyboards with Motion Control to Replace Mouse
- Thread starter MacRumors
- Start date
- Sort by reaction score
It's inevitable
I thought Apple might try a full-sized virtual keyboard, because it would allow instant international keyboard layout changes and could serve as a secondary touchscreen. But that would force users to look at the keyboard far more often. And physical keys, even extremely short-action keys, are what people are used to.
Still, detecting multi-touch gestures right on the keyboard would be a huge win. The next time you're at your desktop Mac for more than a few minutes, just keep a mental total of the number of times you need to swing your hand back and forth between keyboard and mouse. And how far that movement is.
If you're right-handed and you have a full-sized keyboard with numeric keypad, that distance is about 1 foot. And it takes at least a second to move and get set each time. Over a year, that could add up to miles of travel and hours of travel time.
Oh, and I seem to remember another Apple patent covering multi-touch gestures *above* a surface. Combine this keyboard / multitouch patent with that patent and you can see how a Mac could instantly detect typing vs. gesturing. If any one of your fingers is touching the keyboard, the system could assume you're typing. If none of your fingers is touching the keyboard and you're moving one or more fingers within 2 inches of the keyboard, the system could assume you're gesturing. Fingers that aren't moving could be ignored. Just a thought.
I thought Apple might try a full-sized virtual keyboard, because it would allow instant international keyboard layout changes and could serve as a secondary touchscreen. But that would force users to look at the keyboard far more often. And physical keys, even extremely short-action keys, are what people are used to.
Still, detecting multi-touch gestures right on the keyboard would be a huge win. The next time you're at your desktop Mac for more than a few minutes, just keep a mental total of the number of times you need to swing your hand back and forth between keyboard and mouse. And how far that movement is.
If you're right-handed and you have a full-sized keyboard with numeric keypad, that distance is about 1 foot. And it takes at least a second to move and get set each time. Over a year, that could add up to miles of travel and hours of travel time.
Oh, and I seem to remember another Apple patent covering multi-touch gestures *above* a surface. Combine this keyboard / multitouch patent with that patent and you can see how a Mac could instantly detect typing vs. gesturing. If any one of your fingers is touching the keyboard, the system could assume you're typing. If none of your fingers is touching the keyboard and you're moving one or more fingers within 2 inches of the keyboard, the system could assume you're gesturing. Fingers that aren't moving could be ignored. Just a thought.
I understand that, but noone is going to want to hover their fingers just above the key surface. They will undoubtedly lightly rest their fingers on the surface just as you would with a trackpad. Try sliding your fingers over your keys while lightly touching them. It's not the experience I would want. I guess it would depend on how high you could hover your fingers but I would rather have them resting on a surface.
The mouse is ahead of it's time. Still has many many many years left in the little device. The reason it's so hard to replace is because you can leverage a lot of screen movement with very little physical movement. Using a laptop touchpad will never have the precise control movement and speed of a mouse. Try playing quake3 with a touchpad. The secret is the mouse leverages the wrist and the touchpad uses your finger(s).
You can activate control instantly and release in a split second. Motion control systems you have to have an on mode and an off mode whereby every single motion will be tracked and most of the time you want to move freely and not have every action tracked. Even the Minority report freehand gestures are not as efficient as a mouse. The mouse is king.
You can activate control instantly and release in a split second. Motion control systems you have to have an on mode and an off mode whereby every single motion will be tracked and most of the time you want to move freely and not have every action tracked. Even the Minority report freehand gestures are not as efficient as a mouse. The mouse is king.
This is a terrible idea, just like the iPod, and a phone with only a single button on its face. It will never sell. 
fingerworks user here
I used a fingerworks touchstream for a few years before it broke - and I can tell you that it is quite intuitive to switch from typing to mousing. In fact there is no 'mode' to switch from.
the 'gesture' to type is one finger making a vertical movement on a key.
the 'gesture' to mouse around (at least on the fingerworks) was two fingers down in close proximity.
Anytime the keyboard processor saw two fingers down - it interpreted it as a mouse event. If only one finger was down - it was a keypress.
quite natural actually - put you fore and index finger together and run it along the top of your keys - that would be a mouse pointer movement.
fingerworks sequestered mouse movements for the right hand, and cursor keys for the left hand (up arrow, down arrow, etc)
ok,
I used a fingerworks touchstream for a few years before it broke - and I can tell you that it is quite intuitive to switch from typing to mousing. In fact there is no 'mode' to switch from.
the 'gesture' to type is one finger making a vertical movement on a key.
the 'gesture' to mouse around (at least on the fingerworks) was two fingers down in close proximity.
Anytime the keyboard processor saw two fingers down - it interpreted it as a mouse event. If only one finger was down - it was a keypress.
quite natural actually - put you fore and index finger together and run it along the top of your keys - that would be a mouse pointer movement.
fingerworks sequestered mouse movements for the right hand, and cursor keys for the left hand (up arrow, down arrow, etc)
yea - you don't want to eliminate the keys - that was the shortcoming of the fingerworks - you can run your fingers across keys without pressing them - it allows yours fingers/wrist to 'rest' - thus improving RSI resistance. the ps3 keyboard is similar.
Depends if you are a touch typist: moving your fingers from home row is bad all around. Think of fighter jets - they put the essentials controls on the throttle and stick (HOTAS-> hands on throttle and stick) and use gestures (buttons hi-hat, etc) to allow control of radar and other flight essential systems without having to lean forward and take your hands off the task at hand.
kind of insulting to a large portion of the world that uses chop sticks - just saying.
Precision with any digital input device takes practice / training and technique - my trackpad is extremely precise by rolling my fingers (versus dragging my fingers) on the surface when I want to be very accurate. I find it more accurate than a mouse/pen/stylus.
nah, fingerworks already had it working fine years ago
fingers will rest on the keys - reduces fatigue in the arms/wrist.
It took me a few hours to get used to not grabbing for a mouse on the side of the keyboard - but other than that - it would be very easy. the trouble is using someone else's computer and having to grab that mouse again - it makes it quite evident how inefficient mousing is.
proper typing means the eyes stay on the screen, not the keyboard - of course that assumes proper typing.
Wirelessly posted (Mozilla/5.0 (iPhone; U; CPU iPhone OS 4_1 like Mac OS X; en-us) AppleWebKit/532.9 (KHTML, like Gecko) Version/4.0.5 Mobile/8B117 Safari/6531.22.7)
I think that the only thing this does is allow for a device to be smaller, by eliminating a trackpad. it doesn't seem to have any ergonomic benefit since the surface of keys is up and down and fingers will only get caught on the edges unless you hover with them. Need a completely smooth surface for trackpadding..
I think that the only thing this does is allow for a device to be smaller, by eliminating a trackpad. it doesn't seem to have any ergonomic benefit since the surface of keys is up and down and fingers will only get caught on the edges unless you hover with them. Need a completely smooth surface for trackpadding..
Wirelessly posted (Mozilla/5.0 (iPhone; U; CPU iPhone OS 4_1 like Mac OS X; en-us) AppleWebKit/532.9 (KHTML, like Gecko) Version/4.0.5 Mobile/8B117 Safari/6531.22.7)
Trackpad beats mouse in terms of pointer manipulation, try it JUST for pointing in a game. Trackpad doesn't work for games because of the lack of requisite 2-3 buttons that would not also interfere with pointing.
tdream said:
Trackpad beats mouse in terms of pointer manipulation, try it JUST for pointing in a game. Trackpad doesn't work for games because of the lack of requisite 2-3 buttons that would not also interfere with pointing.
That makes me wonder... I'm I the only one still doing Photoshop clipping with a mouse?
Trkbrd
Apple may have purchased this technology or stole it, either way it surfaced as one of the presenters Master's Project in last years Interaction10 conference in Savanah, Georgia. http://www.trkbrd.it/
"The TRKBRD design was born out of my final project for the 1st year of the Interaction Design Masters program at Malmö University, Sweden. What started as using a physical prototype to illustrate a new idea, has turned into a crusade to see the design brought to reality. Apple? Dell? HP? LG? I'm talking to you here... I want to see a TRKBRD in your next laptop or netbook!"
Apple may have purchased this technology or stole it, either way it surfaced as one of the presenters Master's Project in last years Interaction10 conference in Savanah, Georgia. http://www.trkbrd.it/
"The TRKBRD design was born out of my final project for the 1st year of the Interaction Design Masters program at Malmö University, Sweden. What started as using a physical prototype to illustrate a new idea, has turned into a crusade to see the design brought to reality. Apple? Dell? HP? LG? I'm talking to you here... I want to see a TRKBRD in your next laptop or netbook!"
I think a lot of people would disagree with you, myself included. I've no problem picking a pixel out on a 30 inch screen. Have you seen quake 3 players or any good counterstrike/fps players for that matter. They can pick out a pixel in a split second. Modern mice have a sensitivity switch for increased precision depending on the situation. Sniping mode vs normal.
To replace something it has to be better, much better. Not equal or different, it has to be better. Because everyone knows how to use a mouse, it's simple to use. That's why fingerworks didn't take off and neither will this. Touchscreen works well in a mobile environment, but at a desk nothing on the market is better than a keyboard and mouse. And a fingerworks clone isn't going to better that it either.
Last edited:
Don't think there is 'one input device fits all' - the current combination of keyboard and extra mouse is a good compromise. But depending on what you do on the computer it might be more mouse centric or more keyboard centric. Good news is that Apple is researching in all directions (the keyboard from this article, the mouse with the lcd panel, ....). Someday they might come up with something that is better for some group of people - maybe not everyone, but for those it is designed for perfect. Just because something does or does not work for you, does not mean that it is the same for everyone else - and don't worry, apple won't forget your use case.
Not sure ....
I really like the idea, in theory ... but I think there might be problems with the idea of having one's hand hover over the tops of the keys to do the mouse gestures. People like the tactile feeling of actually pressing/touching something when they're performing actions on a device.
Just from pretending to have such a thing on the keyboard sitting at my desk right now? I find that something like a long, vertically-oriented button/bar just to the left of the Tab, Caps-Lock and left Shift keys might work well as something you hold down with the side of your left-hand pinky finger while using the right hand to do the mouse motions. But it feels odd trying to pretend to use a mouse scroll-wheel or something with my hand, while not physically touching anything.
Maybe they can come up with a new variation of the chicklet keyboard keys that feel ok as keys you press in groups at a time with your fingertips as you swipe around the keyboard layout?
I really like the idea, in theory ... but I think there might be problems with the idea of having one's hand hover over the tops of the keys to do the mouse gestures. People like the tactile feeling of actually pressing/touching something when they're performing actions on a device.
Just from pretending to have such a thing on the keyboard sitting at my desk right now? I find that something like a long, vertically-oriented button/bar just to the left of the Tab, Caps-Lock and left Shift keys might work well as something you hold down with the side of your left-hand pinky finger while using the right hand to do the mouse motions. But it feels odd trying to pretend to use a mouse scroll-wheel or something with my hand, while not physically touching anything.
Maybe they can come up with a new variation of the chicklet keyboard keys that feel ok as keys you press in groups at a time with your fingertips as you swipe around the keyboard layout?
Don't get me wrong, it probably was way ahead of its time.
But - based on my usage a long time ago - I don't think you could whip across the screen and grab a small on-screen item as quickly as accurately as with a mouse or trackpad. Maybe I'm just not a nipple-expert.
Yep. As do other companies.
My current favorite touch-keyboard patent application is from Microsoft:
You know how people complain that virtual keyboards don't fit them, or that it's hard to touch type, especially if you have to lift a hand and then put it back down again?
Microsoft's idea was that wherever you lay your primary fingers on the screen, determines where and how big the keyboard halves are. So you just place each hand down with their four typing fingers, and the virtual keys appear underneath them. Instead of you having to make sure you are on the keys, the keys adapt to your finger placement and size.
--
After decades of using voice, touch, mice, pads, etc, one thing is clear:
There is no one perfect input device. A combination is good, but they should be aware of each other, just as humans combine multiple senses to communicate with each other.
Don't be silly, it comes with a built-in Maid Bot, cleans your keys while you sleep. Only adds $2,000 to the total.
The Perfect Apple Keyboard
Step 1: Take regular Apple extended-style keyboard.
Step 2: Replace numeric keypad area with Magic Trackpad surface.
Step 3: ????
Step 4: Profit!
Step 1: Take regular Apple extended-style keyboard.
Step 2: Replace numeric keypad area with Magic Trackpad surface.
Step 3: ????
Step 4: Profit!
I'm sure this will be an awesome product when the time is right. As with the majority of other Apple patents, I don't think this concept will materialize in the near future. Apple is a master at not jumping the gun and waiting for new technology to mature. This thing doesn't seem ready for primetime, but then again, you never know with apple.
E-Nuff already!!!!
I don't mind Apple spending money and wasting time researching better interfaces for say iPads, iPhones, and other "touch" devices. I think they have some some clever things thus far with their touch technology.
However, for computer input, there is nothing quicker and easier than the good old mouse and keyboard. Period. I bought the Magic Trackpad and tried to like it. I mean, I like the touchpad of my MBP, but use it only when I have to and if I do have space to drop my mousepad and mouse, I will always prefer a mouse. The mouse has pretty much reached the zenith in its evolution, but it still does the same basic thing, clicks and moves a pointer. No matter what they do to make that happen differently, it still comes down to pointing and clicking. And that is the way it should be.
Why do I want to be moving my hands/fingers to get my computer to do something that I can do with a flick of my wrist in a fraction of the time and be much more accurate. For browsing the web, I guess touch is ok, but for actually getting things done or working, you can't beat a mouse/keyboard.
I want to put my grubby hands "less" on my hardware, not more. It's bad enough having to keep glass iPhones and glass trackpads clean, without having to worry about all this other "touch" tech that Apple may want to implement into the Mac.
Mouse + Keyboard > Touch/Motion for Mac
I don't mind Apple spending money and wasting time researching better interfaces for say iPads, iPhones, and other "touch" devices. I think they have some some clever things thus far with their touch technology.
However, for computer input, there is nothing quicker and easier than the good old mouse and keyboard. Period. I bought the Magic Trackpad and tried to like it. I mean, I like the touchpad of my MBP, but use it only when I have to and if I do have space to drop my mousepad and mouse, I will always prefer a mouse. The mouse has pretty much reached the zenith in its evolution, but it still does the same basic thing, clicks and moves a pointer. No matter what they do to make that happen differently, it still comes down to pointing and clicking. And that is the way it should be.
Why do I want to be moving my hands/fingers to get my computer to do something that I can do with a flick of my wrist in a fraction of the time and be much more accurate. For browsing the web, I guess touch is ok, but for actually getting things done or working, you can't beat a mouse/keyboard.
I want to put my grubby hands "less" on my hardware, not more. It's bad enough having to keep glass iPhones and glass trackpads clean, without having to worry about all this other "touch" tech that Apple may want to implement into the Mac.
Mouse + Keyboard > Touch/Motion for Mac
NO. Neither are the old trackball input devices. Just because something also co-exists doesn't mean it has replaced something as the de facto standard. *Many* more people use mice than trackpads.
Touch? I could see getting touch to be a good alternative. But motion? So you have to move your hands in fluid motions, no shaking, etc. Plus, I'm going to remember 100 different gestures just to get around my desktop? No thanks.