Got a tip for us?
Let us know
Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.
Apple Working on Advanced Gesturing and Multi-Touch Version of iWork
- Thread starter MacRumors
- Start date
- Sort by reaction score
Input method
I think I remember an Apple patent a while back that described handwriting recognition - but was much more sophisticated than trying to recognise what a stylus was doing on the screen. There was no stylus involved - it was a case of "place your hand on the screen and imagine you're holding a pen, now write". It worked by recognising the pad of the index finger and the side of the hand resting on the screen. This is the most intuitive solution I can imagine and it would be THE killer app if it was good. It's also something that would be "surprising" and something that Steve could be "very happy with"
Incidentally, the ability of content creation on the tablet would cannibalise even further from the plain white MacBook. Given Apple's habit of taking stuff away even as they give new products - I would say this gives further weight to my speculative predictions that the tablet will be the next generation MacBook.
I think I remember an Apple patent a while back that described handwriting recognition - but was much more sophisticated than trying to recognise what a stylus was doing on the screen. There was no stylus involved - it was a case of "place your hand on the screen and imagine you're holding a pen, now write". It worked by recognising the pad of the index finger and the side of the hand resting on the screen. This is the most intuitive solution I can imagine and it would be THE killer app if it was good. It's also something that would be "surprising" and something that Steve could be "very happy with"
Incidentally, the ability of content creation on the tablet would cannibalise even further from the plain white MacBook. Given Apple's habit of taking stuff away even as they give new products - I would say this gives further weight to my speculative predictions that the tablet will be the next generation MacBook.
Apple's Fingerworks connection
From beststuff.com regarding Fingerworks stuff before they were bought by Apple:
"Our TouchStream gesture keyboards pack a lot of power," explains Jeff Green, FingerWorks Mac-Products Sales Manager. "People are amazed by all the things a hand gesture user interface provides. We have a large number of easy- to-use gestures that cover just about every common computer operation. Users don't have to reach for hot-keys because gestures are faster and easier to do. Reaches for the touchpad are also eliminated because the pointing function always stays with the fingers. With our new gesture customization utilities, users are able to map any gesture to any function in their favorite application, and our XWinder utility allows users to move and resize windows in one step with hand rotation."
One MacNTouch user commented: "This technology is so cool; I'm surprised Apple didn't come up with it first. It's really nice not having to move my hands off the keyboard to reach for the touchpad anymore, and I just love the gestures. One thing's a little weird though -- typing is done in complete silence -- it's like one of those keyboards you see on Star Trek. "
Here's the MacRumors link for the keyboard circa 2007. Now imagine such a thing popping up as a software keyboard. Seems... Apple.
https://www.macrumors.com/2007/11/01/apple-researching-touch-surface-keyboard/
From beststuff.com regarding Fingerworks stuff before they were bought by Apple:
"Our TouchStream gesture keyboards pack a lot of power," explains Jeff Green, FingerWorks Mac-Products Sales Manager. "People are amazed by all the things a hand gesture user interface provides. We have a large number of easy- to-use gestures that cover just about every common computer operation. Users don't have to reach for hot-keys because gestures are faster and easier to do. Reaches for the touchpad are also eliminated because the pointing function always stays with the fingers. With our new gesture customization utilities, users are able to map any gesture to any function in their favorite application, and our XWinder utility allows users to move and resize windows in one step with hand rotation."
One MacNTouch user commented: "This technology is so cool; I'm surprised Apple didn't come up with it first. It's really nice not having to move my hands off the keyboard to reach for the touchpad anymore, and I just love the gestures. One thing's a little weird though -- typing is done in complete silence -- it's like one of those keyboards you see on Star Trek. "
Here's the MacRumors link for the keyboard circa 2007. Now imagine such a thing popping up as a software keyboard. Seems... Apple.
https://www.macrumors.com/2007/11/01/apple-researching-touch-surface-keyboard/
Just found this link which I thought was worth reposting. This is where they got up to with the Newton before it was canned. Pretty impressive. It would be even more impressive if you didn't have to use a stylus:
http://homepage.mac.com/weeno/.Movies/newtonsayhello.mov
http://homepage.mac.com/weeno/.Movies/newtonsayhello.mov
Wirelessly posted (Mozilla/5.0 (iPhone; U; CPU iPhone OS 3_1_2 like Mac OS X; en-us) AppleWebKit/528.18 (KHTML, like Gecko) Version/4.0 Mobile/7D11 Safari/528.16)
Nope it's more likely to be via the iPod dock which apple have set as their default niw
Nic0 said:
Nope it's more likely to be via the iPod dock which apple have set as their default niw
I remember this page very well a few years ago and have seen images of hand diagrams showing advanced gesture movements. A good friend of mine who worked for IBM back then bought the multi-touch keyboard for his PC and he was very happy with it. The reason the keyboard is in such a niche industry was designed, I believe, originally to relieve people with carpal tunnel's/tendonitis by combining the trackpad and keyboard into one unit.
They had to take the page down right before they got bought out and moved their company within Apple as a new division.
Anyway, I strongly believe the tablet/slate will have more advanced multi-touch techniques that will be EXACTLY the same as the original Fingerworks keyboard, not seen or done on iphone/ipod touches. The keyboard will be the standard QWERTY as usual either as an Ion hard keyboard flipped over on top of it OR as a 'virtual keyboard'. I'm betting it's virtual just like the iphone/ipod touch and other touchscreen devices.
The advanced multi-touch techniques are also used to access hotkeys and shortcuts to get the job done. Now imagine all Photoshop hot-key/shortcuts, within 3 to 5 years, using multi-touch shortcut commands. That's extremely plausible and applicable.
They had to take the page down right before they got bought out and moved their company within Apple as a new division.
Anyway, I strongly believe the tablet/slate will have more advanced multi-touch techniques that will be EXACTLY the same as the original Fingerworks keyboard, not seen or done on iphone/ipod touches. The keyboard will be the standard QWERTY as usual either as an Ion hard keyboard flipped over on top of it OR as a 'virtual keyboard'. I'm betting it's virtual just like the iphone/ipod touch and other touchscreen devices.
The advanced multi-touch techniques are also used to access hotkeys and shortcuts to get the job done. Now imagine all Photoshop hot-key/shortcuts, within 3 to 5 years, using multi-touch shortcut commands. That's extremely plausible and applicable.
Are you suggesting we put our hand on the trackpad, pretending to hold a pen and write? One problem with this idea is that you have to MOVE your hand from left to right just like on paper to formulate a sentence naturally.
It makes no sense to just keep your hand still and "write" because it's counter-intuitive.
It makes no sense to just keep your hand still and "write" because it's counter-intuitive.
That's exactly what it is. Besides, the MacNTouch keyboard was quite expensive at the time. I think it cost close to between $250-$300. And yes, they were AHEAD of Apple on this. If I'm not mistaken, someone else wanted to buy FingerWorks out but Apple decided to make that move quickly. I can't remember exactly but it was a minor story behind the acquisition. When Apple saw what FingerWorks did, they wet their pants and were impressed, and envious in the same time wishing they had developed this originally.
The TouchStream LP was made for the Windows platform and they also designed a Mac version of it. Now, ask yourself this:
"Why did'nt Steve Jobs implement a hard keyboard with multi-touch just like TouchStream on the MacBook Pros or other laptop products??"
Think about it. With flat, hard keys with multi-touch, when you close the lid, you don't have to worry about scratching the glass like the old laptops did when standard keys had to be squeezed in. And plus, no need to worry about keys 'wearing out' over time.
They had FingerWorks right there in their hands in 2007. They could have transitioned that technology to get rid of standard keys to reduce maintenance or manufacturing expenses. Don't get me wrong. He could have done this last year but I'm betting that IF he's that smart enough to see this opportunity, it's a matter of when he's going to either use 'virtual keyboards' or multi-touch keyboards (a la TouchStream) on future laptops.
If he wakes up and smells the coffee, that is. It would be fantastic because it would benefit a lot of users who have cramped hands or have bouts of carpal tunnel's. After all, it would be a sense of corporate responsibility to make things better and productive for us.
The TouchStream LP was made for the Windows platform and they also designed a Mac version of it. Now, ask yourself this:
"Why did'nt Steve Jobs implement a hard keyboard with multi-touch just like TouchStream on the MacBook Pros or other laptop products??"
Think about it. With flat, hard keys with multi-touch, when you close the lid, you don't have to worry about scratching the glass like the old laptops did when standard keys had to be squeezed in. And plus, no need to worry about keys 'wearing out' over time.
They had FingerWorks right there in their hands in 2007. They could have transitioned that technology to get rid of standard keys to reduce maintenance or manufacturing expenses. Don't get me wrong. He could have done this last year but I'm betting that IF he's that smart enough to see this opportunity, it's a matter of when he's going to either use 'virtual keyboards' or multi-touch keyboards (a la TouchStream) on future laptops.
If he wakes up and smells the coffee, that is. It would be fantastic because it would benefit a lot of users who have cramped hands or have bouts of carpal tunnel's. After all, it would be a sense of corporate responsibility to make things better and productive for us.
From beststuff.com regarding Fingerworks stuff before they were bought by Apple:
"Our TouchStream gesture keyboards pack a lot of power," explains Jeff Green, FingerWorks Mac-Products Sales Manager. "People are amazed by all the things a hand gesture user interface provides. We have a large number of easy- to-use gestures that cover just about every common computer operation. Users don't have to reach for hot-keys because gestures are faster and easier to do. Reaches for the touchpad are also eliminated because the pointing function always stays with the fingers. With our new gesture customization utilities, users are able to map any gesture to any function in their favorite application, and our XWinder utility allows users to move and resize windows in one step with hand rotation."
One MacNTouch user commented: "This technology is so cool; I'm surprised Apple didn't come up with it first. It's really nice not having to move my hands off the keyboard to reach for the touchpad anymore, and I just love the gestures. One thing's a little weird though -- typing is done in complete silence -- it's like one of those keyboards you see on Star Trek. "
Here's the MacRumors link for the keyboard circa 2007. Now imagine such a thing popping up as a software keyboard. Seems... Apple.
https://www.macrumors.com/2007/11/01/apple-researching-touch-surface-keyboard/
It did work a few days ago (it was the 6th) - dang, i had to get up and do something and didn't save it.
I agree that it is "strange" that it is gone now..... (actually, in looking at my history it was "gesture_keymap" that was posted before, and it's now gone too.)
All i have is this mini screen snap from the cache.... and gee, you can't see jack on it! LOL!!
Attachments
I am getting pretty excited by the Tablet, but I don't see myself using it instead of a laptop. I may get it for other reasons, but iWork would have to be pretty amazing on it to get me to stop carrying my Macbook Pro to work.
I really never understood why Apple doesn't include iWork for free when someone purchases a new Mac. It would add to the adoption rate of iWork rather than having a trial expire and then they just buy Microsoft Office (I have both, but I use iWork for almost everything).
I really never understood why Apple doesn't include iWork for free when someone purchases a new Mac. It would add to the adoption rate of iWork rather than having a trial expire and then they just buy Microsoft Office (I have both, but I use iWork for almost everything).
That's the one I saw . I think they were afraid to reveal any surprises but it's really old news to those who owned the TouchStream keyboards. Those who owned the keyboards are ahead of the iphone and 'touchscreen' generation and have greater familiarity with it.
I suspect that if Apple wakes up and smells the coffee, they should seriously consider implementing the 'hard keys' on the laptops and with new 'bumps' that Steve Jobs was talking about for 'haptic feedback', if I understand this right. It can be done. Otherwise, ten years from now, the tablet could replace their laptop line entirely possibly.
They could develop separate haptic feedback keyboards for iMac and Mac Pros as well. Why not, right? DITCH the Magic Mouse and use the combo "TouchStream" version.
EDIT: Case in point, Wacom's Bamboo Touch is a sign of that general direction where things are going in the industry. It's a touchpad for finger gestures when operating Photoshop or other applications and it can also be used to draw/illustrate designs or manipulate imagery. That's why I think Magic Mouse is pointless when a 'touch' keyboard makes more sense.
. I think they were afraid to reveal any surprises but it's really old news to those who owned the TouchStream keyboards. Those who owned the keyboards are ahead of the iphone and 'touchscreen' generation and have greater familiarity with it. I suspect that if Apple wakes up and smells the coffee, they should seriously consider implementing the 'hard keys' on the laptops and with new 'bumps' that Steve Jobs was talking about for 'haptic feedback', if I understand this right. It can be done. Otherwise, ten years from now, the tablet could replace their laptop line entirely possibly.
They could develop separate haptic feedback keyboards for iMac and Mac Pros as well. Why not, right? DITCH the Magic Mouse and use the combo "TouchStream" version.
EDIT: Case in point, Wacom's Bamboo Touch is a sign of that general direction where things are going in the industry. It's a touchpad for finger gestures when operating Photoshop or other applications and it can also be used to draw/illustrate designs or manipulate imagery. That's why I think Magic Mouse is pointless when a 'touch' keyboard makes more sense.
I think a big part of the "magic" most people are missing is the back of the device will be like a magic mouse. It's where your fingers are anyway, it is an entirely new paradigm as compared to any mass-marketed product, and thus would require a "learning curve".
Touch sensitive rear cover. Hence the debate over what material it will be. I am betting it interferes with radio signals, like a screen door does.
Some of the older tablet patents showed an optical drive door in the center of the back of the device. If this device had a dock connector in the center of the back, it would "hang" on the dock like an iMac does from its stand. The dock could have all the physical connections needed. Or there could be an Airport/Apple TV type box it talks to by wifi.
Scoop Rocketman
Touch sensitive rear cover. Hence the debate over what material it will be. I am betting it interferes with radio signals, like a screen door does.
Some of the older tablet patents showed an optical drive door in the center of the back of the device. If this device had a dock connector in the center of the back, it would "hang" on the dock like an iMac does from its stand. The dock could have all the physical connections needed. Or there could be an Airport/Apple TV type box it talks to by wifi.
Scoop Rocketman
I hoping they go out on the limb with this - i can see the debate on whether the "real world" is ready for such a thing going on for the last few years. Something like that would take a pretty big leap of faith that the general population would adopt it. I think that today, with the amount of small touch screen devices - we are there.
One can really hope so anyway!
A simple question for you: Would you record your music on a boombox tape recorder? If not, then why would you even want to try doing high-end audio on such a limited device as even the so-called iSlate is going to be? Your commentary lacks common-sense logic.
You seem to miss what market a device like this should fill--the space between a smart phone and a full-fledged desktop computer. Basically, a good tablet/slate device should handle the portable data-entry niche where it can then synchronize with a stationary desktop either wirelessly or through a docking device. Envision a job site foreman who carries this on (or as part of) his clipboard, having access to engineering drawings and inspection forms for gestural and touch input. Using either a finger or a specific type of stylus, handwritten notes could be added and, if another of Apple's patent applications is true, converted to textual notations in that same documentation.
Finally, do you really want 'openness?' I would think that apps written specifically for such a device's OS and features would be superior to those written for a generic device.
Which is exactly where a tablet/slate device should be.
As I commented on another thread, you don't want a full OFFICE on such a device. What you want is something like Works, which gives you the basic capabilities without all the fluff and bloat that Office (whether MS, Star or Open) contains. As someone else mentioned, at least for now, capacity is the limiting factor, and to get Solid-State storage capacity, you're going to be spending some money--at least for now. I expect the first couple versions of Apple's device will probably top out at 80GB or so. While that may sound like a lot, if you stuff a full version of Office, your music library and some of the other apps people have mentioned here, that 80 Gig will be swallowed up in no time. Just as an example, my own music library alone takes over 78 Gig. Obviously I wouldn't even want to put my full library on a portable device like that. For one thing, the battery would hardly be able to play for 41 days straight.
If a buyer wants to pay the premium that Microsoft charges for Office:mac over the price of iWork, then the buyer wants Office:mac. Look. The success of Apple computers depends in large measure on a healthy developer community supporting its platforms.
The original Mac shipped with MacWrite and MacPaint. Many elements of the developer community and some commentators howled like banshees in protest. OTOH, Apple must ensure that its customers have access to the tools to do useful work. It's a balancing act. If no third party is willing to develop the software the take advantage of the capabilities of the Mac, then Apple steps in to fill the breech.
Microsoft is the largest Mac developer outside Apple. Apple does not intend to drive Microsoft away. iWork does not compete with Office:mac. If the day comes when it does, then for whatever reason it will be a bad thing.
You are both right and wrong here. Yes, all current tablets are inferior--not because of the touch technology, but because the operating systems used and the applications installed were not written with the touch interface in mind; they all use the touch as a pointing device, not as a true interface in itself.
You're definitely wrong about how tablet devices are aimed, too. You make the assumption that since iPhones, ebook readers and other similarly-sized devices are currently made for that niche, then all such devices can only be used in that niche. That's like saying smart phones are only good for enterprise use. Open your mind and look at where such a device could really be used. You might be surprised.
Think about what you just said here. iPhoto, iMovie, even iWork are streamlined, simplified versions of much more complex software. A tablet/slate with iLife and iWork installed would almost perfectly fit the niche of truly portable computing over and above the content consumption market that so many commenters here want to restrict it into.
As a writer, I could strongly disagree with you here. I don't get my story ideas by just sitting in front of a keyboard all day; in fact, most of my ideas come to me in dreams and the quickest way to get these down is by having a voice recorder close to hand. Now, if that voice recorder happened to have the ability to convert speech to text (something I don't yet expect, by the way) then this device would be almost perfect for my needs.
A very nice idea, but how many of us know how to read/write in Braille? I could see this device as potentially able to read Braille input in the way you describe, but unless you're blind, you're right that the learning curve would be steep.
For one thing, if the iPhone's virtual keyboard scales with screen size (as it does in some apps between Portrait and Landscape orientation,) then the virtual keyboard may be all you need for casual, short-term typing. I would also expect (and others have expressed this idea as well) that Apple's Bluetooth keyboard or maybe a USB keyboard will be compatible as well, considering the overall concept of this device. However, it seems you've missed the fact that Apple has patented (or an application for one, anyway) the means for a device to read strung-together characters (handwritten?) and recognize them as words rather than single characters, something like the way you are taught to touch-type by thinking the word, not the spelling. Hunt and peck may be good enough for some people, but with computers becoming as ubiquitous as they are in the modern world, touch-typing is not just for secretaries any more.
The point is that Apple's device (almost certainly to be announced on the 26th/27th) may allow for virtual keyboard, real keyboard and handwriting input. The first and last are practically certain, the second a good possiblity.
Well, as a fellow human being who is forced to share said commuter train, plane, etc. with you, I would appreciate if you keep your ideas to yourself. If I want to be subjected to them I'll buy your book. I don't want to have to listen to you speak them aloud while trapped with you in an enclosed space.
You seem to be assuming that Voice to Text is all I want. I agree with you that there's a time and place for everything--that includes Voice to Text. On the other hand, at the projected size of the face of this device, a virtual keyboard would serve for all those times when voice won't.
I am making no such assumption about what you want. You responded to my comment about text-to-speech being inappropriate in many places by saying you disagreed with me. Hence I responded on that point.
Wirelessly posted (Mozilla/5.0 (iPhone; U; CPU iPhone OS 3_1_2 like Mac OS X; en-us) AppleWebKit/528.18 (KHTML, like Gecko) Version/4.0 Mobile/7D11 Safari/528.16)
In the same way as the iPhone does. It runs a BSD based os which has roots in OSX but has been customized for the iPhone hardware an UI. If you jailbreak the iPhone then you get all the unix access and so forth.
The slate will be the same it will be BSD based but customised for the slate. Where we get access to the underlying OS or not will remain to be seen. I suspect not!
---Zed
macuser154 said:
In the same way as the iPhone does. It runs a BSD based os which has roots in OSX but has been customized for the iPhone hardware an UI. If you jailbreak the iPhone then you get all the unix access and so forth.
The slate will be the same it will be BSD based but customised for the slate. Where we get access to the underlying OS or not will remain to be seen. I suspect not!
---Zed
Ah, the ever-tired, ever-bogus argument that the superiority of Apple products over Microsoft's implies perfection (or that they can't be tweaked by power users who so choose).
Is there a standard list of talking points the Winbots are issued by Redmond HQ? Wait, I think I hear your fax machine ringing now.
P.S. the period should appear inside the quotation mark.
Ah, the ever-tired, ever-bogus argument that the superiority of Apple products over Microsoft's implies perfection (or that they can't be tweaked by power users who so choose).
Is there a standard list of talking points the Winbots are issued by Redmond HQ? Wait, I think I hear your fax machine ringing now.
Costs more
Doesn't "just work"
Less choice
Form over function
Unacceptable for business
Not "open" like Windows
Snobbish status symbol
Cult of Steve
Not as good as alternatives because fewer features, but brainwashed "fanbois" can't see through RDF
P.S.: the period-before-quotes thing was discussed in the NY Times recently. Apparently in England they prefer quote followed by period, and that's been catching on in the U.S. due to the use of computers, but I can't remember why. Something to do with detecting the end of a sentence more easily.