Using iPhone: Text and Data Entry vs T9, Graffiti, Thumb Keyboards
July 17th, 2007
Daniel Eran Dilger
Apple’s iPhone eschews the two dominant traditions in mobile text entry–ink and mini keyboards–to deliver a unique way to enter text. Here’s why Apple chose a different path, some advantages and disadvantages, a few hints and tricks, and a look at how iPhone text entry might develop in the near future.
The False Future of Pen Computing.
Back in the early 90s, the buzzword in mobile devices was Pen Computing. We imagined a world where digital pens wrote on digital pads, replacing any need for our clunky keyboards and restoring the traditional method of inscribing our ideas with a handheld stylus rather than a mechanical contraption designed for typewriters.
This sounded great on digital paper, but the entire concept ended up in the digital trash. A series of attempts to deliver a pen driven interface failed, including Go Computing’s PenPoint OS and Apple’s Newton. It turned out that a number of problems stood in the way of the digital pen:
• accurate handwritten recognition required more power than early devices could deliver.
• a pen is not very fast for general purpose text input when compared to a keyboard.
• it is difficult to provide a simple, small, and effective user interface for pen input.
The failure of pen input was largely associated with the Newton Message Pads, which were the first mainstream devices that actually delivered pen input with real handwritten recognition.
After an initial failure with the Casio Zoomer PDA, Jeff Hawkins began work on a simplified pen input system for the Newton to provide an alternative to the often frustrating text recognizer in the first version of the Newton OS.
Graffiti and Jot.
The result was Graffiti, a system of simplified printing that that substituted quick gestures in place of complete letters. It greatly reduced the complexity of the recognizer, enabling smaller, lower powered devices to be able to accurately input text via a stylus.
Hawkins later developed the Palm Pilot, which incorporated Graffiti as its main text input system. It provided an on screen keyboard too, but avid Palm users found it easier and faster to learn the Graffiti gestures than to poke out letters on a keyboard using the stylus.
After Apple pulled the plug on the Newton in 1998, the much cheaper Palm Pilot hit an apex of popularity around 2000, then began to fade as PDA functions began to appear in mobile phones.
Palm was sued by Xerox for patent infringement involving Graffiti, and was forced to replace it with a competing input technology called Jot.
Palm sold Jot under the name Graffiti 2 while it appealed the Xerox patent claim, which was eventually found invalid due to the existence of prior art.
By the time Graffiti was exonerated, it had become obsolete. The standalone PDA was already dead, and Palm was moving its business to follow the demand among mobile phones.
Mobile Mini Keyboards.
Phone users wanted a faster input system than a user wielding a stylus could manage. Palm’s Treo line presented a mini physical keyboard designed to be typed using thumbs. This was commonly regarded as better at entering text than using a stylus, and few people bothered to install the Graffiti software on it.
Nearly all mobile phones now have either a mini QWERTY keyboard, or a numeric keyboard using T9 text input, where numbers are typed in rapid sequence to spell out text. The teenagers who quickly picked up Graffiti in the late 90s were replaced with a new generation of kids who could fire out hundreds of text messages using rapid number pokes or thumb keyboards.
Of course, while it’s the kids who pick things up quickly, it’s the adults who have the money to buy new things. Little progress has occurred in the area of text input on mobiles because of the dysfunctional mobile market.
Progress Held Up By the Dysfunctional Mobile Phone Market.
Innovation in the phone handset business has been driven into the ground by mobile service providers who lure in new customers with ultra low handset prices.
Particularly in the US market, the actual cost of service plans are left undisclosed in most advertising, perpetuating a bait and switch fraud where customers are tantalized with $50 phones, only to be unwittingly signed up to high priced, long term service contracts.
The service providers don’t want to compete on service price, they want to compete in the number of new subscribers gained. This creates a market where false advertising reigns supreme and subsequently prevents competition from introducing real choice and lower prices.
A side effect to this fraudulent market of fixed service prices is that there is also limited competition among phone makers, because the service providers sell nearly all the phones. Since they only want to sell cheap phones to gain new subscribers at their set rates, the choice and sophistication of phones in the US market has grown stagnant.
Phone makers like Motorola have introduced hit phones like the RAZR at initial price points of $500, only to see the price drop into the $50 range as they are quickly turned into a commodity object used by service providers to sign up new subscribers. The system encourages cheap phones that don’t do anything novel.
Apple’s iPhone introduces a number of innovations designed to launch it as a competitive product into a non-competitive market with significant price barriers. One of the main differences of the iPhone is its touchscreen text input.
This greatly simplifies its overall appearance and manufacturing, turning the iPhone into a software-based tablet rather than a complex mechanical device devoting much of its face to more than 40 illuminated mini keys.
The iPhone’s multitouch screen is reminiscent of the Newton, but rather than using a pen based Ink or Graffiti stylus input system, it works using finger touches. A metal or plastic stylus won’t do anything on the iPhone, because it only responds to specific changes in capacitance over its surface, triggered with a bare finger.
Other touch input screens can also respond to a finger press, but they are much less accurate. For example, the Palm Treo screen can be touched, but its too difficult to accurately hit the tiny targets developed by Palm for use with a sharp-tipped stylus. It’s also difficult to apply consistent pressure with a finger tip.
The iPhone solves this using a new breed of touchscreens and a smart layer of touch logic that is designed to predict which targets the user is aiming at and adapt target sensitivity in response. The system is also designed to incorporate large targets that are forgiving by nature. It also incorporates simple gestures such as the finger slide, which is easy to input with a finger but difficult to enter accidently.
iPhone Keyboard Text Entry.
The iPhone has a great demo, but how well does its keyboard work in reality? I’m a fair typist on a standard keyboard, and felt proficient on the Treo mini keyboard. I was once very good at Graffiti, but I have never liked T9. I also have big hands. I found myself typing faster on the iPhone onscreen keyboard than I could on the Treo’s physical keyboard, after only using it for a few days.
I talked to a handful of new iPhone users, including a proficient T9 user with small hands and a Windows Mobile user with average sized hands, and all agreed that they quickly got up to speed on the iPhone and were using it faster than their previous phone within a week.
In my initial review, I complained about there being no sideways keyboard in any other apps beyond Safari, but I’ve since decided that it’s actually more difficult to type using the horizontal keyboard in landscape orientation, despite the keys being larger.
That’s because keys are closer together, especially in their vertical margins. I can type much faster and more accurately in vertical mode than I can holding it sideways. Who would have thought that? Held vertically, the keyboard also takes up less of the screen’s real estate as well, so you can see more of what you are editing.
Compare the vertical margins between keys and the dark document area exposed by each keyboard orientation:
At first blush, it appeared that using the iPhone’s onscreen keyboard required more attention. Each key animates to indicate it has been hit. That’s critical, since there is no tactile feedback when poking at a flat screen.
After using the iPhone for two weeks, I was surprised to return to my Treo thumb pad and discover how clumsy it felt. The tactile feedback from the Treo’s mini keys was nothing like I seemed to remember. Individual keys were too small to distinguish between, even in a coarsely relative sense. Treo keys are also smooth and rounded, making the utility of touch feedback entirely imaginary.
My recollection of being able to blindly type on its keyboard was also simply a false memory. I had to watch the keyboard just as closely as the iPhone, if not more. Part of the reason for this is because the iPhone corrects words behind the scenes, making it easier to accurately enter text at dangerously sloppy high speeds.
iPhone Keyboarding Tricks.
• Most apostrophes are auto suggested, but it doesn’t auto suggest it’s when you type its, because you might actually mean its. To push the it’s suggestion, type itsa.
• Similarly, hell won’t autosuggest he’ll, but typing an extra L, helll, will.
• Once you use a word, the iPhone will remember it and continue to suggest it when you type something close.
• By default, the system auto capitalizes words when it assumes a new sentence is starting. This can be turned off in Settings, but if you find yourself wanting to override it occasionally, you’ll have to start the word with another character and then delete it afterward. [Update: Eric Merrill points out its even simpler than that; the shift key illuminates when it thinks it needs to auto capitalize, so you can simply tap shift to override it. Palm OS does something similar, but requires triple tapping shift. Merrill notes that Windows Mobile auto capitalizes after the word is entered, requiring a manual fix.]
• After typing the wrong letter, you can slide your finger to the correct letter in the same motion. However, if you’re typing fast and hit a wrong letter, the iPhone will commonly guess which letter you intended to hit.
•To avoid switching back and forth between [ABC] and [.?123] modes when typing, you can touch the [.?123] punctuation key and slide to the desired character in one motion. After typing it, the keyboard returns to ABC. [Update: this is actually slower than hitting individual keys in rapid sequence.]
•If you have a lot of text to delete, holding down delete will begin deleting letters, then accelerate to deleting whole words. In a URL, it will jump to deleting entire path directory names.
iPhone Text Editing.
Despite its very good correction system, I can manage to outsmart its logic and enter misspelled words if I type fast enough. When this happens, the iPhone’s text input system actually makes it seem harder to correct mistakes compared to other phones I’ve used, because there are no arrow keys to move the cursor back.
In place of arrow keys to position the cursor, the iPhone provides a magnifying glass, which pops up whenever you touch and hold a finger down near editable text. It works like a loupe to drop the cursor exactly where it needs to be, but I had to adapt my editing to fit it.
If a word has two errors–and I generally make mistakes in multiples–it’s faster to delete the entire word rather than to try to reposition the cursor multiple times.
Apple might need to revisit some aspects of iPhone text editing, although I’m not certain about what I would actually recommend. I find do myself jonesing for arrow keys, but part of that might be from entrenched behaviors of pedantically editing mistakes one letter at a time. A better solution might a different mechanism for after the fact corrections.
While the iPhone does an outstanding job of correcting words using its spacebar completion trick that requires almost zero thought, there’s no way to spell check after a word is completed. The only options are to backspace toward the error, or to bust out the loupe and target each word with the painstaking efforts of an ancient art restorationist.
One bit of rough area involves text entry into a web form. Safari does not draw standard scroll bars anywhere, even on long text entry areas in a form. Instead, it pops up a custom text editing box. In some forms, this makes it impossible to scroll to the top of the text to reread what you’ve entered, and the loupe is of no help, because it doesn’t know to auto scroll the text area either. In horizontal mode, the loupe is even less helpful.
[Update: Several readers have pointed out that the contents of a web form--such as entered text--can be scrolled through using two finger scrolling. In order to do this, you must often dismiss the text entry mode, or efforts to scroll will be interpreted as a sporadic magnifying glass. It needs fixing, but it does work.]
Three Missing Features: Shortcuts, Pasteboards, and Searching.
There are three things I miss in the area of text entry from the Palm. The first is shortcuts. One gesture used in Graffiti signaled to the system that a shortcut code was coming, for example dts for date time stamp.
Enter [shortcut] + [dts] and Graffiti would spew out something like “6:30 AM on July 15, 2007.” One could even define their own shortcuts, so that any short set of letters could be replaced with a long word or phrase.
A second common feature I strain to find a replacement for is copy and paste. There’s no way to even select text on the iPhone. On the Treo, one can select using a stylus, although even that can be frustrating sometimes. The Palm does make it easy to copy and paste long bits of selected text from one email to another however.
A third missing idea is search. The original Palm Pilots had a way to search the entire device to find matches that would bring up contacts, emails, notes, and anything else it found. I should point out that the Treo phone I have lacks both shortcuts and search from the earlier Graffiti-using Palms, or at least those features were so buried that I never found them in the two years that I had it.
iPhone Searching and Sorting.
The iPhone has no general search system like the Palm or Spotlight on the Mac. It sure would be a welcomed addition to be able to rapidly type in a name and pull up matching contacts, or related notes, emails, or calendar items.
This might require more power than the little device can manage, or may demand so much background accounting that the little ARMs couldn’t ever fall asleep, cutting into its battery life. If it is feasible, the iPhone’s closed architecture will make it easy for Apple to add later, as there won’t be any third party problems to sort out.
I do find myself wanting a search field in Contacts and in Bookmarks, but a key trick to making both far more useful on the iPhone is to organize them on the desktop.
Searching for my doctor’s name takes some time when I’m at a loss of what it even is, but since I organized my contacts in Address Book, it’s a breeze to pull up a short list of Health Care Providers, or whatever other groups I might need to consult.
Similarly, sorting Bookmarks by subject on the desktop version of Safari makes visiting favorite sites much handier on the iPhone when I’m left with a few moments to spare between other tasks. This slick integration on the iPhone is delicious.
iPhone Copy and Paste.
In place of a traditional copy and paste system, iPhone only offers to shoot data around in various strategic places. For example, in Safari you can “Share” a URL, which puts it into the body of a new email. Notes have a shortcut button that forwards its text into a new email as well, and Photos has a similar mail-me shortcut.
A standard copy and paste system would have to account for large bits left after a copy operation, so Apple’s approach with only sending specific bits of information around in a way that leaves no breadcrumbs of data stored on the shelf seems to make sense. However, it leaves no provision for pasting out data multiple times, something that the iPhone sorely needs.
Apple needs to set up a way to insert shortcuts, perhaps by offering a button to insert a Note into any place where text can be entered. This would allow users to enter shortcut phrases as Notes, and reference them from other apps, making Notes a sort of multi-item pasteboard like the classic Mac OS Scrapbook.
This same mechanism could be used to provide multiple signatures for Mail, or for inserting common phrases in SMS texts. The only drawback to this approach may be that Notes is set to get a major overhaul, and such a mechanism might complicate other plans already in place.
The next article will look at a few tricks with Notes, PDFs, Office Files, and other data, and Note’s future potential.
Like reading RoughlyDrafted? Share articles with your friends, link from your blog, and subscribe to my podcast! Submit to Reddit or Slashdot, or consider making a small donation supporting this site. Thanks!