Software Everywhere — David Beers on Mobile Computing

Strategy Analytics says touchscreen phones are about to take off. They're predicting that 40% of mobile phones will have touch-sensitive screens within 6 years, which seems pretty optimistic given that we are somewhere around 2% today. They also say the timing of the inflection point (predicted around the end of next year) will depend on three conditions:

1. touchscreen prices must drop (as they are expected to)
2. revenue-generating applications must be developed to differentiate these phones
3. an iconic touchscreen phone needs to appear in a blockbuster movie

Is it just me or do the rest of you feel your heart sink when you hear that the future course of our industry depends on whether Tom Cruise taps a hardware button or a touchscreen during a three-second clip from Mission Impossible 4?

But Strategy Analytics may be right. Only I think they are missing an important condition for touchscreen adoption. And don't think of Mission Impossible, here. Think Minority Report. You know the scene I'm talking about:

The visual of Tom Cruise peeling through seeming terabytes of visual data was mind-blowing not because of the curving wall-sized glass tableau that was his computer screen (although I think that did speak to our lust for gigantic monitors). It was more the way he was able to navigate and manipulate the data so fluidly and rapidly with nothing but hand gestures.

Gestures--simple movements of which small variations can convey rich meaning--are not something you can do well with a mouse and keyboard, still less with a mobile phone keypad. We have "point," "click," and the occasional "drag" as available gestures on the PC. On most phones we have "click" and "click." And "click" and "click" some more. The coarseness of the input we can give to our mobile devices accounts for the biggest complaint we hear about phone user interfaces: the menus.

Navigating the handset with the "popup-> scroll-> select-> view screen-> repeat" motif is bad UI not just because it takes many clicks to do simple, everyday tasks. It's bad because you can't even see what your options are until you start opening menus. And because looking at the menu affords little or no view of the road ahead. Obscure menu options like "Settings" or "My Stuff" can seem almost completely opaque from the standpoint of a user who wants to perform a certain task. We often wonder why so few users bother to learn what their phone is capable of, still less to install third party software. Well, menus are a major reason.

The sad thing is that while a touchscreen can offer much better modes of navigation, no mobile OS vendor has developed a stylus-based user interface that's appreciably better. Palm OS and Windows Mobile replaced some of the menus with on-screen buttons, which helps a little. You can at least see some of your options on the screen without popping up a mystery menu. And yes, they offer stylus gesturing systems as a means of entering handwritten text--something you don't want to do a lot of on a mobile device. But for navigation the motif is still that of the PC: point and tap, view the next screen, point and tap some more. The possibility of performing a task like "check my Gmail" or "make an appointment with Judy" with a single gesture of the stylus seems never to have been considered, though it's thoroughly practical with a touchscreen and stylus (or finger).

Here's just one example of how this could be done. Imagine that instead of the usual smartphone graphical menu--a grid of icons to tap--we had the icons arranged in, say, a ring with a tiny "+" to mark the center. That mark is where you begin the gesture to perform a new task. To check your Gmail account you move the stylus from the center of the screen toward the Email icon, which in turn enlarges and moves to greet your stylus point. Other icons shrink and move out of the way--you're not interested in them now. As the pen approaches the Email icon the most common email tasks emerge as icons and text around it: perhaps Fetch, New, and Read. Moving the pen smoothly toward Fetch it expands and account option icons blossom from it: Work, Gmail, and All. Change the direction of the pen movement to meet the Gmail icon and lift the stylus point. The email client launches and checks your Gmail account after a single stroke of the stylus. The visual effect could be stunning--variously rendered as flying into the interface or watching a vine sprout in stop motion video. It would be easy to make this the stuff of a Hollywood sci-fi thriller.

Hollywood can help sell such a smartphone, but consider what this might mean for increasing peoples' use of its smart features, which is the brick wall we're hitting now with popular smartphone platforms. No memorization of gestures is required, so unlike "Grafitti" the learning curve is near zero. In fact, exploring the capabilities of the phone for the first time is vastly easier and more fun. As you start a gesture, you're given a view "down the road" of the tasks you can choose even before the applications that perform these tasks have launched. With a few minutes of eye-popping doodling a user could gain a very good idea of most of the phone's capabilities and how to use them. Over time, common tasks will be committed to muscle memory as quick gestures that make access to these tasks dramatically faster than is possible on current smartphones or PDAs.

This kind of gestural navigation requires a new software platform. Applications must register with the system their menu hierarchies and certain data queries for the "view ahead" interface to work. (A service-oriented architecture like I've discussed here and here would help.) This platform would provide rich new opportunities for personalization as it could be skinnable to produce all kinds of beautiful visualizations.

If touchscreen devices do take off as Strategy Analytics predict, I contend it will happen because they offer the first really discoverable and fun user interfaces for mobile phones. Stylus input needn't be a watered down version of mousing on a PC: it can and should be an experience that is a great improvement over the PC experience. One of my company's current projects is to explore this possibility, but I hope that others will try.