unmediated

Russell has recently posted a few interesting posts this one and this one on the function and design of hardware. I've been enjoying thinking about these issues again, because inherently many of the problems mobile interaction designers have, is how to design things which integrate into extremely disparate hardware form factors.

Today Russ posted this:

Mobiles are the ultimate consumer computer. They are meant to be used by 12 year olds, teens, college kids, business people and your mother in law. But right now, the design of the interface is still way too confusing. Even Nokias which rank high on the usability scale have problems when it comes to using their phones. I was just talking to someone today about the "overloading" problem with Nokia. The re-use the same button for different, completely disparate tasks: like your power button to change audio profiles. What?! And the fact that if you click the menu button once, you go back to the home screen, and if you click it again you go to the menu, and if you hold it down, you get a list of running apps. On other phones, they have a tendency to do things like combine the power and hang up keys. Huh?

The problem he is describing here, is a classic problem of modal interfaces. As Jef Raskin so clearly coined: "Modes can change the effects of habitual actions they can cause errors or draw away the user’s
locus of attention from the intended task."

In summary:

In his book, Jef argues that all modes are evil, whether we are talking modal dialog boxes, different selection modes in graphical editors, or even having different applications that behave differently. Since humans are creatures of habit, we have a hard time remembering which mode the computer is, because we want to focus on the task at hand; modes divert our attention to keeping track of the mode, and hence slow us down.

Often too much functionality will be crammed into one button or object, with the intent of space saving. It seems that manufactures have been revisiting the multi-modal interface. The problem, however, as Russ points out is pattern recognition and recall. We think the button means one thing, but then it means something different. Moreover it brings up a questions of language. Trying to use the same term to mean two different modes creates errors. For example, back meaning clear. "The problem with overloading the clear and back buttons is that if you get into a text entry screen and change your mind half way through? If you don't have a back button, you can't escape without deleting all your text character by character first." But I bet a lot of people try to do it, thinking that they will go "back" to their previous state.

As 3rd party developers it's challenging to decide to follow similar interaction patterns to their native software or to create a new interaction paradigm for the user to learn. Users have often already learned (or perhaps created) how to use their devices, recognizing that there are problems with them, do we try to recreate the interface or choose to accept the principles and design in accord to what the user already knows?

When I was at Yahoo! we thought about this a lot. Building a brand... do you a. create a new interaction that remains consistent across devices or b. make it easier on the user by customizing each interaction mechanism, per device. The second is obviously better to the end user. Building on mechanisms which they already understand, customize each experience per handset. However, that becomes outrageously costly. Porting each application is one thing, but customizing the entire interface is another altogether. What we decided then, and I've stuck with, is to find the common hardware elements that are mostly consistent across devices and utilize those to be the main controls and interaction controls into your app. The question then is what are those hardware elements? At one point I had suggested that we survey old phones and try to ascertain which hardware interfaces are sticking around and which have gone by way of the roadside. Remember the side direction buttons instead of the joystick for directional navigation? Trying to find patterns in the hardware interaction features can then help us guess which ones may be present in the future and thus help drive the software interaction models.

I still wish someone would do this survey.

Currently, what I go on as common features are: 5-way joystick. If necessary a secondary, 2 softkeys. If I can design things which only utilize those hardware features, I’m pretty confident that the experience will a. work in almost the same manner across all phones b. be easily portable (ie save time and money) c. not break any current models the user has for how those hardware features work. Directional navigation, select, back. That's it! (Sometimes that means that there are two back keys. I believe that's better than none.)

It sure would be nice, though, if some people decided to start converging on things like back, home, and clear! Duplication within our apps then wouldn't have to exist. We can change the language of our apps to be consistent across devices, but why should we have to?

The first step, I think, to recognizing which hardware mechanisms on the device really do make sense is abstracting all the things we want to do. I think we may just be getting to this point. Then obviously think about the common interactions and keep abstracting. When a common pattern can't be found put another element! The iPod is such a great example at an abstracted interface. Navigation and select... that's it! (Yes, back would be nice). What would a phone look like with just a directional navigation and select? The problem is that we have already decided that there needs to be dedicated hardware to specific functions. It's when we start to pull these specific functions out and give them each a new button that things get crowded. We already decided that 9 numeric keys are necessary. But who's to say that's true. Mfoundry has an awesome text entry interface, that is perhaps faster then T9 and only uses directional navigation and select. Two buttons. Maybe it's a problem of too many shortcuts. Or perhaps not understanding enough which ones are really essential.

"So wait, am I just recreating the web browser on my mobile phone? Sorta... and even if I am, that may not be a bad thing." That makes sense, Russ, cause after all aren't we in the browser war of the mobile revolution? ... and just beginning to really understand what users are doing with their mobiles?

Anita