In conversation with Phil McKinney on NUI

26 08 2009

Phil McKinney and Anne in the Halo Room

2 weeks ago I was invited to participate in a discussion with HP‘s CTO, Phil McKinney, about Natural User Interfaces(NUI). As Phil is a very busy person, the discussion was held via HP’s cutting-edge video conference technology, Halo. Why cutting edge? It is because I felt as though Phil was sitting on the same meeting table as me in real-time. I could even see what Phil was sketching on a paper clearly.

Richness vs Reach/Mobility

Above, we see a sketch that Phil drew to start off his discussion. This is a graph showing the state of technology right now in the balance between Richness (user experience, high definition, sound quality, etc) and Reach/Mobility (reaching out and making technology available to more markets). For example, devices like televisions are high in Richness but low in Reach/Mobility.

The line slanting downwards shows the state of technology for the various products at the moment. Unfortunately, there is a void in the middle of products that are high in mobility with average richness. Not many products are able to balance this well while maintaining usability. There is also the “Laws of Physics” that is preventing technologies like a 75″ TV fitting into your pocket.

The challenge faced by most technology players right now is:
1) Filling the void
2) Getting off and above the line

HP’s strategy for achieving (2) is by exploiting Touch technology to increase the Reach of PCs to untapped markets. Currently, 80% of the market has never owned a PC due to literacy barriers. Another reason for this is the intimidation with using a keyboard and mouse. Some of us might find it difficult to understand why using a mouse/keyboard may be intimidating but if you look at it, there are people around us even in developed nations (e.g: the elder generation) who are afraid of performing mouse clicks in fear of what unintended actions may be triggered on the PC.

Why Touch?
Apart from there being non-gimmicky uses for Touch technology, it is a form of interaction which we have been using in our everyday lives – for example, the ATM machine. The learning curve for Touch technology is also less steep as users has one lesser step of learning how to use a new device.

Challenges faced with Touch
HP’s motivation for using Touch is to drive the ease of use of new technologies by designing devices that adopt to users. The very first question that popped into my head was:

“There are so many different type of uses around the world with different levels of literacy. Wouldn’t HP have to design many different devices for different communities?”

Sharings from the lab
Phil shared some research findings in HP’s labs. They tested touch with 2700 users in homes of mixed literacy and discovered that people touch devices differently based on where they grew up.

Here are some of the examples Phil mentioned:
– Some used a whole hand, their thumb or items like an eraser and pencil
– Pulling both sides of an image to enlarge an image
– Grabbing one corner to enlarge the image
– When grabbing one corner and realizing the image follows the hand, the user shakes his hand rigorously as though he was trying to shake it off

From all this research they were able to map out which set of gestures were natural for the different types of users. Based on the first few gestures used by the user, the computer will be able to tell which set of gestures to load for the rest of the user’s interactions. A similar analogy would be how users would select a language on the computer before your mobile phone loads up in the appropriate language.

Do note that this is just for research and not all gesture sets will be going into the final product.

Moving into emerging markets
Lastly, I was wondering how HP plans to break into emerging markets where there are a large number of non-PC owners. Due to cost and infrastructure issues, current PCs are too expensive for people to own in these countries. E.g: In Africa, the cost of one month of broadband is equal to 14 months of their salary).

Instead of focusing on bringing existing PCs into emerging markets, HP is focusing on building basic PCs with the right technologies to meet their needs. They are currently experimenting with concept PCs in some of these markets.

For a later post..
Phil did send me quite a bit of information on their work on emerging markets and also segments of the community with special needs. However, as this post is getting lengthy, I will keep it for another day. 🙂

Concluding words
From the session, I could tell that most of the players in the PC market are jumping into the bandwagon of natural user interfaces. There are lots of other cool technologies HP is experimenting with out there like gaze, motion and tactile feedback. However, most of these are still just “gimmick”s and not at a stage where it can be integrated into HP’s products. HP went with Touch as it is found to be the most practical and usable of them all.

Personally, I can’t wait to see what technologies will come into our computers 5 years down the road! (Maybe some cool holography/Augmented Reality stuff). On the other hand, I’m glad that we are not flooded with useless tech filled with only “gimmicks” for novelty. 😀

P.S: Oh yes! Before I forget, many thanks to the Amelia and Calvin from Waggener Edstrom for giving me this opportunity to be in a discussion with Phil. 😀