Kinect versus Windows

The more that I interact with the Kinect, the more I want to also play with it (or a brother of it) on my Windows PC. Considering that the Kinect is a USB device and that most of the other Xbox 360 USB devices have slowly gained official devices I don't think it is a stretch to imagine that many people at Microsoft have already discussed just this.

One of the things that I find interesting is that so many of the individual component technologies have already been in Windows for some time, albeit underutilized. Out of the box, Windows 7 has support for multi-touch and speech recognition and can already support many of the UI patterns that work for "Kinect hovering". It is an interesting thing, in fact, that we don't see more uses of these built-in Windows tools, and more explorations of them as user interaction tools.

As someone with tablet envy some days, I continue to debate the merits of purchasing a multi-touch monitor for my desktop, or a standalone Windows tablet. It still surprises me how many people don't know that Windows supports touch input (much less how long that support has been available).

Speech recognition is the definitive tale of underutilized Windows feature. It has been built-in to Windows since XP (with free versions for several versions previous to XP), and has supported everything the Kinect speech rec does, plus more. How many people do you think have tried it? (For the curious Control Panel -> Ease of Access -> Speech Recognition in Windows 7.)

The big thing that separates the Kinect and most desktop speech recognition is that the Kinect's microphone is an "array microphone" and most computers do not ship with one and few people sell them. Windows Vista's speech recognition wizard suggested purchasing an array microphone, and I believe that at one point Microsoft tried suggesting them to OEMs to sell computers with them.

There is an obvious and funny little pattern that follows: If people had array microphones on their computers they would use speech recognition more. But because "nobody" uses speech recognition, manufacturers don't see why they should sell expensive array microphones on their systems. In the end, Windows' speech recognition has been relegated to "just a toy", because consumers don't buy (and don't know to buy, and sometimes do not have access to buy) array microphones. Windows' Speech Recognition itself doesn't see some of the priority it needs to get better, because users aren't using.

The Kinect is a genius [3] departure from the usual rules in this ugly feedback cycle. By focusing on being a "toy" first and foremost, the Kinect is putting front and center, and in people's living rooms, just the sorts of specialty hardware that Windows wishes it could put on people's desktops.

I've heard it said before that the Xbox 360 has been great for Microsoft because it acts as a clean, proprietary slate for Microsoft to research future Windows UI improvements. I think this is even more the case with the Kinect. The Kinect is quite possibly the best HCI research lab in a box that has yet to be deployed commercially. For now we shall be glad and happy to put it to its banal entertainment uses, but I for one plan to watch excitedly as people begin to grasp that whatever it is that we've started here: it is more than just a toy. I am going to play my role as rat in this new research maze, because that will fun. I am also looking forward to whatever roles as researcher I can take.

Converge the UIs!

So, this is obviously research in progress. I think nowhere is this more apparent than in the sometimes odd sub-contexts of the Kinect Hub and Kinect Guide. It seems unfortunate that in this first release the Xbox team was unable to provide the same user interface/experience regardless of whether you were Kinect hand/speak-waving through it or controller waggling at it.

First of all, I think the Kinect Guide (which you pull up by flag semaphoring a "G" for "Guide" to the Kinect) is actually superior to the current home "blade" of the modern Guide, and almost reminiscent of the old home (Xbox Live) "blade" on the original firmware dashboard (for those that remember that). A controller would not have to much difficulty waggling over it, but it violates the current pivot control of the Guide's blades by making use of both directions. Some other pivot would be needed, but is already needed given the fact that the Kinect Guide is lacking much of the Xbox Guide's tools as it is.

I would almost welcome an option to try using the Kinect Guide as a new default for my controllers, with some mechanism such as a double-tap to revisit the more featureful Xbox Guide.

On the other hand, the Kinect Hub is a weird cousin to the current Xbox dashboard. Lacking a pivot control that is anything like the dashboard's, the Kinect Hub seems content to just provide a hodge-podge of items from across the dashboard's pivots, spread out in a couple of rows of tiles. There don't appear any visible differences between the latest less-3D dashboard update's tiles and the Kinect's tiles, and yet one responds to hand-waving and provides voice prompts while the other does not. At this point the distinction seems arbitrary and useless.

It seems like Microsoft is just a stone's throw from a truly useful convergence of its "dashboard" UIs: at this point Media Center, Xbox, Kinect, and Windows Phone 7 all seem to share similar concepts of pivots and tiles. There just weirdly seem to be so many subtle differences between the various approaches, and having three of the four accessible on an Xbox 360 seems particularly like a silly kick in the pants.

It is possible that the touch, controller, and hand-waving are too disparate to bridge into a single UI structure, but I'd be surprised if that were the case. Again, it seems that Microsoft is so close to establishing a great design pattern and useful UI across all of these disparate input systems and use cases. I hope that the current differences are much more a matter of coding velocity and priority needs for this particular launch than siloed warfare. [4] I can only assume that time will tell, but I've got a feeling there should be at least one internal evangelist currently crusading on this topic.

Now if you excuse me, I'm going back to the maze. I really want to win that adventure club watch for my avatar...