Early Prototype Sample

Just a follow up to Melissa's post, in this post I've embedded the flash file we used for our proof-of-concept prototype. If you have a Kinect running on your PC is will work as intended right in the browser, otherwise you can hover over the buttons and pretend you have a Kinect :-/

Read more to try out our flash prototype!

Progression of Gestural User Interface

A visual tour of the gestural interface progression January 2012-May2012. 
This tour will take you from Initial research and groupings until the high fidelity mockup used in our final interface.

Click read more below to watch how the transformation happened in pictures...

Convincing Xbox360 Kinect to Play Nice with a Small Apartment

Whenever I am discussing our NUI CDR project with someone for the first time I normally need to explain to them we are using the “PC” version of Kinect, rather than the “Xbox” version, and often times people seem surprised there’s a difference. The PC version primarily exists for software developers to build on, and the Xbox version is targeted at end users looking to control their gaming console. It should be mentioned there's plenty of open-source software out there to hack the Xbox Kinect using a PC, and some may say it was this growing trend that encouraged Microsoft to release a version for PC with an official SDK. In any case, it seemed like a good idea to buy a Kinect for my Xbox360 to learn more about using gesture interactions and to gain a familiarity with what game developers have been serving to their audience up to this point. While some reading this may be quick to accuse me of gaming during times I should be working, it should be noted that this is research, yes I am playing Xbox and it is research!

Oh yeah, by the way, before we even touch any gaming there's a little calibration process...

Quick Updates | Prototype Planning

Time is of the essence right now... Lately a lot has been happening as Melissa, Karen, and I have been branching out with independent tasks with the hopes of amalgamating everything into a functional prototype at the end of this week!

I've been learning Flash in a hurry, Melissa has been doing fundamental UX/IxD work planning the user interface, while Karen has been learning the coding with the PC Kinect. Recently we decided to scale back the initial design vision, this was a result of various factors such as: using Xbox360 Kinect for gaming, checking out a Kinect kiosk prototype at a local company (unnamed due to confidentiality), re-assessing the needs at the CDR, and of course keeping our deadline in mind. If we can make something basic, that can give a user information related to the systems, then we can always add to it later but for now the task is to build a basic prototype with energy savings and consumption data. The lights at the CDR are already on sensors and when evaluating areas for innovation that fell low on the list due to the fact the existing system work well, it senses motion and turns on the lights, there's little need to add manual overrides for that in our interface. Integrated manual controls for the electrochromic window tint would be nice but will most-likely be pushed back into a later version of the prototype.

Image below shows system information data that Melissa collected, the data communicated was referenced from photographs we took of the existing CDR energy systems (click to enlarge)

Below is a Cmap image that Melissa put together using IHMC CmapTools, it illustrates some of the information we are sorting through between the solar and wind systems. (click to enlarge)

Building the Cmap image helped us compare the information from the different systems to establish what pieces are valuable to the user and what can be left backstage. Some information became more apparent through this exercise, and it also shows the late removal of the light and temperature controls. The focus shifted towards information access and away from system controls, in any case we are saving the work off to the side for future reference.

The next blog post will cover some technical issues that I have experienced with the Xbox360 Kinect in my apartment, in the meantime it's back to work for me as I've got plenty of work to do on our prototype!

Getting PC Kinect to Recognize the Basics

Unlike the Xbox 360 Kinect, the PC version is not plug-and-play and it requires some coding to get started. Fortunately our team member Karen is capable of handling the coding requirements. She has a background in software development and has used C and C++, but never C#, so she borrowed some sample code to start out...

Ideo's Kinect Physics Demo

Ideo has a Kinect physics demo on their Vimeo page, it's been up for a while but somehow slipped under our radar until now. This is a great demonstration of the accuracy the sensor is capable of:



Recently we've been doing some brainstorming about things like switches and wheels, virtual representation of controls with limited affordances that a user can interact with naturally, watching this validates some of our discussion as it seems the sensor should certainly be capable of those things.

Right now we're in the early stages of prototyping with a group meeting scheduled for tomorrow night at the CDR, next update I will fill in the details of how we got to this point and what we are hoping to achieve, stay tuned!

Kinect and PC Arrived

Our hardware arrived earlier this month so we all met up at the CDR to dig into it on March 4th. After we got everything hooked up we quickly noticed the computer wasn't really noticing the Kinect at all, it became clear to us right away that we needed to manipulate code in the Software Development Kit (SDK) to really get started. It was a productive start to our project, more info and images after the break.

Kinect at the Grocery Store

In addition to posting about our NUI/CDR project I want to use this blog to share other innovative uses of the Kinect hardware. Here's a video demonstrating some Kinect technology being used with a grocery cart, apparently Whole Foods is testing this prototype:

The system will recognize a user's frequent shopper card to identify them, it then gives the option to load a shopping list, follows the shopper automatically, and then scans items as they are placed in the cart.

Hardware and Systems at the Center for Design Research

On Saturday 2/25/2012 we stopped over at the CDR to take some photos of the current system. Much of the energy data is communicated through varying software platforms on a large touch-screen display located near the lobby/entry area. Some things like the lights and windows are omitted from the touch-screen system and are connected to their own separate controls, my understanding is that even though they do not interface through the touchscreen they still send data to the backbone PC. This lack of consistency makes simple tasks confusing and gives the system integration an unfinished feel. While it's impressive that this building exists with all this great technology and it's certainly efficient, there's a significant usability gap we are hoping to close to provide an enjoyable user experience.

Welcome to the NUI Blog!

This blog is to document our graduate design work at the University of Kansas (KU) relating to Natural User Interfaces (NUI). The actual timeline of the class will run for the Spring 2012 semester, but chances are this may turn into a thesis or extended project that spans far beyond that. Our immediate task is to fearlessly dive into this and share our progress here.