Project FluidWall

This is a project I have been working on with another student from Texas A&M Visualization Lab, Austin Hines.

Here's a link to the project's initial draft on Austin's blog.
We've also made this project opensource and created a whole separate blog for FluidWall.
FluidWall Source: http://code.google.com/p/fluidwall/
FluidWall Blog: http://fluidwall.blogspot.com

The idea was to use the depth information and user tracking from the Kinect to be able to interact with a fluid simulation on the screen. Austin Hines worked on all aspects related to the fluid simulation, while I handled exploring and integrating the SDK that would let us communicate with the Kinect with the most ease. Using the OpenNI  along with with NITE middle ware we were able to get some really nice and simple depth data working with our simulation. This allowed us to have silhouettes of people / objects (whatever the depth sensor returned) to interact with fluid on the screen. Any movements would trigger the fluid which would dissipate in a bluish-white whirl. as depicted in pictures of the original idea on Austin's blog (mentioned above).

Soon we were able to track users and send appropriate data into the fluid simulation. This allowed us to start emitting different colors for different users.

The final result was quite amazing...

The Kinect SDK of Choice [OpenNI + Primesense + CLNUI]

Driving the Kinect Camera
Using the OpenNI SDK with PrimeSense's Driver and middle-ware, life has been much simpler :)
Here's what it can do right after the installation:

The Motor was Another Story Altogether

The only problem with the PrimeSense driver was it does not support the Kinect Motor. So, after several experiments and some time searching online forums for a solution, it was discovered that the PrimeSense Sensor's camera driver and CLNUI motor + audio drivers can be installed to run side-by-side.

This is what the Device manager might look like on a Windows XP machine with the two installed together:

On a Win XP Machine

On Windows 7 it somehow mixes up the CL Devices and PrimeSensor names, but the drivers still work fine.

How to make them work together (on Windows):
We had to do the triple installation for the OpenNI+Sensor+NITE and the installation for CLNUI Devices could be done either before or after them (we've tried both versions on different machines).

Once installed, the Kinect will probably have the first driver you installed as its primary driver. So, you'll have to go into the Device Manager and manually change the driver for the Camera to 'Kinect Camera' and the Motor to 'NUI Motor'.

And now, for something entirely different: Taming the Kinect

I had started a separate blog for my Kinect-based projects, but somehow it doesn't seem to be as popular. Therefore I've decided to transfer my Kinect related posts to this blog. So, here goes:

-------------------------------------

Here I share with you some general information and helpful links to all the Kinect-related open source content and websites I was able to find along the way during my search for the solution for using the Kinect on my computer:

The OpenKinect Community
OpenKinect is a very informative open community for updates and latest information on the current progress on the Kinect libraries and other software being developed by users all over the internet. There is also a lot of useful hardware information for understanding how the Kinect works and about USB devices and protocols in general for anyone interested in finding out more.

Here are all the open-source and freely available SDK's I came across when I first started searching for ways to use the Kinect with my computer. The pros and cons I've discussed here are based solely on my own experience using the SDK's.


Attempt #1: Code Lab's NUI Platform

The first result I found when searching for an SDK for the Kinect was the Code Laboratories' NUI Platform. The libraries were developed by AlexP, the first person to have hacked a Kinect sensor for use with a computer since after its release.

Pros:

• Pristine C#  libraries and sample code. 
• Using Visual Studio Express, the provided sample code compiled and runs quite easily.
• C++ API is also available from the Code Lab website (No samples in C++ though)
• Installation and running the sample code was a breeze on Windows 7 (I tried it on a quad core machine running an NVidia GT9800 graphics card).

Cons:

• No samples for C/C++  (I found a sample code by another user on their forum, but I wasn't able to run it). Still, creating a simple code on C++ by just using their API was simple enough once I figured out the flow of control from the C# samples.
• There are no skeleton-extraction and /or tracking options on this SDK (yet). One can possibly work out how to use some middle-ware (like the NITE libraries available for use with the OpenNI framework) to accomplish that. But there isn't any support available on the CodeLab website for skeleton detection and tracking (...yet).
• Doesn't seem to work on my old Dell laptop (XPS M1210 running WinXP: dual core, NVidia Go7400). The RGB video would show, but it refused to display any depth data. (the laptop does have a USB 2.0, so it's not a USB port issue) I don't believe the problem is caused by any requirements from the Kinect but the limitation might be on the driver/SDK side.

Still, overall a very neat and nicely put together SDK without any messy compilations and multiple installs - just install the 'CL NUI Platform Installer' (available on the right-hand-side panel) and you're done. That should install the Drivers & the SDK, along with all  the required windows paths to the system, and even add the samples in the installed directory with Visual Studio project solutions ready to be run. 

Very neat, indeed!

Attempt #2: Libfreenect
This is the platform actively supported by the OpenKinect community. It is constantly being updated and includes a ton of user support on its forums.

Pros:

• Cross Platform drivers, SDK and support.
• Wrappers for multiple languages 
• Documentation, very detailed. 
• Both High Level and Low Level APIs available
• Compilation instructions for several programming environments, including Visual Studio

Cons:

• No pre-compiled binaries for the SDK available. You have to compile them separately for the SDK to work for your specific system and programming environment.
• Complicated compilation process - several different libraries have to be downloaded from various sources and installed separately from the main code. 
• No people tracking, gesture/skeleton tracking options. No integration with a middle-ware (such as NITE) available.

Attempt #3: OpenNI + PrimeSense Sensor + PrimeSense NITE

This was a set of three different components that had to be installed separately, but they worked together quite nicely to provide the most out of my Kinect experience! The setup comprises of the following three to be installed in order as follows:

• OpenNI (Kinect SDK)
• PrimeSensor/SensorKinect from PrimeSense (Driver)
• NITE from PrimeSense (Middle-ware for skeleton & gesture recognition etc.)

There are some compatibility issues with different versions of these, however the combination of versions on this site (although a little old) worked together perfectly for me.

Pros:

• Works great on both computers I tried it on, the Win7 quad-core machine as well as the WinXP dual-core laptop (both mentioned above)!
• Cross Platform (Windows & MacOSX support, I wasn't able to find info on Linux usage).

OpenNI Installation

• Works great when all the right versions are installed in the correct order.
• Several different sample provided with source and pre-compiled binaries for doing all sorts of stuff: from getting simple RGB & depth images  to detecting people as separate objects and tracking them.
• Documentation included! (Very detailed too!)

NITE Installation

• NITE installs separately and provides its own list of amazing samples. Works as the middle-ware for detecting people when using OpenNI.
• Includes its own Documentation! (Also very detailed)
• Several samples provided (separate from the ones from OpenNI). Again source and binaries both included. Let's you explore several hand-tracking options. 

Cons:

• Making sure I had all the compatible versions of OpenNI and Primesense was a bit of a pain. 
• Installs as three parts - as compared to the 1 installation for the CL NUI Devices mentioned above.
• No Motor or Audio driver included in the PrimeSense Sensor (not a game-breaker though, it plays well with the CLNUI package to let you control the motor through the CLNUI SDK).