«
»

Kinect on Ubuntu with OpenNI

UPDATED May 1 2012 for the latest versions of everything!

I’ve spent all this morning trying to talk to the Microsoft Kinect using OpenNI. As it turns out, the process is not exceptionally difficult, it’s just there doesn’t seem to be any up to date documentation on getting it all working. So, this post should fill the void. I describe how to get access to the Kinect working using Ubuntu 11.04, OpenNI, and NITE.To talk to the Kinect, there are two basic parts: OpenNI itself, and a Sensor module that is actually responsible for communicating with the hardware. Then, if you need it, there is NITE, which is another module for OpenNI that does skeletal tracking, gestures, and stuff. Depending on how you plan on using the data from the Kinect, you may not need NITE at all.

Step 1: Prerequisites

We need to install a bunch of packages for all this to work. Thankfully, the readme file included with OpenNI lists all these. However, to make life easier, this is (as of writing) what you need to install, in addition to all the development packages you (hopefully) already have.

sudo apt-get install git build-essential python libusb-1.0-0-dev freeglut3-dev

There are also some optional packages that you can install, depending on whether you want documentation, Mono bindings, etc. Note that on earlier versions the install failed if you didn’t have doxygen installed, even though it is listed as optional.

sudo apt-get install doxygen graphviz mono-complete

Step 2: OpenNI

OpenNI is a framework for working with what they are calling natural interaction devices.Anyway, this is how it is installed:

Check out from Git

OpenNI is hosted on Github, so checking it out is simple:

git clone https://github.com/OpenNI/OpenNI.git

From there, change into the Platform/Linux-x86/CreateRedist directory, and run the RedistMaker script. Note that even though the directory is named x86, this same directory builds 64 bit versions just fine. So, don’t fret if you’re on 64bit Linux.

cd OpenNI/Platform/Linux/CreateRedist
chmod +x RedistMaker
./RedistMaker

The RedistMaker script will compile everything for you. You then need to change into the Redist directory and run the install script to install the software on your system.

cd ../Redist/OpenNI-Bin-Dev-Linux-[xxx]  (where [xxx] is your architecture and this particular OpenNI release)
sudo ./install.sh

Step 3: Kinect Sensor Module

OpenNI doesn’t actually provide anything for talking to the hardware, it is more just a framework for working with different sensors and devices. You need to install a Sensor module for actually doing the hardware interfacing. Think of an OpenNI sensor module as a device driver for the hardware. You’ll also note on the OpenNI website that they have a Sensor module that you can download. Don’t do this though, because that sensor module doesn’t talk to the Kinect. I love how well documented all this is, don’t you?

The sensor module you want is also on GitHub, but from a different user. So, we can check out the code. We also need to get the kinect branch, not master.

git clone https://github.com/avin2/SensorKinect
cd SensorKinect

The install process for the sensor is pretty much the same as for OpenNI itself:

cd Platform/Linux/CreateRedist
chmod +x RedistMaker
sudo ./RedistMaker
cd ../Redist/Sensor-Bin-Linux-[xxx] (where [xxx] is your architecture and this particular OpenNI release)
sudo chmod +x install.sh
sudo ./install.sh

On Ubuntu, regular users are only given read permission to unknown USB devices. The install script puts in some udev rules to fix this, but if you find that none of the samples work unless you run them as root, try unplugging and plugging the Kinect back in again, to make the new rules apply.

Step 4: Test the OpenNI Samples

At this point, you have enough installed to get data from the Kinect. The easiest way to verify this is to run one of the OpenNI samples.

cd OpenNI/Platform/Linux-x86/Bin/Release
./Sample-NiSimpleViewer

You should see a yellow-black depth image. At this point, you’re left with (optionally) installing the higher level NITE module.

Step 5: Install NITE (optional)

Firstly, you need to obtain NITE. Go to the following link, select OpenNI compliant Middleware Binaries.

http://www.openni.org/Downloads/OpenNIModules.aspx

Download either the 32 or 64 bit version, depending on your platform. As of writing they only have versions for Ubuntu 10.10, but these seem to work fine with 11.04 as well.

Extract the archive, and run the installer:

sudo ./install.sh

At some point, you may be asked for a license key. A working license key can be found just about anywhere on the Internet. I don’t think PrimeSense care, or maybe this is a non-commercial license or something. But whatever, just copy that license into the console, including the equals sign at the end, and NITE will install just fine.

Conclusion

After following these steps, you will be able to write programs that use the Microsoft Kinect through OpenNI and NITE middleware. I hope this helps someone, because I spent a lot of time screwing around this morning trying to get it all to work. Like I said, the process is pretty straight forward, it just hasn’t been written down in one place (or I suck at google).

Tags: , , , ,

This entry was posted on Thursday, June 30th, 2011 at 3:06 pm and is filed under Augmented Reality, Linux, Programming. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.

37 Responses to “Kinect on Ubuntu with OpenNI”

  1. Nalh Says:
    July 4th, 2011 at 1:20 pm

    Hi,

    in the step 3, is that normal to have OpenNI path for installing, or we should use Sensor ?

    Nice step by step tutorial !

  2. Michael Says:
    July 4th, 2011 at 2:10 pm

    Hey

    You’re right, it was a copy/paste error. Thanks for letting me know.

  3. Nalh Says:
    July 5th, 2011 at 12:27 pm

    Hi,

    you’re welcome. Thx a lot for the tutorial =)

  4. Nalh Says:
    July 5th, 2011 at 12:36 pm

    Sorry for double post, i didn’t find editing tool.

    In the “apt-get install” part, best to add “git-core”.

  5. Michael Says:
    July 5th, 2011 at 3:36 pm

    Ah yeah. On Ubuntu 11.04 the git package is now just ‘git’, on older versions it is git-core.

  6. Mariano Says:
    July 8th, 2011 at 5:15 pm

    Excellent tutorial!!!

    In step 4 I got: Warning: USB events thread – failed to set priority. This might cause loss of data…
    Any ideas?

  7. Michael Says:
    July 11th, 2011 at 9:39 am

    The same thing happens to me. I don’t know what the cause is, but haven’t seen it cause any problems at runtime.

    Out of curiosity, does it still do it if you run as root? It may be a permissions thing…

  8. Ujwal Says:
    August 2nd, 2011 at 8:05 pm

    Hi

    Real nice documentation. However, I get a problem at step 3 during the install process for the sensor. When I run ./RedistMaker, it gives:
    make: Entering directory `/home/—–/kinect/Sensor/Platform/Linux-x86/Build’
    make -C XnCore CFG=Release
    make[1]: Entering directory `/home/—–/kinect/Sensor/Platform/Linux-x86/Build/XnCore’
    ../EngineLibMakefile:24: /usr/include/ni/CommonMakefile: No such file or directory
    make[1]: *** No rule to make target `/usr/include/ni/CommonMakefile’. Stop.
    make[1]: Leaving directory `/home/——/kinect/Sensor/Platform/Linux-x86/Build/XnCore’
    make: *** [XnCore] Error 2
    make: Leaving directory `/home/——/kinect/Sensor/Platform/Linux-x86/Build’

    From a different post they had advised to provide a link from file /usr/include/ni/CommonMakefile to /usr/include/ni/CommonCppMakefile. I did this using:
    cd /usr/include/ni
    sudo ln -s CommonCppMakefile CommonMakefile
    However, after this when I try the installation(./RedistMaker) it gives this:
    …….
    …….
    ./Release/jutils.o: In function `jcopy_sample_rows’:
    jutils.c:(.text+0x9c): undefined reference to `xnOSMemCopy’
    ./Release/jutils.o: In function `jzero_far’:
    jutils.c:(.text+0×36): undefined reference to `xnOSMemSet’
    ./Release/jutils.o: In function `jcopy_block_row’:
    jutils.c:(.text+0×50): undefined reference to `xnOSMemCopy’
    collect2: ld returned 1 exit status
    make[1]: *** [../../Bin/Release/libXnFormats.so] Error 1
    make[1]: Leaving directory `/home/——/kinect/Sensor/Platform/Linux-x86/Build/XnFormats’
    make: *** [XnFormats] Error 2
    make: Leaving directory `/home/—–/kinect/Sensor/Platform/Linux-x86/Build’

    Any ideas whats wrong?
    I tried installing the sensor modules from the OpenNI repo and continued with the remaining installation but the final release binaries did not work and kept saying ‘InitXmlfiles failed…..’
    For your info:
    OS: Linux 10.10
    Pro: intel xenon

  9. cyb Says:
    September 21st, 2011 at 11:25 am

    Hi

    Have you build openni on android? if do, could you share your expericence? thanks.

  10. Michael Says:
    September 21st, 2011 at 4:31 pm

    Hey
    Nope, haven’t tried building for Android yet.
    Cheers

  11. flsk Says:
    November 10th, 2011 at 9:49 pm

    To all the ppl who still can’t make it.
    the boilerbots git is deprecated, use this one git clone https://github.com/avin2/SensorKinect
    rest of the steps are the same.

  12. Michael Says:
    November 14th, 2011 at 12:05 pm

    Thanks flsk! I’ve updated the instructions with the new git repository.

  13. Kal Says:
    November 15th, 2011 at 9:05 am

    Hi!

    I’ve tried to run the tutorial but I’ve got an error in “git checkout -b kinect origin/kinect”: “fatal: something about git trying to chck out ‘origin/kinect’ commit” However, changing it to “git checkout -b kinect origin” appears to solve the problem. By the way, the line “cd Sensor” should also be changed to “cd SensorKinect”

    Cheers

  14. Michael Says:
    November 15th, 2011 at 11:11 am

    Hey

    You’re right, there was a mistake in the instructions because the git repository changed. Should be all fixed now.

  15. sapa Says:
    November 20th, 2011 at 6:32 pm

    hi ,
    i was trying to execute few samples from NITE. But they are not working. I even tried to build them with
    ~/kinect/nite-bin-linux64-v1.4.2.3/Samples/Build make
    that gave error
    make -C ../Boxes.net -f Boxes.net.mak
    make[1]: Entering directory `/home/nagraj/kinect/nite-bin-linux64-v1.4.2.3/Samples/Boxes.net’
    make[1]: Nothing to be done for `all’.
    make[1]: Leaving directory `/home/nagraj/kinect/nite-bin-linux64-v1.4.2.3/Samples/Boxes.net’
    make -C ../Players -f Players.mak
    make[1]: Entering directory `/home/nagraj/kinect/nite-bin-linux64-v1.4.2.3/Samples/Players’
    g++ -o ../Bin/Release/Sample-Players ./Release/main.o ./Release/SceneDrawer.o ./Release/kbhit.o ./Release/signal_catch.o -L../Bin/Release -lglut -lOpenNI -lXnVNite_1_4_2
    /usr/bin/ld: skipping incompatible ../Bin/Release/libXnVNite_1_4_2.so when searching for -lXnVNite_1_4_2
    /usr/bin/ld: skipping incompatible /usr/lib/libXnVNite_1_4_2.so when searching for -lXnVNite_1_4_2
    /usr/bin/ld: cannot find -lXnVNite_1_4_2
    collect2: ld returned 1 exit status
    make[1]: *** [../Bin/Release/Sample-Players] Error 1

    can you help out

  16. sapa Says:
    November 20th, 2011 at 8:09 pm

    the problem is solved. the problem was with the NITE installation needed to install
    NITE 1.3.0.17
    downloaded it from
    https://dl.dropbox.com/u/11217419/NITE-Bin-Ubuntu-x86-1.3.0.17.tar.bz2

  17. Matt Says:
    November 29th, 2011 at 7:21 am

    Hi,

    Thank you very much for summing this all up, but I have hit a snag.

    After
    git clone https://github.com/avin2/SensorKinect
    cd SensorKinect
    cd Platform/Linux-x86/CreateRedist
    ./RedistMaker

    I receive an error stating.

    make[1]: warning: jobserver unavailable using -j1. Add `+’ to parent make rule.
    …
    ../../../../Source/XnDeviseSesorv2/XnSensorAudioStream.cpp In member function ‘XnStatus XnSensorAudioStream::ReallocBuffer()’:
    ../../../../Source/XnDeviseSesorv2/XnSensorAudioStream.cpp:487:144 error: ‘xn0SCreateSharedMemoryex’ was not declared in this scope

    Do you have any thoughts on this matter?

  18. Pratyush Delicious links for December 2, 2011 | Pratyush Kotturu - KE5YQZ Says:
    December 2nd, 2011 at 5:04 am

    [...] 20papercups :: Michael Marner’s Website » Blog Archive » Kinect on Ubuntu with OpenNI [...]

  19. linxiongmin Says:
    January 9th, 2012 at 7:22 pm

    hi
    thanks for sharing
    i have a problem in step 3
    when i do
    ./RedistMaker

    bash: ./RedistMaker: Permission denied

    but when i do below

    sudo: ./RedistMaker

    sudo: ./RedistMaker: command not found

    why is that? should i :

    sudo apt-get install doxygen graphviz mono-complete

    ?????

    thx 4 the steps again

  20. linxiongmin Says:
    January 9th, 2012 at 11:55 pm

    hi agian

    i have done the installation:

    sudo apt-get install doxygen graphviz mono-complete

    but the problem still can’t be solved

  21. linxiongmin Says:
    January 11th, 2012 at 6:03 pm

    hi again again

    i have solved the problem of “sudo: ./RedistMaker: command not found”

    rigthclick file RedistMaker -properties-permissions

    select “Allow executing file as program”

  22. Mattia Says:
    February 8th, 2012 at 7:21 am

    When running ./Sample-NiSimpleViewer I got following error:
    Open failed: Failed to set USB interface!

    Resolved with:
    sudo rmmod gspca_kinect

    Now it starts to work.

    Mattia

  23. Antonio Domínguez Says:
    February 28th, 2012 at 2:28 am

    Hi there,

    You have realy fill the gap in OpenNI documentation for using Kinect in Linux. It worked directly.

    Thanks for sharing the info.

  24. ITC Says:
    May 17th, 2012 at 10:19 pm

    Hi
    I am getting an error in step 2

    chmod +x RedistMaker
    ./RedistMaker

    The error is:

    /bin/sh: 1: javac: not found
    make[1]: *** [../../../Bin/x64-Release/org.OpenNI.jar] Error 127
    make: *** [Wrappers/OpenNI.java] Error 2
    make: *** Waiting for unfinished jobs….
    ../../../../../Wrappers/OpenNI.net/NodeInfo.cs(51,43): warning CS0219: The variable `description’ is assigned but its value is never used
    failed to execute: make -j16 -C /home/dale/OpenNI/Platform/Linux/CreateRedist/../Build > /home/dale/OpenNI/Platform/Linux/CreateRedist/Output/BuildOpenNI.txt
    Building Failed!

    Is there an additional java dependancy required? I ran sudo apt-get install openjdk-7-jre. Any help would be greatly apreciated.

  25. Development tools for Kinect « joantziouvara Says:
    May 19th, 2012 at 12:10 am

    [...] OpenNI and NITE  installation on Ubuntu, this post was of great [...]

  26. enddl22 Says:
    May 26th, 2012 at 5:20 pm

    It’s awesome tutorial.

    In step 3,

    I faced this error:

    ../../../../Source/XnDDK/XnFrameBufferManager.h:30: fatal error: XnEventT.h: No such file or directory
    compilation terminated

    Do I need to install some packages to resolve this?

    Cheers.

  27. andrew Says:
    May 31st, 2012 at 3:02 am

    Hi,

    Just thought I should chip in by saying that when using the avin2 repo for SensorKinect, you need to use the unstable branch of OpenNI.

    This can be done by just using

    git checkout unstable

    within the OpenNI directory.

    Cheers for the walkthrough by the way.

  28. Tom Says:
    June 1st, 2012 at 12:55 am

    Note that the current HEAD versions of OpenNI and SensorKinect (as at 01-JUNE-2012) are not compatible with each other – I had to roll back SensorKinect to 12-DEC-2012 in order to get it to compile. The HEAD version of SensorKinect seems to be developed against some internal version of OpenNI.

    And, for those uninitiated into the git universe, that’s:

    cd SensorKinect
    git checkout faf4994fceba82e6fbd3dad16f79e4399be0c184

  29. Christian G Says:
    June 5th, 2012 at 1:26 am

    Hi

    i just found your tutorial after spending the whole day to bring openni, nite and
    kinectsensor to work on gentoo and ubuntu :D

    But anyway thanks for sharing! maybe someone else finds it earlier =)

    Chriss

  30. Michael Says:
    June 5th, 2012 at 10:24 am

    Hey
    Thanks for the heads up. These git repositories change so often, I can’t keep up.

    Cheers
    Michael

  31. Michael Says:
    June 5th, 2012 at 10:25 am

    Hey
    Have a look at some of the other comments with regards to unstable branches that you need to use.

    Cheers
    Michael

  32. Mazin Says:
    July 3rd, 2012 at 6:51 am

    Thanks a lot for the guide! I’m still having trouble getting past the Step 3. I get this error that a lot of other people seem to have faced as well

    make[1]: *** [x86-Release/Bayer.o] Error 1
    make[1]: Leaving directory `/home/mazin/Documents/Kinect/SensorKinect/Platform/Linux/Build/XnDeviceSensorV2′
    make: *** [XnDeviceSensorV2] Error 2

    I rolled back to an earlier version of the SensorKinect as Tom suggested and I’ve checkedout the unstable version of OpenNI as Andrew suggested. But I’m still getting this error. Any ideas why?

  33. Mazin Says:
    July 3rd, 2012 at 6:59 am

    Actually I got it, it was my bad

    I was supposed to run this command before beginning

    sudo apt-get install git-core cmake freeglut3-dev pkg-config build-essential libxmu-dev libxi-dev libusb-1.0-0-dev doxygen graphviz mono-complete

  34. Fred Says:
    July 18th, 2012 at 8:58 am

    Thanks so much! This was a huge help.
    For me to get things to work, I had to download the unstable branch of OpenNI manually from their website (I’m not sure how to do that from Git), but all other instructions worked perfectly!

    Symptoms of this problem are that there are three or so undefined symbols in a header named something like XnDeviceSensorIO.h (probably not the correct name of the file, but close) around line 98. I got this to pop up while installing SensorKinect at the step “sudo ./RedistMaker”

    Thanks again!

  35. BB Says:
    August 12th, 2012 at 2:29 pm

    Thanks Tom for the hint about the incompatibility of the head branch of SensorKinect. I was finally able to compile the library without that XnEvent.h error.

  36. Daniel Says:
    September 9th, 2012 at 6:26 am

    Hi
    i’m trying to get openni working on beagleboard_xm … i managed to compile the OpenNI , but KinectSensor gives XnEvent.h error. i tried to change HEAD version by “git checkout faf4994fceba82e6fbd3dad16f79e4399be0c184″ but when i run the RedisMaker it says “Unknown machine type: armv7l”
    any suggestions ?

  37. Michael Says:
    September 10th, 2012 at 7:53 pm

    Woah, what kind of machine are you compiling on? I have never tried compiling OpenNI on any arm processor, don’t know if it would work..

Leave a Reply