create new tag
, view all tags
-- AlexandreProust - 2011-08-17

How use Kinect with OpenCV

OpenCV fonctions (libfreenect_cv)

Yours programs using OpenCV need to be located in the directory


Indeed, you need two files to get frames and control your kinect. The header of your program will have:

 #include "libfreenect_cv.h"

This file (libfreenect_cv.h) has essentials functions to get RGB frame, Depth frame, control le led color and stop the Kinect. These functions are written in the second file (libfreenect_cv.cpp) and use the functions of the driver files "libfreenect.h" and "libfreenect_sync.h"

If you want to add function for an OpenCv utilisation you need to write your code in the libfreenect_cv.cpp file. After add the header of your function in the libfreenect_cv.h.

Get RGB frame

To get a RGB frame we will use the function “freenect_sync_get_rgb_cv”. It returns an IplImage* with a size of 640*480 on 8bits with 3 channels. This frame is in RGB color so if you want the real color you need convert in BGR color.

In c++ you can use this function like that:

while (waitKey(1)<10) {
///get rgb frame from the kinect
                     Mat image(freenect_sync_get_rgb_cv(0));
                                if (image.empty()) {
                                           printf("Error: Kinect not connected?\n");
                                           return -1;
                     cvtColor(image, image, CV_RGB2BGR);//convert color
                     flip(image.t(),image,0); //rotate image
imshow( "RGB", image);//rgb display


Get Depth Frame

To get a Depth frame we will use the function “freenect_sync_get_depth_cv”. It returns an IplImage* with a size of 640*480 on 16bits with 1 channel.

We can use this function like that :

 ///get depth frame from the kinect
      IplImage *depth(freenect_sync_get_depth_cv(0));
         if (!depth) {
            printf("Error: Kinect not connected?\n");
            return -1;

Actually the depth information is stored in &data of the IplImage variable. So you can’t have directly a depth representation. We will use a function called GlViewCode to convert depth data in grayscale color.

IplImage* GlViewCode(IplImage *depth)
   static IplImage *image = 0;
   if (!image) image = cvCreateImage(cvSize(640,480), 8, 3);
   unsigned char *depth_mid = (unsigned char*)(image->imageData);
   int i;

      for (i = 0; i < 480*640; i++) {
         int lb = ((short *)depth->imageData)[i]/8;
         depth_mid[3*i+2] = lb;
         depth_mid[3*i+1] = lb;
         depth_mid[3*i+0] = lb;
   return image;

Now we can display the depth frame like that :

cvShowImage( "Depth", GlViewCode(depth));


Topic attachments
I Attachment Action Size Date Who Comment
PNGpng Depth_view.png manage 88.6 K 2011-08-17 - 06:48 AlexandreProust  
PNGpng RGB_view.png manage 573.4 K 2011-08-17 - 06:48 AlexandreProust  
C source code filecpp libfreenect_cv.cpp manage 1.5 K 2011-08-17 - 06:47 AlexandreProust  
Header fileh libfreenect_cv.h manage 0.5 K 2011-08-17 - 06:47 AlexandreProust  
Topic revision: r1 - 2011-08-17 - AlexandreProust
This site is powered by the TWiki collaboration platformCopyright © 2008-2020 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback