Kinect V2 for Skeleton Tracking

[Achu Byju]

Hi all

Does anyone have any experience working with the Kinect V2 sensor and SDK? We're looking to use it for some very simple kinematic measurements and need a way to have it track the skeleton and push coordinates to a data file. If you know of software that does this, or have a little code snippet that reads the skeleton that the SDK spits out, do let me know.

Thanks!
Achu Byju
Motion Biomechanics Lab
Texas A&M University

[Bill Sellers]

Re: Kinect V2 for Skeleton Tracking

Yes, I just implemented a Kinect V2 interface to GaitSym for public engagement events. It was very easy and I'll be releasing the code fairly soon. It lets visitors interactively control dinosaur simulations. The SDK comes with code examples that do what you want (the BodyBasics-D2D demo). As long as you don't mind working in Windows 8 using Visual Studio then it really is straightforward. I ended up using Qt because I prefer that and that wasn't too bad either. As far as I know, with version 2 (unlike version 1) there are no open source libraries. The V2 sensor is awesome although not terribly accurate, however it is ideal for certain sorts of measurements.

Cheers
Bill

--
Bill Sellers, Tel 01612751719, Mob 07857655786, http://www.animalsimulation.org
University of Manchester, D1239 Michael Smith Building, Manchester, M13 9PT.

[Achu Byju]

Re: Kinect V2 for Skeleton Tracking

Thanks Bill!

The V2 sensor is awesome although not terribly accurate, however it is ideal for certain sorts of measurements.

Would you mind elaborating on this? What level of accuracy have you gotten from the new sensor? We're just looking to make some simple trunk angle measurements and we're evaluating the Kinect sensor for this application.

[Bill Sellers]

Re: Kinect V2 for Skeleton Tracking

It depends on distance. The Kinect V2 is a 30 fps depth camera with a resolution of something like 500x400. It works form about 1m to 4m so when you are close the resolution is quite good - 2mm or so but this rapidly drops off to 5mm or so. I've not actually played with the raw depth maps that you get out - the actual numbers might be a bit better than that. I've been using the joint centres that come out from the software fitting a figure to the depth map. This is very clever (it can track up to 6 people at one), but it alters the data quite dramatically to get the fit to work. This means that joints can jump around quite a bit from frame to frame. Saying that, if you have a well designed experiment where the figure fitting can work well then I expect that would be OK. You can also work with the depth map directly which is likely to be much more accurate for a scientific study. If you are measuring the trunk angle I'd be quite tempted to go directly from the point cloud and just fit an appropriate shape to the data. You wouldn't even need to do any programming for that because I think there is a demo that saves the depth data (and if there isn't, it is really easy to access). Then you could use something like CloudDigitiser to digitise a few points from the clouds which would let you check what the accuracy is like. The built in skeleton fitting will give you about 4 points along the back but I think you'd find that data was quite variable in quality.

I keep hearing rumours that Microsoft might be making a clinical version of the Kinect which would be great if true. If they could double the resolution and the framing rate then it would be much more useful for motion analysis. It is an unbelievable clever piece of kit (especially for the money) but it isn't quite precise enough for a lot of uses. Saying that, the right software with the right Kalman filters tuned for a specific experiment would probably make the current system very usable. Also some sort of add on gating so you could use more than one at once shouldn't be impossible. A system with 8 Kinect V2 sensors would be stunning.

Cheers
Bill

[Achu Byju]

Re: Kinect V2 for Skeleton Tracking

Thanks! That's useful info. We're looking into using the depth data directly right now.

We also found a little piece of code that just exports the joint locations that the SDK spits out in case anyone else is interested.

https://github.com/ininex/KinectV2_to_MATLAB

[Paavo Vartiainen]

Re: Kinect V2 for Skeleton Tracking

Hi,
I quicly reviewed Kinect V2 publications focusing on gait analysis applications. I am trying to figure out whether Kinect could be used to build a simple mocap system for treadmill walking or running. The frame rate 30 Hz seems to be a major limitation.
I found a paper "Model-Based Tracking at 300Hz Using Raw Time-of-Flight Observations":

http://www.cv-foundation.org/openaccess/content_iccv_2015/html/Stuhmer_Model-Based_Tracking_at_ICCV_2015_paper.html

Does anyone happen to know similar applications, where frame rate higher than 30Hz has been extracted from Kinect?