PDA

View Full Version : Latency in real-time motion capture systems



greaves46
12-18-2007, 07:44 AM
Regarding Ravi's posting on BIOMCH-L and the latency of data in real time capture systems:

You have heard that there are several items that can affect the latency associated with a motion capture system. There are two things that we at Motion Analysis have found to be the most relevant:

1- Frame Rate: The most important item is the capture frame rate. This generally depends more on the performance of the tracking software than the maximum speed of the cameras: You can turn the frame rate fairly simply on a camera, but to get the performance from the other "factors" in a system is the biggest challenge. Such as 3D ray intersections, such as identifying markers in real time, such as skeleton calculations and especially robustness. The biggest delays are measured in frames, so you want to keep the frame rate as high as possible to keep the delay as small as possible. The critical question for any system is: "What is the maximum frame rate for real time operation?". The MAC system operates with full camera resolution up to 500 fps in real time which translates to a low latency.

2- Tool to Measure Latency: Some of our robotics customers are using real time feedback and they asked the same questions. And they needed to know and it is unfortunately NOT SIMPLE to figure out the latency without tools. No simple answer because every setup is different and the variables are complex. Like: How fast are the processors? How many CPUs? How many threads? What type of camera ? What frame rate for the camera? How many markers? How many horses? Simple or optimized skeleton? With or without analog data? What analog sample rate? So we worked with them and came up a software tool which tells them in real time the precise delay for each frame of data. From the time the strobe flashes in the capture volume to the instant the UDP data packet pops out Ethernet port to the waiting application.

The packet with marker and or analog and or skeleton data is received in microseconds via the Gigabit Ethernet channel. Now the researchers can make their own tradeoffs: Do we operate the system at a higher frame rate? Do we have time for tracking more markers? Do we need a faster CPU or more of them? Can we calculate skeletons with global optimization? How many bones can we process? They can measure and decide for themselves.

Good luck,
John Greaves,
Sr. Technical Guy, Motion Analysis