Real-time latency of motion capture systems.

[Gabriel Diaz]

I am currently scouting for a full-body motion capture system. My goal is to represent the data visually in real-time in an interactive simulation, and I am considering purchasing options. I do not trust the numbers provided by the manufacturers, and wanted to ask the group if they have any insight into the available options. I'm most curious about Vicon's new "Bonita System."

I've tested a Phasespace system running at 240 hz on a very fast workstation with high end graphics cards and an extremely minimal virtual environment, and found 33-50ms of system delay (the screen updated to reflect movement 3 frames later @ 60 hz).

We tested delay by comparing two measures of the time at which a physical object collided with the ground. Both measures were represented on an oscilloscope for comparison. To get a measure of phase-space delay, the motion-tracked object would bring about both a global change in screen luminance that was detected by a photometer, and represented on the oscilloscope. The object was also fitted with 9v battery, and a circuit-breaking switch that would trigger an instantaneous step-change in voltage upon collision with the ground. Lag was equal to the difference between the step-change in voltage and the change in screen luminance.

Does anyone have anything to add?

[Manvendra Singh]

Re: Real-time latency of motion capture systems.

Optical system's reported latencies are for 3DOF motion data(positions only). For 6DOF rigid body tracking motion data, actual latencies will be higher, variable(indeterministic) and depending on many factors.
For time critical research applications, Electromagnetic Motion Trackers are a good option, having least latnecies for 6DOF data and good repeatability of data measurements under right conditions.
But modern Optical Motion Capture systems too can provide reliable motion data for Real-time interactive simulation usage, considering the advancements in Hardwares and processing softwares/algorithms.

[Peter Shull]

Re: Real-time latency of motion capture systems.

Hi Gabriel,

I'm not sure about Vicon's Bonita system, but I ran several real-time studies with Vicon's T-Series cameras and Nexus and measured latencies of approx 75-100 ms.

Pete


--
Pete Shull
Assistant Professor
Shanghai Jiao Tong University, Shanghai, China

[Neil Schell]

Re: Real-time latency of motion capture systems.

To add to Mr. Singh's reply, if you're speaking of the latency of the tracker itself, to give you some hard numbers for Polhemus EM trackers, the latency ranges from 3.5ms (for the Liberty system) to 10ms (for the G4 system). We would be happy to share our measurement methodology to avoid any issue with trust in these specs. But essentailly we measure latency from the time the event is meaured by the sensor to the time it is ouput to the PC. This includes position and orientation, and is unaffected by environmental conditions.

[Ross Miller]

Re: Real-time latency of motion capture systems.

At my post-doc we had some demos from all the major vendors (Vikon, Qualisys, Motion Analysis) on the latency in reconstructing and displaying marker positions in real-time. If I remember correctly they were all around 10-20 ms. The vendors can probably tell you specifically what it is for certain systems, and in my experience their reports were accurate.

Ross

[Lasse Roren]

Re: Real-time latency of motion capture systems.

I am currently scouting for a full-body motion capture system. My goal is to represent the data visually in real-time in an interactive simulation, and I am considering purchasing options. I do not trust the numbers provided by the manufacturers, and wanted to ask the group if they have any insight into the available options. I'm most curious about Vicon's new "Bonita System."

I've tested a Phasespace system running at 240 hz on a very fast workstation with high end graphics cards and an extremely minimal virtual environment, and found 33-50ms of system delay (the screen updated to reflect movement 3 frames later @ 60 hz).

We tested delay by comparing two measures of the time at which a physical object collided with the ground. Both measures were represented on an oscilloscope for comparison. To get a measure of phase-space delay, the motion-tracked object would bring about both a global change in screen luminance that was detected by a photometer, and represented on the oscilloscope. The object was also fitted with 9v battery, and a circuit-breaking switch that would trigger an instantaneous step-change in voltage upon collision with the ground. Lag was equal to the difference between the step-change in voltage and the change in screen luminance.

Does anyone have anything to add?

Hi Gabriel,

As it happens, I did extensive tests with a Vicon system last year to try and assess the real time latency, and I measured the latency to be below ten milliseconds.

The experimental setup in my case was as follows: I put three markers on an old-fashioned analog goniometer, which were measured by the Vicon system and streamed through the Vicon Nexus software. I wrote a little program that connected to the real time stream from the Nexus software and calculated the angle between the 3 markers. I also had a National Instruments AD card with analog outputs, where I could set an output voltage as follows: when the angle between the 3 markers exceeded 45 degrees, I set the output voltage to 5 volts, when the angle was less than 45 degrees, I set the output voltage to 0 volts.

I then connected both the analog output directly from the goniometer as well as the analog output from the National Instruments card to a AD card hooked up to a separate computer and recorded the data as I changed the goniometer repeatedly from 0 to 90 degrees and back. This would then generate a reading of degrees from the goniometer, as well as a pulse triggered at the 45 degree point from the Vicon system via the National Instruments card.

Assuming that the analog goniometer's direct reading has zero latency, it was then possible to measure the Vicon system's latency by seeing how far behind the pulse "lagged" the goniometer's reading. Of course, the 45 degree point for the goniometer wasn't 100% calibrated with the 45 degree point for the 3 markers, but by taking the average of the latency between the "going up" and "going down" pulse points, the actual system latency could be calculated.

The latency measured consisted of the following components:
1) The time from the marker's movement to the Vicon camera's shutter opening and then closing
2) The time for the Vicon camera to read the data off the sensor and send it to the PC
3) The time for Nexus to compute the XYZ positions of the markers
4) The time to send this information to and process in my own real time software
5) The time to raise the analog output pulse in the National Instruments card.

The faster you run the Vicon cameras, the shorter the latency. In my experiment, I ran the cameras at 250Hz. At this speed, I consistently measured latencies below ten milliseconds. Note that steps 4) and 5) - latencies not caused by the Vicon system - are included in this number.

It is worth mentioning that Nexus has a system parameter called "Minimize Latency", which must be checked to achieve the best results. It is also worth mentioning that the Vicon Tracker software is even more optimized for real time, and I believe that others have measured even shorter latencies with this software.

Best regards,

[Meredith Evans]

Re: Real-time latency of motion capture systems.

Hi Gabriel,

I understand your wariness of vendor specifications, as vendor measurements do not always address the complete latency issue. We have been doing real-time biofeedback applications since our inception 20 years ago and have learned the hard way about latency; their sources; and ways to test for them. Most of our applications involve the capture of data from multiple sources with feedback to the subject in the form of audio, visual, analog or digital output. We have several examples at: http://innsport.com/videos/biofeedback.aspx

We have found that latency has many sources. These include:

1. Display refresh rates. At 60 hz there is a minimum of 16ms delay between the processing of data and its display. Even in our stereoscopic immersive displays that have refresh rates of 120hz, the presentation to each eye sequentially means that you have a 16ms delay.

2. Data Processing. There is a minimum of one frame of data processing required before data is available in a form that can be displayed visually. The visual’s complexity, graphics card, multi-threading software design and multi-threading hardware capabilities are all factors that affect how fast the display can be “repainted”. If data is not available and processed faster than 60hz you will be adding to the delay in point 1.

3. Communication time between the tracking device’s processing unit and the Data Processing Software is also a factor. Some devices are collected directly on the Data Processing Software computer (PCI based A/D cards; MotionStar cards from Ascension; Local Host Ethernet for Vicon, MAC, Qualisys, Optitrack cameras) and conceivably add only 1 sample frame to the delay. However even with these connections drivers may introduce delays, eg. block transfers of Measurement Computing AD data involve 10ms delays regardless of sampling rate. Other hardware may be connected externally by Ethernet or USB which can have additional sources of delay.

4. Communication time between sensor and its processing unit. Most tracking devices require processing of signals collected at the sensor. For optical systems, the transfer time from camera to processor is very different for an Ethernet based system than a USB based system because of data buffers. If the devices are wireless (eg. IMU’s, Noraxon DTS or Delsys Trigno) you have air time to consider as well.

Bottom line there is not a simple answer to this question and the system latency will depend on the integrated system’s design rather than a single component. We have both Vicon Bonita 10s and T20’s running on The MotionMonitor base unit in our training center. We have measured the latency from recorded event to feedback at 1-3 frames or 16-48ms depending on measurement rate. Let me know if this is helpful or if you would like to talk further.

Best,

Meredith Evans

Client Support Engineer
mevans@TheMotionMonitor.com
Innovative Sports Training, Inc; www.TheMotionMonitor.com