Tweet
Dear All Biomech-L subscribers,
The issue of real time latency from Vicon in particular, and motion
capture systems in general has been raised on this list recently.
To (hopefully) clarify the situation, I would like to explain what
contributes to latency, and how the data flows through the Vicon system.
I assume other passive marker-based optical motion capture systems are
similar, however the below analysis applies to Vicon MX systems only.
This is the chain of events:
1) A number of markers are moving around in the calibrated Vicon space.
The strobe connected to the MX camera will emit light, which will
reflect off the markers and be projected back onto the digital sensor
inside the camera. The freeze-frame shutter used by the Vicon cameras
will open to co-incide with the strobe light. For low frame rates (up to
approx. 120Hz) the shutter is open for 1ms, but as the frame rate
increases the shutter time (and strobe time) drops according to the
formula t (in ms) = 120 Hz / frame rate (in Hz).
2) As soon as the shutter closes, the camera will scan the sensor. A
digital sensor is scanned line by line, and this takes some time. The
Vicon Vegas 4 mega-pixel sensor used in MX-F40s is able to scan all 4
mega-pixels in 2.4ms, whereas the sensor used by the previous generation
MX40 takes 5ms to scan the sensor. At 1000Hz, the sensor is only
partially scanned, and the scanning takes 1ms minus the time the shutter
is open at that speed, i.e. 1ms - 0.12ms = 0.88ms (obviously it can't
take longer).
3) The data is then circle fitted and packaged up for transmission to
the PC via ethernet. The time this takes depends on the amount of data
to process, but for low marker counts (say 10-40) it is less than 0.5ms.
4) The PC then has to gather all ethernet packages from all cameras. If
the amount of data is limited, say 10-40 markers, this takes
microseconds since the cameras send centroid (x,y) data only.
5) The PC then reconstructs and labels all the camera data. Clearly, the
time this takes is heavily dependent on the number of cameras, the
number of markers, the power of the PC's CPU(s) and the complexity of
the labelling problem. However, again working with typical marker sets
of 10-40 markers, experiments have indicated that this takes on average
1ms to 5ms.
6) Finally, the PC does something with the data. For example, it can
render to the screen, generate a sound or control a device. Rendering to
screen or projector introduces significant latency, depending on the
update frequency of the screen and how the data is rendered, but this
could well add 10-30ms to the latency, and is largely dependent on other
factors than those that can be controlled by the manufacturer of the
motion capture system.
It is worth noting that performance will depend on the number of other
tasks the Windows operating system performs. It is therefore important
to avoid as much as possible running other applications and services as
these may cause increased average latency.
The above figures should give the correct ball park: at 1000Hz the
latency on the camera + transmission to the PC should be in the region
of 1.5ms, at 100Hz in the region of 3ms. In addition there is the
processing that takes place on the PC, which is dependent on the PC
itself, the number of markers/cameras, the complexity of the labelling
problem and other processes/services that compete for the CPU time. This
has experimentally been shown to be between 1 and 5 ms for commonly used
marker sets between 10 and 40 markers.
Comparing passive optical systems to active marker systems is also
interesting. Active marker systems have one major advantage and one
major disadvantage when it comes to latency. The advantage is that
active systems do not have to do real time labelling of the
reconstructed 3D points since only one marker is visible at the time.
The disadvantage is that because only one marker can be visible at the
time, the latency will scale linearly with the marker count. For
example, if the active system tracks at 5000Hz, each marker will add
0.2ms to the inherent measurement latency, so 10 markers will take 2ms,
20 markers 4ms, etc.
Best regards,
Lasse Roren
Product Manager Life Sciences
Vicon
__________________________________________________ ______________________
This e-mail, and any attachment, is confidential. If you have received it in error, do not use or disclose the information in any way, notify me immediately, and please delete it from your system.
__________________________________________________ ______________________
The issue of real time latency from Vicon in particular, and motion
capture systems in general has been raised on this list recently.
To (hopefully) clarify the situation, I would like to explain what
contributes to latency, and how the data flows through the Vicon system.
I assume other passive marker-based optical motion capture systems are
similar, however the below analysis applies to Vicon MX systems only.
This is the chain of events:
1) A number of markers are moving around in the calibrated Vicon space.
The strobe connected to the MX camera will emit light, which will
reflect off the markers and be projected back onto the digital sensor
inside the camera. The freeze-frame shutter used by the Vicon cameras
will open to co-incide with the strobe light. For low frame rates (up to
approx. 120Hz) the shutter is open for 1ms, but as the frame rate
increases the shutter time (and strobe time) drops according to the
formula t (in ms) = 120 Hz / frame rate (in Hz).
2) As soon as the shutter closes, the camera will scan the sensor. A
digital sensor is scanned line by line, and this takes some time. The
Vicon Vegas 4 mega-pixel sensor used in MX-F40s is able to scan all 4
mega-pixels in 2.4ms, whereas the sensor used by the previous generation
MX40 takes 5ms to scan the sensor. At 1000Hz, the sensor is only
partially scanned, and the scanning takes 1ms minus the time the shutter
is open at that speed, i.e. 1ms - 0.12ms = 0.88ms (obviously it can't
take longer).
3) The data is then circle fitted and packaged up for transmission to
the PC via ethernet. The time this takes depends on the amount of data
to process, but for low marker counts (say 10-40) it is less than 0.5ms.
4) The PC then has to gather all ethernet packages from all cameras. If
the amount of data is limited, say 10-40 markers, this takes
microseconds since the cameras send centroid (x,y) data only.
5) The PC then reconstructs and labels all the camera data. Clearly, the
time this takes is heavily dependent on the number of cameras, the
number of markers, the power of the PC's CPU(s) and the complexity of
the labelling problem. However, again working with typical marker sets
of 10-40 markers, experiments have indicated that this takes on average
1ms to 5ms.
6) Finally, the PC does something with the data. For example, it can
render to the screen, generate a sound or control a device. Rendering to
screen or projector introduces significant latency, depending on the
update frequency of the screen and how the data is rendered, but this
could well add 10-30ms to the latency, and is largely dependent on other
factors than those that can be controlled by the manufacturer of the
motion capture system.
It is worth noting that performance will depend on the number of other
tasks the Windows operating system performs. It is therefore important
to avoid as much as possible running other applications and services as
these may cause increased average latency.
The above figures should give the correct ball park: at 1000Hz the
latency on the camera + transmission to the PC should be in the region
of 1.5ms, at 100Hz in the region of 3ms. In addition there is the
processing that takes place on the PC, which is dependent on the PC
itself, the number of markers/cameras, the complexity of the labelling
problem and other processes/services that compete for the CPU time. This
has experimentally been shown to be between 1 and 5 ms for commonly used
marker sets between 10 and 40 markers.
Comparing passive optical systems to active marker systems is also
interesting. Active marker systems have one major advantage and one
major disadvantage when it comes to latency. The advantage is that
active systems do not have to do real time labelling of the
reconstructed 3D points since only one marker is visible at the time.
The disadvantage is that because only one marker can be visible at the
time, the latency will scale linearly with the marker count. For
example, if the active system tracks at 5000Hz, each marker will add
0.2ms to the inherent measurement latency, so 10 markers will take 2ms,
20 markers 4ms, etc.
Best regards,
Lasse Roren
Product Manager Life Sciences
Vicon
__________________________________________________ ______________________
This e-mail, and any attachment, is confidential. If you have received it in error, do not use or disclose the information in any way, notify me immediately, and please delete it from your system.
__________________________________________________ ______________________