Acceptable Vicon Image Errors

[jons47]

Dear All

I appreciate that all setups for Vicon Nexus Systems differ from lab to lab, however would be interested to find out what values of image error are acceptable to users.
This may allow us to (a) set representative limits and (b) quantify the effect of this error within repeatability studies. As a rough average guide we normally can get MX3 cameras to below 0.1 and T40’s around or below 0.13.

Also I would be further interested how other labs define or quantify marker/force vector accuracy on the 2D video image. when using the 3D overlay.

Thanks

Jon Spriggs
Lab Technician
Specialist Services
Clinical Measurements Laboratory
Birmingham Community Healthcare NHS Trust
West Midlands Rehabilitation Centre
91 Oak Tree Lane, Selly Oak.
Birmingham. B29 6JA
Tel: 0121 6271627 ext 58311

Email: jonathan.spriggs@sbpct.nhs.uk

[ecramp48]

While the "errors" reported by the software calibration functions are a guide to the system performance, realistically you're probably more interested in the accuracy of the 3D reconstruction from day to day. Since the Vicon system outputs its data using the C3D file format you can see the results of the 3D construction by looking at the "residual" values, stored in the C3D files for each frame of 3D data.

"The residual of a 3D Point is calculated at the same time that the position of the 3D point is calculated (or reconstructed). It is the average error distance, calculated by the photogrammetry software, which prevents all measurement rays from meeting at an identical point in space." - www.c3d.org (http://www.c3d.org)

In general, lower residual numbers tend to indicate that the 3D point locations are more accurate when these numbers are derived from measured 2D data and will always be absolute, non-zero values. Residual values of zero indicate that the point was not directly derived from measurements, i.e. it was generated by interpolation or filtering. Negative residual values indicate that the point is invalid.

A free program that allows you to see the C3D residual values is the MLSviewer (http://www.motion-labs.com/index_downloads.html) from our web site (http://www.motion-labs.com) - other free applications on the same Motion Lab Systems web site allow you to edit, and change the residual values, as well as access the C3D data from MATLAB, LabVIEW, Visual Basic, and Excel etc.


In a perfect world the 3D point residual values stored in the C3D file will always be low and will not vary by much from frame to frame - but in the real world this rarely happens unless the markers are not moving. Large changes in residual values from frame to frame usually indicate that one of more cameras have stopped contributing to the 3D reconstruction for some reason - resulting in a lower confidence in the actual physical marker location. Sudden changes in the residual values usually explain sudden bursts of noise or momentary spikes in the 3D data and in many cases, the camera contribution mask will show you which camera stopped providing information. See the C3D web site (http://www.c3d.org) for more information about this or look at the motion capture system documentation.

So a simple test - recording a single marker moving throughout in the data collection area - will tell you much more about the performance of your 3D motion capture system when you look at the 3D residual information than the numbers produced by calibration procedures.

Edmund Cramp
Motion Lab Systems, Inc. (http://www.motion-labs.com)

[lroren39]

Just adding to the previous post:

The calibration residuals in Vicon Nexus are reported in camera sensor pixels. Therefore, even if the value 0.13 for the T40 indicates a larger error than the 0.1 for the MX3, the actual error is much smaller because the pixels are smaller. A camera's calibration residual will also typically depend on the lens type, the distance to the capture volume, the recorded movement of the calibration wand and the lens settings such as focus and aperture. It is not straightforward to relate the camera calibration residual to 3D accuracy. However, if a camera's calibration residual increases slowly over time, or if it is much bigger in one camera than other cameras in the system with the same lens, this should be a signal to investigate further. It could, for example, indicate that the camera is out of focus, or that the calibration wand has been damaged.

In other words, whilst low calibration residuals do not necessarily guarantee good 3D accuracy, significantly higher than normal residuals certainly indicate that something has changed which could negatively influence accuracy. Of course, there's going to be an inherent variability due to the random nature of the wand wave, but if you keep a record of the residuals you should be able to spot both long term trends and "blips". They are also stored in the XCP file along with the trial, which is an XML-format text file that you can load into Nexus or a text editor.

The 3D residual values recorded in the C3D file can indeed be useful for assessing 3D accuracy. However, bear in mind the following:
1) The C3D file only stores the camera contribution masks for the first 7 cameras, for legacy reasons.
2) Once trajectories have been filtered or otherwise processed, the residual information is lost.

If you want some 3D accuracy quantification as part of the experimental protocol, the most common method is, in my experience, to record a rigid object with some markers on it being moved around in the volume, and then quantify the variability in the calculated distance between the markers.

There is also the Standard Assessment of Motion System Accuracy (http://www.gcmas.org/samsa) (SAMSA), which is a standard device and protocol used to assess accuracy in gait labs.

Best regards,
Lasse Roren
Prophysics AG (http://www.prophysics.ch)

[ecramp48]

If you want some 3D accuracy quantification as part of the experimental protocol, the most common method is, in my experience, to record a rigid object with some markers on it being moved around in the volume, and then quantify the variability in the calculated distance between the markers.

Thanks Lasse - I'd like to add two quick notes:

Many labs use the Motion Lab Systems MTD-2 (http://www.motion-labs.com/prod_qa_mtd.html) to verify their force plate location - the MTD-2 is a rod with five markers on short wands and it's an ideal "quick and dirty" test object. Just walk it around the data collection volume to perform the test that Lasse describes.

Many labs also have a "calibration wand" - because this wand is used to calibrate the Motion Capture system you cannot use it to test the system accuracy - you must use a different object. The reason for this is that any difference between the stated dimensions of the calibration wand and the actual dimensions of the calibration wand will be compensated for in the system calibration. The photogrammetry software will always report that the calibration wand is the length that you enter, even if it has to distort the calibration volume to make the numbers come out right.

[DeQuinceyPDF]

3D imaging (http://www.rasteredge.com/how-to/csharp-imaging/image-converting/) is a technique for creating or enhancing the illusion of depth in an image by means of stereopsis for binocular vision, these two-dimensional images (http://en.wikipedia.org/wiki/3D_image) are then combined in the brain to give the perception of 3D depth. This technique is distinguished from 3D displays that display an image in three full dimensions, allowing the observer to increase information about the 3-dimensional objects being displayed by head and eye movements.