Robert Newton

12-07-1994, 12:54 AM

I have received a further two excellent replies to my request for advice

on the optimal cutoff frequency for filtering noisy data. Unfortunately

they arrived too late to include in my original summary so I am

including them in this second posting.

Regards

Robert Newton

=================================================

I have a couple of points concerning the use of residuals for choosing

filter cut-off frequencies.

The residual method was I believe developed in order to filter noisy

data collected with a camera based system. The technique relies on the

residuals vs filter cut-off frequency plot becoming linear as the

increasing cut-off frequency of the filter attenuates the spectrally

flat white noise at the high end of the spectrum close to fs/2. If

there is not a section of white noise at the high frequency end then

the plot will not become linear. The Jackson method is a way of

detecting linearity by detecting when the second derivative becomes

zero.

Is it possible that your encoder doesnt actually have a significant

noise problem. If it is a digital encoder where the angle is converted

directly to a digital code within the encoder itself then there

shouldnt be noise from electrical inteference. Is there noise in the

data when the encoder is stationary ? There will of course be the

unavoidable (white) digitisation noise present in all A/D systems when

the encoder moves and the signal is varying.

I suspect that the residuals at high frequencies close to fs/2 are

insignificant compared to your signal power. In that case you could

choose a cut-off frequency at say the 99% power limit, ie the

frequency below which 99% of the power occurs ( Antonsson and Mann

1985). Alternatively if you can afford one you could buy a tachometer

and measure velocities directly (which would improve your angular

acceleration estimate also).

I havent seen any other references to the residual technique, but of

course there are a great many discussing optimal filtering of noisy

data by Woltring etc.

Antonsson, E.K. and Mann R.W. (1985) The Frequency Content of Gait,

J. Biomechanics, 18, 1, 39-47.

Best Wishes

Duncan Rand

================================================== =

After reading the responses to your request regarding the determination

of the optimal cutoff frequency for filtering noisy data sequences prior to

differentiation, I was surprised about the lack of information about existing

methods that prevails among biomechanists.

The problem has been dealt with in detail, and solved in principle, by Herman

Woltring and myself in the early 1980's. While Woltring used spline functions

(based on Utreras' work and computer programs),I developed an optimally

regularizing algorithm based on Fourier series approximation. In both cases,

the optimal filtering window (which determines the optimal cutoff frequency) is

computed automatically, not only for the data sequence itself, but also for the

(optimally filtered) first and second derivatives. The latter are computed

simultaneously with the optimally smoothed data sequences, thereby providing

an efficient and accurate means of rapidly processing noisy biomechanical data.

The user does not have to bother about estimating cutoff frequencies, as these

are automatically and optimally determined by the algorithm which performs

(automatically) a detailed statistical analysis of the information contained in

the

periodogram of the noise-contaminated data sequences. The only requirement

on the data is that their detrended version should represent a weakly stationary

stochastic process which requirement is fulfilled by most data sequences

occuring

biomechanical investigations. Details of the method can be found in the Journal

of Biomechanics, 14, pp. 13-18, 1981 (The use of optimally regularized Fourier

series for estimating higher-order derivatives of noisy biomechanical data),

which article also contains some of the shortcomings inherent in Jackson's

method.

Hope this hint is helpful.

H. Hatze

===================== End of Summary =======================

--

Robert Newton Phone Int+ 1 814

865 7107

Center for Sports Medicine Fax Int+ 1 814 865 7077

The Pennsylvania State University Email RUN1@PSU.EDU

117 Ann Building

University Park, PA 16802

United States of America

on the optimal cutoff frequency for filtering noisy data. Unfortunately

they arrived too late to include in my original summary so I am

including them in this second posting.

Regards

Robert Newton

=================================================

I have a couple of points concerning the use of residuals for choosing

filter cut-off frequencies.

The residual method was I believe developed in order to filter noisy

data collected with a camera based system. The technique relies on the

residuals vs filter cut-off frequency plot becoming linear as the

increasing cut-off frequency of the filter attenuates the spectrally

flat white noise at the high end of the spectrum close to fs/2. If

there is not a section of white noise at the high frequency end then

the plot will not become linear. The Jackson method is a way of

detecting linearity by detecting when the second derivative becomes

zero.

Is it possible that your encoder doesnt actually have a significant

noise problem. If it is a digital encoder where the angle is converted

directly to a digital code within the encoder itself then there

shouldnt be noise from electrical inteference. Is there noise in the

data when the encoder is stationary ? There will of course be the

unavoidable (white) digitisation noise present in all A/D systems when

the encoder moves and the signal is varying.

I suspect that the residuals at high frequencies close to fs/2 are

insignificant compared to your signal power. In that case you could

choose a cut-off frequency at say the 99% power limit, ie the

frequency below which 99% of the power occurs ( Antonsson and Mann

1985). Alternatively if you can afford one you could buy a tachometer

and measure velocities directly (which would improve your angular

acceleration estimate also).

I havent seen any other references to the residual technique, but of

course there are a great many discussing optimal filtering of noisy

data by Woltring etc.

Antonsson, E.K. and Mann R.W. (1985) The Frequency Content of Gait,

J. Biomechanics, 18, 1, 39-47.

Best Wishes

Duncan Rand

================================================== =

After reading the responses to your request regarding the determination

of the optimal cutoff frequency for filtering noisy data sequences prior to

differentiation, I was surprised about the lack of information about existing

methods that prevails among biomechanists.

The problem has been dealt with in detail, and solved in principle, by Herman

Woltring and myself in the early 1980's. While Woltring used spline functions

(based on Utreras' work and computer programs),I developed an optimally

regularizing algorithm based on Fourier series approximation. In both cases,

the optimal filtering window (which determines the optimal cutoff frequency) is

computed automatically, not only for the data sequence itself, but also for the

(optimally filtered) first and second derivatives. The latter are computed

simultaneously with the optimally smoothed data sequences, thereby providing

an efficient and accurate means of rapidly processing noisy biomechanical data.

The user does not have to bother about estimating cutoff frequencies, as these

are automatically and optimally determined by the algorithm which performs

(automatically) a detailed statistical analysis of the information contained in

the

periodogram of the noise-contaminated data sequences. The only requirement

on the data is that their detrended version should represent a weakly stationary

stochastic process which requirement is fulfilled by most data sequences

occuring

biomechanical investigations. Details of the method can be found in the Journal

of Biomechanics, 14, pp. 13-18, 1981 (The use of optimally regularized Fourier

series for estimating higher-order derivatives of noisy biomechanical data),

which article also contains some of the shortcomings inherent in Jackson's

method.

Hope this hint is helpful.

H. Hatze

===================== End of Summary =======================

--

Robert Newton Phone Int+ 1 814

865 7107

Center for Sports Medicine Fax Int+ 1 814 865 7077

The Pennsylvania State University Email RUN1@PSU.EDU

117 Ann Building

University Park, PA 16802

United States of America