PDA

View Full Version : Optimal cutoff frequency for filter



Robert Newton
11-27-1994, 10:21 AM
Thankyou very much to all of you who responded to my request for information and
opinions on selecting the optimal cutoff frequency for a Butterworth filter
applied to
displacement - time data collected from a rotary encoder. Please find
following my
original posting and the replies that I received.

Regards
Robert Newton

MY POSTING
I have been analysing the signal from a rotary encoder which provides
displacement time data in an attempt to determine the optimal cut-off
frequency for filtering the data prior to differentiation to provide
velocity and acceleration data. I have been smoothing the data using
a Butterworth 4th Order digital filter with cutoff frequencies ranging from
1 to 60Hz and subsequently calculating the residual as the mean square
difference between the filtered and raw data. Having plotted the residual
against cutoff frequency I have been attempting to determine the optimal
cutoff frequency by projecting the linear part of the resulting curve to
the vertical axis and then back to the curve to determine the cutoff. The
process is outlined on pages 41-43 of David Winter's book
"Biomechanics and Motor Control of Human Movement, 2nd Edition".

My problem is that the plot at the higher cutoff frequencies is not linear
but curvelinear and I an unable to determine over what range of cutoff
frequencies should I project my line from. The calculated optimal cutoff
frequency is affected to a great extent by what range I define as the
linear part of the curve.

Can anyone provide advice on how I might determine my optimal
cutoff frequency? Is there a source of a more detailed explanation of the
method for determining optimal cutoff frequency?

REPLIES
================================================== ========
Date: Mon, 21 Nov 1994 15:38:51 -0600 (CST)
X-Ph: V4.1@genesis
From: YUB@rcf.mayo.edu
To: run1@psu.edu
Subject: RE: Optimal cutoff frequency for data smoothing

Hi, Robert,
When I was at Kansas State Univeristy, I did a study on determination of the
optimum cutoff frequency for the digital filter data smoothing procedure. I
used a set of theoretical data as standard data, and added random errors into
this set of standard data to get different sets of "raw data". The raw data
was smoothed using the digital filter you used at different cutoff frequencies.
I calculated the accelerations from the smoothed raw data. When the calculated
acceleration data had the maximum similarity with the theoretical acceleration
data, the cutoff frequency was considered as the optimum. It was found that
the optimum cutoff frequency and the sampling frequency were significantly
correlated. The optimum cutoff frequency can be estimated using

Fc = (1.4845 + 0.1523 Fs^1/2)^2

This equation explained over 75% of the total variation in the optimum cutoff
frequency. This equation has been used in the last five years for different 2d
and 3d coordinate data in different human body motions, and the results are
satisfactory. The explanation for this relationship is that the higher the
sampling frequency, the high the frequency of the random error (the further
the random error components will go to the high frequency end of the frequency
spectrum, see Dr. Winter's book, Biomechanics of human movement.). You may try
this equation if you think it makes sense or its smoothing results make sense.

I also have another equation for determination of optimum cutoff frequency,
which requires FFT and freqeuncy analysis. It explained over 85% of the total
variation of the optimum cutoff frequency. However, I found that sometimes this
equation works pretty well, sometimes doesn't. If you are interested in, I can
give you all the details.

Several years ago, I tried to get the study published in Journal of
Biomechanics as a technical note. One of the reviewer attacked me saying that
this study had no contribution to the biomechanics. But the how to determine
the optimum cutoff frequency has been frequently asked by many researchers in
biomechanics in the last several years. It may be the time for me to
re-consider publishing this study.

Bing Yu, Ph.D.
Orthopedic Biomechanics Laboratory
Mayo Clinic
Rochester, MN 55905
================================================== =======
Date: Mon, 21 Nov 1994 16:19:15 -0600
X-Ph: V4.1@genesis
From: Duane Knudson
Subject: RE>Optimal cutoff frequency for data smoothing
To: Robert Newton

Greetings Robert!

I bet you get a large number of responses to this post since data smoothing
has been a persistent problem in our field. I have RMS residuals for many
kinds of kinematic data and get curves very similar to Winter's 2.25 on page
43. The curves tend to bottom out at the measurement error for the situation.
I do not suspect the mean square error would be any different.

The problem is that even the "automated" smoothing programs essentially are
still arbitrary selections (note that Winter p. 42 says " If we decide both (
signal distortion and noise passed) should be equal . . ." The other
arbitrary "automated" method is the Jackson (1979) method that takes the
second derivative of the linear interpolation of the residuals. It may be a
chicken/egg situation where we cannot objectively separate the signal and
noise of our kinematic data. Even fourier analysis, ultimately must be based
on some guess (95% signal power?) as a good compromise of signal distortion
and nois attenuation.

We need more accelerometer studies and a common standard of what is acceptable
signal to noise ratio, or what are appropriate frequencies for specific kinds
of biomechanical data. Good luck in your quest.

Jackson, K.M. (1979) Fitting of mathematical functions to biomechanical data.
IEEE Trans Biomed Eng. 26:122-124.

================================================== ====
Robert:

Just out of curiuosity, what is your sampling rate? Since you are smoothing
up to 60 Hz, it almost sounds as if you are violating the Nyquist limit of
the Butterworth digital filter (see J. Walton's dissertation). Once beyond
(0.25 * SAMPLING_RATE), the Butterworth digital filter behaves strangely.
Could this be the problem? If, for example, you are collecting at 100 Hz,
try using your algorithm in the cutoff range of 1-25 and see if that
eliminates the strange sections of the residual curve. Good luck -- let me
know what happens.

Peter Vint
Arizona State University
Exercise and Sport Research Institute
VINT@ESPE1.LA.ASU.EDU

(Note sampling frequency was 500 Hz - Robert Newton)

================================================== ==
Date: Tue, 22 Nov 94 11:24:08 EST
X-Ph: V4.1@genesis
From: Tim=Wrigley%PhysEd_Rec%VUT@gnu.vut.edu.au
Subject: optimal filtering
To: run1@psu.edu
Cc:

Hi Rob

You might try the Jackson 'knee' method:

Jackson, KM (1979) Fitting of mathematical functions to biomechanical data.
IEEE Trans. Biomed. Eng., vol ?:122-124.

I haven't got Winter in front of me, and I can't remember the specifics of
the method he suggests. It may even be the Jackson method, in which case I
haven't helped you much !

The Jackson method is now used by the Peak system for optimal filtering by
Butterworth, cubic spline, or fourier series. It seems to work well for
kinematic data, but I haven't tried it for anything more complex.

Good luck !

Cheers

Tim
================================================== ===
X-Ph: V4.1@genesis
To: run1@psu.edu
From: "Alan Walmsley"
Organization: School of Physical Education, Otago
Date: Tue, 22 Nov 1994 15:03:17 GMT+1200
Subject: Re: Optimal cutoff frequency for data smoothing
Priority: normal

Dear Robert,

Have you considered spectral analysis to obtain the major frequency
components, and then choosing a cut-off frequency at least an octave
above the major peak?
Alan Walmsley
School of Physical Education
Division of Sciences
University of Otago
Dunedin, New Zealand.
Ph (03) 4799122, Fax (03) 4798309
=================================================
X-Ph: V4.1@genesis
To: run1@psu.edu
From: Rob Neal
Date: Tue, 22 Nov 1994 12:41:11 EST5EDT
Subject: Re: Optimal cutoff frequency for data smoothing
Priority: normal

I don't have the references but the problem seems very similar to the
one exercise physiologists have for determining ventilatory threshold
or anaerobic threshold. There are a few papers detailing various
methods to solve this problem. I could try to find them from the guys
at the QAS if you would like.

Cheers,

Rob

Robert Neal, PhD
Department of Human Movement Studies
The University of Queensland
QLD, AUSTRALIA
ph 61 7 365 6240
FAX 61 7 365 6877
EMAIL NEAL@HMS01.HMS.UQ.OZ.AU
================================================
Date: Tue, 22 Nov 1994 09:33:26 -0500 (EST)
X-Ph: V4.1@genesis
From: stuart mcgill
Subject: Re: Optimal cutoff frequency for data smoothing
To: Robert Newton

Hello Robert,

"Residual analysis" as described in Winter assumes that the noise
component is white- yours appears not to be. Perhaps you should attempt
another method- you didn't describe the signal that must be smoothed-
this would help in choosing another way to smooth. Good luck.
Stu McGill
================================================
Date: Tue, 22 Nov 1994 12:45:32 MET-DST
X-Ph: V4.1@genesis
From: "Giovanni LEGNANI. Uni. of Brescia, Italy EC"
Subject: Re: Optimal cutoff frequency for data smoothing
To: run1@psu.edu
X-Vms-To: IN%"run1@PSU.EDU"

-----------------------

The frequency should be proportional to the frequency of the incoming pulses
coming from the encoders. (you are forced to choose the maximum speed).

then you have to choose a frequency that is lower than the half of the
incoming signal of angle to avoid fenomena similar to aliasing.

so if you have an encoder haning 1000 steps, you will have 4000 samples
per turn.
if your encoder rotates ad a speed of K turns per second you have a data
frequency of 4000 Hz. I suggest you to filter chosing a low-pass filter
having a bandwith lower than 2000 Hz.
Better a little lower.

take in mind that an encoder give an approximate value for the angle.
the absolute error is 1 step. when the encoder rotates you have a noise
having an amplitude of 1 step and a frequency proportional to the
encoder speed and to the number of the encoder steps.

bye

giovanni legnani
===============================================
X-Ph: V4.1@genesis
From: "Tom Lundin"
Date: Tue, 22 Nov 94 11:04:32 EDT
Reply-To:
X-Popmail-Charset: English
To: run1@psu.edu
Subject: cutoff frequencies

Robert,

I have recently encountered a similar problem with filtering motion data.
The best algorithm I could come up with to select a cutoff was to
differentiate the RMS error vs. cutoff frequency twice and search for where
the ensuing curve approximated zero (point Z). I found the slope of the
line described by Winter from the first derivative of the RMS curve at Z.
Then using the equation of that line I found the cutoff frequency as I
presume you already know how to do. If you have any questions or comments
please feel free to write back. I hope this helps and I'll be interested
to see the other responses you receive.

Regards,
Tom Lundin
The Cleveland Clinic Foundation
================================================== ==
X-Ph: V4.1@genesis
From: Paul Guy
Subject: Re: Optimal cutoff frequency for data smoothing
To: run1@psu.edu
Date: Tue, 22 Nov 1994 12:30:55 -0500 (EST)
Content-Length: 4111

Having worked with Dave Winter for many years in his lab, I'll give
you an answer you might not want to hear.
In short, there is no decent mathematical method that I've seen based
on conventional or residual analysis that covers all the situations. If
you are interested in say just the displacements, then a residual
analysis of them will probably do, if you wanted to see what gave the
best results in a power or kinetics situation, then you'd need to do
residuals based on those variables.
The best way to deal with it, is to have some previous knowledge of
the system you are measuring, what its dynamic characteristics are, and
what the behaviour is of the data once it arrives at your computer. Such
things as whether you used interlaced video would be very important
(large 30 hz noise components), or where the resonant frequencies were
on your force plates, transducers etc.
For filtering data from the human body, we will filter different
segments at different frequencies, for example the trunk markers at 1-3
Hz, the foot at 8 to 15 Hz depending on the activity.
Where the accelerations become very important, we find that it's often
worthwhile to raise the sampling frequency, especially if you are doing
stuff like FFT's (and you need long records too). The ratio of cutoff
frequency to sampling frequency will affect whether you are really
getting an analog equivalent. Filtering at 1/4 the sampling frequency
will not give you the characteristics that you might expect.
Another issue is filter TYPE.... are you using Butterworth, Bessel,
IIR,FIR, 2-way pass etc. ? All these become an issue depending on what
you're looking at, in what domain, and how your applications are going
to react to the various 'corruptions'. For example, we use a so-called
4th order 2-way Butterworth (it's run through two 2nd order
Butterworths, the second is filtered backwards in time, to reduce delay
artifacts). Using this filter with force plates causes a force to appear
on the plate before the foot contacts it. That's clearly silly data.
Similiarly, the horizontal impulse the foot gives at heel contact can
really mess up the determination of the body kinetics.....filtering it
can spread what occurs in 10-50ms over a much longer time, rendering
your analysis useless near heel contact.
Don't buy all that theory stuff, sometimes a look at the big picture
will help more.

-Paul

-----------------------------------------------------------------------------
Paul J Guy work phone:519-885-1211 ext 6371
paul@gaitlab1.uwaterloo.ca home/FAX/:519-576-3090
pguy@healthy.uwaterloo.ca 64 Mt.Hope St.,Kitchener,Ontario,Canada

================================================== =
Date: Tue, 22 Nov 1994 11:45:26 -0600 (CST)
X-Ph: V4.1@genesis
From: "Christine Q. Wu"
To: Robert Newton
Subject: Re: Optimal cutoff frequency for data smoothing

Dear Robert: I have the similar problem. Would you please let me know if
you get any solution. Besides, is the range of motion effects the smooth
procedure? If the range of motion is low, may be comparable with the
magnitude of the noise, what will happen?
Good luck!
Christine
================================================== =
Date: 22 Nov 94 16:39:49 EST
X-Ph: V4.1@genesis
From: "Peak Performance Tech."
To: Robert Newton
Subject: Re: Optimal cutoff frequency for data smoothing

Robert-

If you haven't already, you may want to check out the "Jackson Knee
Method". It plots the 2nd derivative of the percent average residual
curve vs. the cutoff frequency. When three points on that curve fall
below a defined prescribed limit, the smallest frequency of the curve
becomes the optimal.

Jackson, K.M. Fitting of mathematical functions to biomechanical data.
IEEE Transactions on Biomedical Engineering, 1979, pp. 122-124.


*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*
George Miller
Peak Performance Technologies
Englewood, CO
76244.3047@CompuServe.com
*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*
==================================================
Date: Tue, 22 Nov 1994 21:58:37 -0600 (CST)
X-Ph: V4.1@genesis
From: "M. Pizzimenti"
Subject: Re: Optimal cutoff frequency for data smoothing
To: Robert Newton


Robert,

Try Jackson's algorithm where the residuals are differentiated.

Jackson, K.M. Fitting of mathematical functions to biomechanical data.
IEEE Transactions of Biomedical Engineering BME26(2), 122-124, 1979

Hope this helps


Marc Pizzimenti
University of Iowa
Department of Exercise Science

====================== END OF REPLIES =================






--
Robert Newton Phone Int+ 1 814
865 7107
Center for Sports Medicine Fax Int+ 1 814 865 7077
The Pennsylvania State University Email RUN1@PSU.EDU
117 Ann Building
University Park, PA 16802
United States of America