No announcement yet.

Optimal cutoff frequency for filter

This topic is closed.
  • Filter
  • Time
  • Show
Clear All
new posts

  • Optimal cutoff frequency for filter

    Thankyou very much to all of you who responded to my request for information and
    opinions on selecting the optimal cutoff frequency for a Butterworth filter
    applied to
    displacement - time data collected from a rotary encoder. Please find
    following my
    original posting and the replies that I received.

    Robert Newton

    I have been analysing the signal from a rotary encoder which provides
    displacement time data in an attempt to determine the optimal cut-off
    frequency for filtering the data prior to differentiation to provide
    velocity and acceleration data. I have been smoothing the data using
    a Butterworth 4th Order digital filter with cutoff frequencies ranging from
    1 to 60Hz and subsequently calculating the residual as the mean square
    difference between the filtered and raw data. Having plotted the residual
    against cutoff frequency I have been attempting to determine the optimal
    cutoff frequency by projecting the linear part of the resulting curve to
    the vertical axis and then back to the curve to determine the cutoff. The
    process is outlined on pages 41-43 of David Winter's book
    "Biomechanics and Motor Control of Human Movement, 2nd Edition".

    My problem is that the plot at the higher cutoff frequencies is not linear
    but curvelinear and I an unable to determine over what range of cutoff
    frequencies should I project my line from. The calculated optimal cutoff
    frequency is affected to a great extent by what range I define as the
    linear part of the curve.

    Can anyone provide advice on how I might determine my optimal
    cutoff frequency? Is there a source of a more detailed explanation of the
    method for determining optimal cutoff frequency?

    ================================================== ========
    Date: Mon, 21 Nov 1994 15:38:51 -0600 (CST)
    X-Ph: V4.1@genesis
    Subject: RE: Optimal cutoff frequency for data smoothing

    Hi, Robert,
    When I was at Kansas State Univeristy, I did a study on determination of the
    optimum cutoff frequency for the digital filter data smoothing procedure. I
    used a set of theoretical data as standard data, and added random errors into
    this set of standard data to get different sets of "raw data". The raw data
    was smoothed using the digital filter you used at different cutoff frequencies.
    I calculated the accelerations from the smoothed raw data. When the calculated
    acceleration data had the maximum similarity with the theoretical acceleration
    data, the cutoff frequency was considered as the optimum. It was found that
    the optimum cutoff frequency and the sampling frequency were significantly
    correlated. The optimum cutoff frequency can be estimated using

    Fc = (1.4845 + 0.1523 Fs^1/2)^2

    This equation explained over 75% of the total variation in the optimum cutoff
    frequency. This equation has been used in the last five years for different 2d
    and 3d coordinate data in different human body motions, and the results are
    satisfactory. The explanation for this relationship is that the higher the
    sampling frequency, the high the frequency of the random error (the further
    the random error components will go to the high frequency end of the frequency
    spectrum, see Dr. Winter's book, Biomechanics of human movement.). You may try
    this equation if you think it makes sense or its smoothing results make sense.

    I also have another equation for determination of optimum cutoff frequency,
    which requires FFT and freqeuncy analysis. It explained over 85% of the total
    variation of the optimum cutoff frequency. However, I found that sometimes this
    equation works pretty well, sometimes doesn't. If you are interested in, I can
    give you all the details.

    Several years ago, I tried to get the study published in Journal of
    Biomechanics as a technical note. One of the reviewer attacked me saying that
    this study had no contribution to the biomechanics. But the how to determine
    the optimum cutoff frequency has been frequently asked by many researchers in
    biomechanics in the last several years. It may be the time for me to
    re-consider publishing this study.

    Bing Yu, Ph.D.
    Orthopedic Biomechanics Laboratory
    Mayo Clinic
    Rochester, MN 55905
    ================================================== =======
    Date: Mon, 21 Nov 1994 16:19:15 -0600
    X-Ph: V4.1@genesis
    From: Duane Knudson
    Subject: RE>Optimal cutoff frequency for data smoothing
    To: Robert Newton

    Greetings Robert!

    I bet you get a large number of responses to this post since data smoothing
    has been a persistent problem in our field. I have RMS residuals for many
    kinds of kinematic data and get curves very similar to Winter's 2.25 on page
    43. The curves tend to bottom out at the measurement error for the situation.
    I do not suspect the mean square error would be any different.

    The problem is that even the "automated" smoothing programs essentially are
    still arbitrary selections (note that Winter p. 42 says " If we decide both (
    signal distortion and noise passed) should be equal . . ." The other
    arbitrary "automated" method is the Jackson (1979) method that takes the
    second derivative of the linear interpolation of the residuals. It may be a
    chicken/egg situation where we cannot objectively separate the signal and
    noise of our kinematic data. Even fourier analysis, ultimately must be based
    on some guess (95% signal power?) as a good compromise of signal distortion
    and nois attenuation.

    We need more accelerometer studies and a common standard of what is acceptable
    signal to noise ratio, or what are appropriate frequencies for specific kinds
    of biomechanical data. Good luck in your quest.

    Jackson, K.M. (1979) Fitting of mathematical functions to biomechanical data.
    IEEE Trans Biomed Eng. 26:122-124.

    ================================================== ====

    Just out of curiuosity, what is your sampling rate? Since you are smoothing
    up to 60 Hz, it almost sounds as if you are violating the Nyquist limit of
    the Butterworth digital filter (see J. Walton's dissertation). Once beyond
    (0.25 * SAMPLING_RATE), the Butterworth digital filter behaves strangely.
    Could this be the problem? If, for example, you are collecting at 100 Hz,
    try using your algorithm in the cutoff range of 1-25 and see if that
    eliminates the strange sections of the residual curve. Good luck -- let me
    know what happens.

    Peter Vint
    Arizona State University
    Exercise and Sport Research Institute

    (Note sampling frequency was 500 Hz - Robert Newton)

    ================================================== ==
    Date: Tue, 22 Nov 94 11:24:08 EST
    X-Ph: V4.1@genesis
    Subject: optimal filtering

    Hi Rob

    You might try the Jackson 'knee' method:

    Jackson, KM (1979) Fitting of mathematical functions to biomechanical data.
    IEEE Trans. Biomed. Eng., vol ?:122-124.

    I haven't got Winter in front of me, and I can't remember the specifics of
    the method he suggests. It may even be the Jackson method, in which case I
    haven't helped you much !

    The Jackson method is now used by the Peak system for optimal filtering by
    Butterworth, cubic spline, or fourier series. It seems to work well for
    kinematic data, but I haven't tried it for anything more complex.

    Good luck !


    ================================================== ===
    X-Ph: V4.1@genesis
    From: "Alan Walmsley"
    Organization: School of Physical Education, Otago
    Date: Tue, 22 Nov 1994 15:03:17 GMT+1200
    Subject: Re: Optimal cutoff frequency for data smoothing
    Priority: normal

    Dear Robert,

    Have you considered spectral analysis to obtain the major frequency
    components, and then choosing a cut-off frequency at least an octave
    above the major peak?
    Alan Walmsley
    School of Physical Education
    Division of Sciences
    University of Otago
    Dunedin, New Zealand.
    Ph (03) 4799122, Fax (03) 4798309
    X-Ph: V4.1@genesis
    From: Rob Neal
    Date: Tue, 22 Nov 1994 12:41:11 EST5EDT
    Subject: Re: Optimal cutoff frequency for data smoothing
    Priority: normal

    I don't have the references but the problem seems very similar to the
    one exercise physiologists have for determining ventilatory threshold
    or anaerobic threshold. There are a few papers detailing various
    methods to solve this problem. I could try to find them from the guys
    at the QAS if you would like.



    Robert Neal, PhD
    Department of Human Movement Studies
    The University of Queensland
    ph 61 7 365 6240
    FAX 61 7 365 6877
    Date: Tue, 22 Nov 1994 09:33:26 -0500 (EST)
    X-Ph: V4.1@genesis
    From: stuart mcgill
    Subject: Re: Optimal cutoff frequency for data smoothing
    To: Robert Newton

    Hello Robert,

    "Residual analysis" as described in Winter assumes that the noise
    component is white- yours appears not to be. Perhaps you should attempt
    another method- you didn't describe the signal that must be smoothed-
    this would help in choosing another way to smooth. Good luck.
    Stu McGill
    Date: Tue, 22 Nov 1994 12:45:32 MET-DST
    X-Ph: V4.1@genesis
    From: "Giovanni LEGNANI. Uni. of Brescia, Italy EC"
    Subject: Re: Optimal cutoff frequency for data smoothing
    X-Vms-To: IN%"run1@PSU.EDU"


    The frequency should be proportional to the frequency of the incoming pulses
    coming from the encoders. (you are forced to choose the maximum speed).

    then you have to choose a frequency that is lower than the half of the
    incoming signal of angle to avoid fenomena similar to aliasing.

    so if you have an encoder haning 1000 steps, you will have 4000 samples
    per turn.
    if your encoder rotates ad a speed of K turns per second you have a data
    frequency of 4000 Hz. I suggest you to filter chosing a low-pass filter
    having a bandwith lower than 2000 Hz.
    Better a little lower.

    take in mind that an encoder give an approximate value for the angle.
    the absolute error is 1 step. when the encoder rotates you have a noise
    having an amplitude of 1 step and a frequency proportional to the
    encoder speed and to the number of the encoder steps.


    giovanni legnani
    X-Ph: V4.1@genesis
    From: "Tom Lundin"
    Date: Tue, 22 Nov 94 11:04:32 EDT
    X-Popmail-Charset: English
    Subject: cutoff frequencies


    I have recently encountered a similar problem with filtering motion data.
    The best algorithm I could come up with to select a cutoff was to
    differentiate the RMS error vs. cutoff frequency twice and search for where
    the ensuing curve approximated zero (point Z). I found the slope of the
    line described by Winter from the first derivative of the RMS curve at Z.
    Then using the equation of that line I found the cutoff frequency as I
    presume you already know how to do. If you have any questions or comments
    please feel free to write back. I hope this helps and I'll be interested
    to see the other responses you receive.

    Tom Lundin
    The Cleveland Clinic Foundation
    ================================================== ==
    X-Ph: V4.1@genesis
    From: Paul Guy
    Subject: Re: Optimal cutoff frequency for data smoothing
    Date: Tue, 22 Nov 1994 12:30:55 -0500 (EST)
    Content-Length: 4111

    Having worked with Dave Winter for many years in his lab, I'll give
    you an answer you might not want to hear.
    In short, there is no decent mathematical method that I've seen based
    on conventional or residual analysis that covers all the situations. If
    you are interested in say just the displacements, then a residual
    analysis of them will probably do, if you wanted to see what gave the
    best results in a power or kinetics situation, then you'd need to do
    residuals based on those variables.
    The best way to deal with it, is to have some previous knowledge of
    the system you are measuring, what its dynamic characteristics are, and
    what the behaviour is of the data once it arrives at your computer. Such
    things as whether you used interlaced video would be very important
    (large 30 hz noise components), or where the resonant frequencies were
    on your force plates, transducers etc.
    For filtering data from the human body, we will filter different
    segments at different frequencies, for example the trunk markers at 1-3
    Hz, the foot at 8 to 15 Hz depending on the activity.
    Where the accelerations become very important, we find that it's often
    worthwhile to raise the sampling frequency, especially if you are doing
    stuff like FFT's (and you need long records too). The ratio of cutoff
    frequency to sampling frequency will affect whether you are really
    getting an analog equivalent. Filtering at 1/4 the sampling frequency
    will not give you the characteristics that you might expect.
    Another issue is filter TYPE.... are you using Butterworth, Bessel,
    IIR,FIR, 2-way pass etc. ? All these become an issue depending on what
    you're looking at, in what domain, and how your applications are going
    to react to the various 'corruptions'. For example, we use a so-called
    4th order 2-way Butterworth (it's run through two 2nd order
    Butterworths, the second is filtered backwards in time, to reduce delay
    artifacts). Using this filter with force plates causes a force to appear
    on the plate before the foot contacts it. That's clearly silly data.
    Similiarly, the horizontal impulse the foot gives at heel contact can
    really mess up the determination of the body kinetics.....filtering it
    can spread what occurs in 10-50ms over a much longer time, rendering
    your analysis useless near heel contact.
    Don't buy all that theory stuff, sometimes a look at the big picture
    will help more.


    Paul J Guy work phone:519-885-1211 ext 6371 home/FAX/:519-576-3090 64 Mt.Hope St.,Kitchener,Ontario,Canada

    ================================================== =
    Date: Tue, 22 Nov 1994 11:45:26 -0600 (CST)
    X-Ph: V4.1@genesis
    From: "Christine Q. Wu"
    To: Robert Newton
    Subject: Re: Optimal cutoff frequency for data smoothing

    Dear Robert: I have the similar problem. Would you please let me know if
    you get any solution. Besides, is the range of motion effects the smooth
    procedure? If the range of motion is low, may be comparable with the
    magnitude of the noise, what will happen?
    Good luck!
    ================================================== =
    Date: 22 Nov 94 16:39:49 EST
    X-Ph: V4.1@genesis
    From: "Peak Performance Tech."
    To: Robert Newton
    Subject: Re: Optimal cutoff frequency for data smoothing


    If you haven't already, you may want to check out the "Jackson Knee
    Method". It plots the 2nd derivative of the percent average residual
    curve vs. the cutoff frequency. When three points on that curve fall
    below a defined prescribed limit, the smallest frequency of the curve
    becomes the optimal.

    Jackson, K.M. Fitting of mathematical functions to biomechanical data.
    IEEE Transactions on Biomedical Engineering, 1979, pp. 122-124.

    George Miller
    Peak Performance Technologies
    Englewood, CO
    Date: Tue, 22 Nov 1994 21:58:37 -0600 (CST)
    X-Ph: V4.1@genesis
    From: "M. Pizzimenti"
    Subject: Re: Optimal cutoff frequency for data smoothing
    To: Robert Newton


    Try Jackson's algorithm where the residuals are differentiated.

    Jackson, K.M. Fitting of mathematical functions to biomechanical data.
    IEEE Transactions of Biomedical Engineering BME26(2), 122-124, 1979

    Hope this helps

    Marc Pizzimenti
    University of Iowa
    Department of Exercise Science

    ====================== END OF REPLIES =================

    Robert Newton Phone Int+ 1 814
    865 7107
    Center for Sports Medicine Fax Int+ 1 814 865 7077
    The Pennsylvania State University Email RUN1@PSU.EDU
    117 Ann Building
    University Park, PA 16802
    United States of America