Tweet
*** THE QUESTION ***
Hi everyone,
This is general question relevant to a lot of our research. It should
be simple, but I don't recall the answer.
How many digits should be used in reporting data?
Here is an example. A biomechanist is trying to show a difference
between Technique A and Technique B. He/she recruits 36 subjects and
records each one of them performing both techniques, using equipment
that measures each person to the nearest 0.1 unit. How many decimal
places should be used in reporting the data? I can make a "common
sense" determination, but is there some accepted procedure or guideline?
Example data:
TECHNIQUE A
Subject 1: 13.3 units
Subject 2: 35.0 units
Subject 3: 22.2 units
Subject 36: 18.4 units
TECHNIQUE B
Subject 1: 12.9 units
Subject 2: 35.0 units
Subject 3: 21.9 units
Subject 36: 18.5 units
Even though the mean within-subject difference is less than 0.1 units, a
paired t-test reveals a statistically significant difference.
Technique A mean: 25.33333 units
Technique B mean: 25.36170 units
Average difference : 0.02837 units
Paired t-test p-value: 0.035
How many decimal places would you use in reporting these mean values and
difference in your paper? Why?
Thanks in advance,
- Glenn S. Fleisig, Ph.D.
Glenn S. Fleisig, Ph.D., Smith & Nephew Chair of Research
American Sports Medicine Institute
833 St. Vincent's Drive, Suite 100
Birmingham, AL 35205
(email) glennf@asmi.org
(tel) 205-918-2139
www.asmi.org
_____
*** THE ANSWER ***
Thank you to all who responded. Based on the responses it is clearly a
community that is bright and polite. :-)
There were many well-written, insightful responses, which I've posted
below. From these responses, my own experience, and conversations with
others (biostatisticians, etc.), I've concluded that there is no hard
scientific rule for significant digits. Therefore, I am please to share
my conclusions from all of these thoughts, named (with tongue-in-cheek):
Fleisig's Four Steps for Significant Digits
1. In general, do not present individual values, mean values,
standard deviations, or other calculated values in smaller units than
the accuracy of your equipment.
2. If you find a statistical significance with a magnitude less
than the accuracy of the measurement, then question whether strict
enough statistics were used. For example, if you find a small
difference to be significant with a t-test, then perhaps the alpha-level
was set too high.
3. If you decide the statistical test was appropriate, then report
the values with the added decimal place.
4. If you report such a statistical difference, then explain
whether or not you believe the difference has practical significance.
In most cases, the researcher will decide the small difference has no
practical significance. If you feel the difference is important, then
you probably should have used more accurate equipment.
_____
*** THE RESPONSES ***
_____
From: Michael Cinelli [mailto:Michael_Cinelli@brown.edu]
Sent: Tuesday, May 22, 2007 7:25 AM
To: Glenn Fleisig
Subject: Re: [BIOMCH-L] Significant digits - a Quiz for our community
Hi Glenn,
You can only use the number of decimal places that is equivalent to the
precision of your instrumentation.
Michael Cinelli
Postdoctoral Fellow
Dept. of Cognitive and Linguistic Sciences
Brown University
190 Thayer St. Rm. 102F
Providence, RI
02912
Michael_Cinelli@brown.edu
Every body continues in its state of rest, or of uniform motion in a
right line, unless it is compelled to change that state by forces
impressed upon it - Sir Isaac Newton
_____
From: Chris Kirtley [mailto:ckirtley@gmail.com]
Sent: Tuesday, May 22, 2007 7:32 AM
To: Glenn Fleisig
Cc: BIOMCH-L@nic.surfnet.nl
Subject: Re: [BIOMCH-L] Significant digits - a Quiz for our community
Dear Glenn,
That's a great example - wish I'd had it to hand when I used to nag my
students about this.
The key point to understand is of course the accuracy of the measurement
- just because there is a readout with 3 significant figures doesn't
mean that the measurement is accurate to 1 part in a thousand.
The t-test also illustrates the difference between statistical
signifance and practical significance. Clearly the measured difference
is practically insignificant even if it is statistically significant
(even assuming that the t-test was appropriate, which it may not be). It
would be interesting to recalculate with the input data rounded off to
integers, though even then it really is the practical signifiacnce of
the difference that's most important.
BTW, there's nothing special about decimal places. We should also round
off integers according to the measurement accuracy. For example, it
makes no sense to record blood pressure to the nearest mmHg - it should
be rounded to the nearest 5 mmHg at least (10 is probably more
realistic).
Chris
_____
-----Original Message-----
From: garyalan59@peoplepc.com [mailto:garyalan59@peoplepc.com]
Sent: Tuesday, May 22, 2007 7:45 AM
To: Glenn Fleisig
Subject: Re: [BIOMCH-L] Significant digits - a Quiz for our community
In your example, the program calculates a difference, but that
difference is
smaller than your ability to differentiate (measure). The general rule
(from
my chemistry background) is that you cannot report values that are more
"accurate" than your original measure, because the reporting implies you
can
actually measure what you are reporting to the degree you are reporting.
Additionally, just because something is statistically significant does
not
mean it is clinically significant.
Just my two cents.
Gary Christopher
Doctoral Candidate
Dept. of Kinesiology
Texas Woman's University
_____
-----Original Message-----
From: Sian Lawson [mailto:sian.lawson@newcastle.ac.uk]
Sent: Tuesday, May 22, 2007 8:11 AM
To: Glenn Fleisig; BIOMCH-L@NIC.SURFNET.NL
Subject: RE: [BIOMCH-L] Significant digits - a Quiz for our community
Glenn,
This is a good illustration of the disparity between a statistically
significant difference and a functionally (or clinically) significant
difference. Your t-test may show a statistical difference in the
populations, but the difference is still negligibly small.
Statistics, even means, cannot create accuracy. If the accuracy of your
recording is to one decimal place, then your means etc will not be any
more accurate than that.
Best regards,
Sian
Dr. Sian E. M. Lawson
Centre for Rehabilitation and Engineering Studies (CREST)
School of Mechanical and Systems Engineering
University of Newcastle upon Tyne, U.K.
CREST: +44 (0)191 222 6170
Direct: +44 (0)191 222 8224
Fax: +44 (0)191 222 8600
_____
-----Original Message-----
From: borgbros@netti.fi [mailto:borgbros@netti.fi]
Sent: Tuesday, May 22, 2007 8:16 AM
To: Glenn Fleisig
Subject: Re: [BIOMCH-L] Significant digits - a Quiz for our community
Pax!
A quick answer is that if one is investigating the mean of x_1, ..., x_N
then the *resolution* improves as stdev{x}/sqrt{N} when the number of
data N increases. Thus, reporting mean value, enough decimals should be
included so as to resolve the group standard deviation.
Regards Frank Borg
skrev
_____
-----Original Message-----
From: Torrence Welch [mailto:twelch@neuro.gatech.edu]
Sent: Tuesday, May 22, 2007 8:50 AM
To: Glenn Fleisig
Subject: Re: [BIOMCH-L] Significant digits - a Quiz for our community
Hi Glenn,
You should report values with the same number of significant digits as
the
least precise measurement in your study. Since the equipment in your
example reported values with 3 sig figs, your average measurements
should
be reported as:
> Technique A mean: 25.3 units
>
> Technique B mean: 25.4 units
>
> Average difference : 0.0284 units
>
> Paired t-test p-value: 0.0350
Even if some of your measurements were over 100 units, you would still
report 3 sig figs. You would move to 4 sig figs only if all measurements
surpassed 100 units and you maintained 0.1 precision on your equipment.
Happy researching,
--
Torrence D.J. Welch, MSE
Graduate Research Assistant / PhD Candidate
Laboratory for Neuroengineering
Wallace H. Coulter Department of Biomedical Engineering
Georgia Institute of Technology / Emory University
313 Ferst Drive
Atlanta, GA 30332-0535
(404) 385-4340 [lab]
_____
From: D. Gordon E. Robertson, PhD [mailto:dger@uOttawa.ca]
Sent: Tuesday, May 22, 2007 9:29 AM
To: Glenn Fleisig
Subject: Re: [BIOMCH-L] Significant digits - a Quiz for our community
For biomechanics courses and elsewhere I use a method originally
outlined
in an engineering mechanics textbook by Beer & Johnston. It is
ultimately based
on the accuracy of slide rules but it works quite well in biomechanics
where we
can't get highly accurate measures of most body segment parameters.
The concept is to make numbers accurate to 0.5%. That means all numbers
are rounded to three significant figures unless the first number is a
"1" in which
case use four. Students like it because it is definitive and easy to
remember.
Note that for a final answer to be accurate to 0.5% you must carry an
extra digit
in intermediate results.
Cheers
_____
-----Original Message-----
From: David Stodden [mailto:dfstodd@bgnet.bgsu.edu]
Sent: Tuesday, May 22, 2007 10:31 AM
To: Glenn Fleisig
Subject: Re: [BIOMCH-L] Significant digits - a Quiz for our community
Greetings Glenn,
I seem to remember from some distant measurement or statistics class
that
you report one decimal place more than your measurement device. Thus, if
you are measuring degrees (e.g., 0-360 degrees) then you provide your
statistic to the tenths of degrees - 90.8. If your calculations provide
you
with an extra 3 or 4 decimal places, you simply round to the nearest
decimal that is one decimal from your designated measurement. Thus,
90.842222 degrees would still be reported as 90.8 degrees. Ultimately, I
believe it depends not only what you are measuring and the specific
instrumentation device, but also what type of sensitivity you demand.
That
is my best recollection.
I agree with the practical significance information that was presented
by
the other two respondents, but the practical significance is related to
the
effect size statistics, which are based on your power estimates and
previous literature effect sizes, that should be reported in your
manuscripts.
Dave
_____
-----Original Message-----
From: David Smith [mailto:david.smithpodiatrist@tesco.net]
Sent: Tuesday, May 22, 2007 12:45 PM
To: Glenn Fleisig; BIOMCH-L@NIC.SURFNET.NL
Subject: Re: [BIOMCH-L] Significant digits - a Quiz for our community
Dear Glenn
I studied statistics for a while but I still don't really understand
them.
For me they are a tautology in that they use their own rules to prove
themselves. If you use different rules to prove something different its
not
valid because you didn't use the rules of statistics.
So if your two different groups had intervention =A and no intervention
=B
and A and B averages had no significant difference but A had a highly
variable sequence of scores and B was fairly even across the board.
Surley
this might intuitively indicate that there was a significant clinical
difference in the out comes, which is what we are interested in, isn't
it?
Also what if the scores recorded had an exponential relationship with
the
baseline value how would that effect the statistical significance. Or if
you
where measuring mass for instance then 0.01grams in a range of 0-2grams
may
be significant but in a range of 1000-10,000 grams it may not be
What if your sensor has poor linearity or your sample rate was to low
etc
etc ie if your recording is to several decimal places then you must be
sure
that the input data is reliable and accurate enough to support that
standard.
At the end of the day is it not down to the reader to decide if the
research
and statistics are suitable for the conclusions that they may want to
take
from a set of results.
Cheers Dave
_____
-----Original Message-----
From: David Smith [mailto:david.smithpodiatrist@tesco.net]
Sent: Tuesday, May 22, 2007 1:12 PM
To: Glenn Fleisig; BIOMCH-L@NIC.SURFNET.NL
Subject: Re: [BIOMCH-L] Significant digits - a Quiz for our community
Hello again Glenn
Just another point
The P value is the indication of the confidence in rejecting the null
hypothesis.
If your P value is 0.035 (alpha =
Hi everyone,
This is general question relevant to a lot of our research. It should
be simple, but I don't recall the answer.
How many digits should be used in reporting data?
Here is an example. A biomechanist is trying to show a difference
between Technique A and Technique B. He/she recruits 36 subjects and
records each one of them performing both techniques, using equipment
that measures each person to the nearest 0.1 unit. How many decimal
places should be used in reporting the data? I can make a "common
sense" determination, but is there some accepted procedure or guideline?
Example data:
TECHNIQUE A
Subject 1: 13.3 units
Subject 2: 35.0 units
Subject 3: 22.2 units
Subject 36: 18.4 units
TECHNIQUE B
Subject 1: 12.9 units
Subject 2: 35.0 units
Subject 3: 21.9 units
Subject 36: 18.5 units
Even though the mean within-subject difference is less than 0.1 units, a
paired t-test reveals a statistically significant difference.
Technique A mean: 25.33333 units
Technique B mean: 25.36170 units
Average difference : 0.02837 units
Paired t-test p-value: 0.035
How many decimal places would you use in reporting these mean values and
difference in your paper? Why?
Thanks in advance,
- Glenn S. Fleisig, Ph.D.
Glenn S. Fleisig, Ph.D., Smith & Nephew Chair of Research
American Sports Medicine Institute
833 St. Vincent's Drive, Suite 100
Birmingham, AL 35205
(email) glennf@asmi.org
(tel) 205-918-2139
www.asmi.org
_____
*** THE ANSWER ***
Thank you to all who responded. Based on the responses it is clearly a
community that is bright and polite. :-)
There were many well-written, insightful responses, which I've posted
below. From these responses, my own experience, and conversations with
others (biostatisticians, etc.), I've concluded that there is no hard
scientific rule for significant digits. Therefore, I am please to share
my conclusions from all of these thoughts, named (with tongue-in-cheek):
Fleisig's Four Steps for Significant Digits
1. In general, do not present individual values, mean values,
standard deviations, or other calculated values in smaller units than
the accuracy of your equipment.
2. If you find a statistical significance with a magnitude less
than the accuracy of the measurement, then question whether strict
enough statistics were used. For example, if you find a small
difference to be significant with a t-test, then perhaps the alpha-level
was set too high.
3. If you decide the statistical test was appropriate, then report
the values with the added decimal place.
4. If you report such a statistical difference, then explain
whether or not you believe the difference has practical significance.
In most cases, the researcher will decide the small difference has no
practical significance. If you feel the difference is important, then
you probably should have used more accurate equipment.
_____
*** THE RESPONSES ***
_____
From: Michael Cinelli [mailto:Michael_Cinelli@brown.edu]
Sent: Tuesday, May 22, 2007 7:25 AM
To: Glenn Fleisig
Subject: Re: [BIOMCH-L] Significant digits - a Quiz for our community
Hi Glenn,
You can only use the number of decimal places that is equivalent to the
precision of your instrumentation.
Michael Cinelli
Postdoctoral Fellow
Dept. of Cognitive and Linguistic Sciences
Brown University
190 Thayer St. Rm. 102F
Providence, RI
02912
Michael_Cinelli@brown.edu
Every body continues in its state of rest, or of uniform motion in a
right line, unless it is compelled to change that state by forces
impressed upon it - Sir Isaac Newton
_____
From: Chris Kirtley [mailto:ckirtley@gmail.com]
Sent: Tuesday, May 22, 2007 7:32 AM
To: Glenn Fleisig
Cc: BIOMCH-L@nic.surfnet.nl
Subject: Re: [BIOMCH-L] Significant digits - a Quiz for our community
Dear Glenn,
That's a great example - wish I'd had it to hand when I used to nag my
students about this.
The key point to understand is of course the accuracy of the measurement
- just because there is a readout with 3 significant figures doesn't
mean that the measurement is accurate to 1 part in a thousand.
The t-test also illustrates the difference between statistical
signifance and practical significance. Clearly the measured difference
is practically insignificant even if it is statistically significant
(even assuming that the t-test was appropriate, which it may not be). It
would be interesting to recalculate with the input data rounded off to
integers, though even then it really is the practical signifiacnce of
the difference that's most important.
BTW, there's nothing special about decimal places. We should also round
off integers according to the measurement accuracy. For example, it
makes no sense to record blood pressure to the nearest mmHg - it should
be rounded to the nearest 5 mmHg at least (10 is probably more
realistic).
Chris
_____
-----Original Message-----
From: garyalan59@peoplepc.com [mailto:garyalan59@peoplepc.com]
Sent: Tuesday, May 22, 2007 7:45 AM
To: Glenn Fleisig
Subject: Re: [BIOMCH-L] Significant digits - a Quiz for our community
In your example, the program calculates a difference, but that
difference is
smaller than your ability to differentiate (measure). The general rule
(from
my chemistry background) is that you cannot report values that are more
"accurate" than your original measure, because the reporting implies you
can
actually measure what you are reporting to the degree you are reporting.
Additionally, just because something is statistically significant does
not
mean it is clinically significant.
Just my two cents.
Gary Christopher
Doctoral Candidate
Dept. of Kinesiology
Texas Woman's University
_____
-----Original Message-----
From: Sian Lawson [mailto:sian.lawson@newcastle.ac.uk]
Sent: Tuesday, May 22, 2007 8:11 AM
To: Glenn Fleisig; BIOMCH-L@NIC.SURFNET.NL
Subject: RE: [BIOMCH-L] Significant digits - a Quiz for our community
Glenn,
This is a good illustration of the disparity between a statistically
significant difference and a functionally (or clinically) significant
difference. Your t-test may show a statistical difference in the
populations, but the difference is still negligibly small.
Statistics, even means, cannot create accuracy. If the accuracy of your
recording is to one decimal place, then your means etc will not be any
more accurate than that.
Best regards,
Sian
Dr. Sian E. M. Lawson
Centre for Rehabilitation and Engineering Studies (CREST)
School of Mechanical and Systems Engineering
University of Newcastle upon Tyne, U.K.
CREST: +44 (0)191 222 6170
Direct: +44 (0)191 222 8224
Fax: +44 (0)191 222 8600
_____
-----Original Message-----
From: borgbros@netti.fi [mailto:borgbros@netti.fi]
Sent: Tuesday, May 22, 2007 8:16 AM
To: Glenn Fleisig
Subject: Re: [BIOMCH-L] Significant digits - a Quiz for our community
Pax!
A quick answer is that if one is investigating the mean of x_1, ..., x_N
then the *resolution* improves as stdev{x}/sqrt{N} when the number of
data N increases. Thus, reporting mean value, enough decimals should be
included so as to resolve the group standard deviation.
Regards Frank Borg
skrev
_____
-----Original Message-----
From: Torrence Welch [mailto:twelch@neuro.gatech.edu]
Sent: Tuesday, May 22, 2007 8:50 AM
To: Glenn Fleisig
Subject: Re: [BIOMCH-L] Significant digits - a Quiz for our community
Hi Glenn,
You should report values with the same number of significant digits as
the
least precise measurement in your study. Since the equipment in your
example reported values with 3 sig figs, your average measurements
should
be reported as:
> Technique A mean: 25.3 units
>
> Technique B mean: 25.4 units
>
> Average difference : 0.0284 units
>
> Paired t-test p-value: 0.0350
Even if some of your measurements were over 100 units, you would still
report 3 sig figs. You would move to 4 sig figs only if all measurements
surpassed 100 units and you maintained 0.1 precision on your equipment.
Happy researching,
--
Torrence D.J. Welch, MSE
Graduate Research Assistant / PhD Candidate
Laboratory for Neuroengineering
Wallace H. Coulter Department of Biomedical Engineering
Georgia Institute of Technology / Emory University
313 Ferst Drive
Atlanta, GA 30332-0535
(404) 385-4340 [lab]
_____
From: D. Gordon E. Robertson, PhD [mailto:dger@uOttawa.ca]
Sent: Tuesday, May 22, 2007 9:29 AM
To: Glenn Fleisig
Subject: Re: [BIOMCH-L] Significant digits - a Quiz for our community
For biomechanics courses and elsewhere I use a method originally
outlined
in an engineering mechanics textbook by Beer & Johnston. It is
ultimately based
on the accuracy of slide rules but it works quite well in biomechanics
where we
can't get highly accurate measures of most body segment parameters.
The concept is to make numbers accurate to 0.5%. That means all numbers
are rounded to three significant figures unless the first number is a
"1" in which
case use four. Students like it because it is definitive and easy to
remember.
Note that for a final answer to be accurate to 0.5% you must carry an
extra digit
in intermediate results.
Cheers
_____
-----Original Message-----
From: David Stodden [mailto:dfstodd@bgnet.bgsu.edu]
Sent: Tuesday, May 22, 2007 10:31 AM
To: Glenn Fleisig
Subject: Re: [BIOMCH-L] Significant digits - a Quiz for our community
Greetings Glenn,
I seem to remember from some distant measurement or statistics class
that
you report one decimal place more than your measurement device. Thus, if
you are measuring degrees (e.g., 0-360 degrees) then you provide your
statistic to the tenths of degrees - 90.8. If your calculations provide
you
with an extra 3 or 4 decimal places, you simply round to the nearest
decimal that is one decimal from your designated measurement. Thus,
90.842222 degrees would still be reported as 90.8 degrees. Ultimately, I
believe it depends not only what you are measuring and the specific
instrumentation device, but also what type of sensitivity you demand.
That
is my best recollection.
I agree with the practical significance information that was presented
by
the other two respondents, but the practical significance is related to
the
effect size statistics, which are based on your power estimates and
previous literature effect sizes, that should be reported in your
manuscripts.
Dave
_____
-----Original Message-----
From: David Smith [mailto:david.smithpodiatrist@tesco.net]
Sent: Tuesday, May 22, 2007 12:45 PM
To: Glenn Fleisig; BIOMCH-L@NIC.SURFNET.NL
Subject: Re: [BIOMCH-L] Significant digits - a Quiz for our community
Dear Glenn
I studied statistics for a while but I still don't really understand
them.
For me they are a tautology in that they use their own rules to prove
themselves. If you use different rules to prove something different its
not
valid because you didn't use the rules of statistics.
So if your two different groups had intervention =A and no intervention
=B
and A and B averages had no significant difference but A had a highly
variable sequence of scores and B was fairly even across the board.
Surley
this might intuitively indicate that there was a significant clinical
difference in the out comes, which is what we are interested in, isn't
it?
Also what if the scores recorded had an exponential relationship with
the
baseline value how would that effect the statistical significance. Or if
you
where measuring mass for instance then 0.01grams in a range of 0-2grams
may
be significant but in a range of 1000-10,000 grams it may not be
What if your sensor has poor linearity or your sample rate was to low
etc
etc ie if your recording is to several decimal places then you must be
sure
that the input data is reliable and accurate enough to support that
standard.
At the end of the day is it not down to the reader to decide if the
research
and statistics are suitable for the conclusions that they may want to
take
from a set of results.
Cheers Dave
_____
-----Original Message-----
From: David Smith [mailto:david.smithpodiatrist@tesco.net]
Sent: Tuesday, May 22, 2007 1:12 PM
To: Glenn Fleisig; BIOMCH-L@NIC.SURFNET.NL
Subject: Re: [BIOMCH-L] Significant digits - a Quiz for our community
Hello again Glenn
Just another point
The P value is the indication of the confidence in rejecting the null
hypothesis.
If your P value is 0.035 (alpha =