gfleisig24

05-23-2007, 07:59 AM

*** THE QUESTION ***

Hi everyone,

This is general question relevant to a lot of our research. It should

be simple, but I don't recall the answer.

How many digits should be used in reporting data?

Here is an example. A biomechanist is trying to show a difference

between Technique A and Technique B. He/she recruits 36 subjects and

records each one of them performing both techniques, using equipment

that measures each person to the nearest 0.1 unit. How many decimal

places should be used in reporting the data? I can make a "common

sense" determination, but is there some accepted procedure or guideline?

Example data:

TECHNIQUE A

Subject 1: 13.3 units

Subject 2: 35.0 units

Subject 3: 22.2 units

Subject 36: 18.4 units

TECHNIQUE B

Subject 1: 12.9 units

Subject 2: 35.0 units

Subject 3: 21.9 units

Subject 36: 18.5 units

Even though the mean within-subject difference is less than 0.1 units, a

paired t-test reveals a statistically significant difference.

Technique A mean: 25.33333 units

Technique B mean: 25.36170 units

Average difference : 0.02837 units

Paired t-test p-value: 0.035

How many decimal places would you use in reporting these mean values and

difference in your paper? Why?

Thanks in advance,

- Glenn S. Fleisig, Ph.D.

Glenn S. Fleisig, Ph.D., Smith & Nephew Chair of Research

American Sports Medicine Institute

833 St. Vincent's Drive, Suite 100

Birmingham, AL 35205

(email) glennf@asmi.org

(tel) 205-918-2139

www.asmi.org

_____

*** THE ANSWER ***

Thank you to all who responded. Based on the responses it is clearly a

community that is bright and polite. :-)

There were many well-written, insightful responses, which I've posted

below. From these responses, my own experience, and conversations with

others (biostatisticians, etc.), I've concluded that there is no hard

scientific rule for significant digits. Therefore, I am please to share

my conclusions from all of these thoughts, named (with tongue-in-cheek):

Fleisig's Four Steps for Significant Digits

1. In general, do not present individual values, mean values,

standard deviations, or other calculated values in smaller units than

the accuracy of your equipment.

2. If you find a statistical significance with a magnitude less

than the accuracy of the measurement, then question whether strict

enough statistics were used. For example, if you find a small

difference to be significant with a t-test, then perhaps the alpha-level

was set too high.

3. If you decide the statistical test was appropriate, then report

the values with the added decimal place.

4. If you report such a statistical difference, then explain

whether or not you believe the difference has practical significance.

In most cases, the researcher will decide the small difference has no

practical significance. If you feel the difference is important, then

you probably should have used more accurate equipment.

_____

*** THE RESPONSES ***

_____

From: Michael Cinelli [mailto:Michael_Cinelli@brown.edu]

Sent: Tuesday, May 22, 2007 7:25 AM

To: Glenn Fleisig

Subject: Re: [BIOMCH-L] Significant digits - a Quiz for our community

Hi Glenn,

You can only use the number of decimal places that is equivalent to the

precision of your instrumentation.

Michael Cinelli

Postdoctoral Fellow

Dept. of Cognitive and Linguistic Sciences

Brown University

190 Thayer St. Rm. 102F

Providence, RI

02912

Michael_Cinelli@brown.edu

Every body continues in its state of rest, or of uniform motion in a

right line, unless it is compelled to change that state by forces

impressed upon it - Sir Isaac Newton

_____

From: Chris Kirtley [mailto:ckirtley@gmail.com]

Sent: Tuesday, May 22, 2007 7:32 AM

To: Glenn Fleisig

Cc: BIOMCH-L@nic.surfnet.nl

Subject: Re: [BIOMCH-L] Significant digits - a Quiz for our community

Dear Glenn,

That's a great example - wish I'd had it to hand when I used to nag my

students about this.

The key point to understand is of course the accuracy of the measurement

- just because there is a readout with 3 significant figures doesn't

mean that the measurement is accurate to 1 part in a thousand.

The t-test also illustrates the difference between statistical

signifance and practical significance. Clearly the measured difference

is practically insignificant even if it is statistically significant

(even assuming that the t-test was appropriate, which it may not be). It

would be interesting to recalculate with the input data rounded off to

integers, though even then it really is the practical signifiacnce of

the difference that's most important.

BTW, there's nothing special about decimal places. We should also round

off integers according to the measurement accuracy. For example, it

makes no sense to record blood pressure to the nearest mmHg - it should

be rounded to the nearest 5 mmHg at least (10 is probably more

realistic).

Chris

_____

-----Original Message-----

From: garyalan59@peoplepc.com [mailto:garyalan59@peoplepc.com]

Sent: Tuesday, May 22, 2007 7:45 AM

To: Glenn Fleisig

Subject: Re: [BIOMCH-L] Significant digits - a Quiz for our community

In your example, the program calculates a difference, but that

difference is

smaller than your ability to differentiate (measure). The general rule

(from

my chemistry background) is that you cannot report values that are more

"accurate" than your original measure, because the reporting implies you

can

actually measure what you are reporting to the degree you are reporting.

Additionally, just because something is statistically significant does

not

mean it is clinically significant.

Just my two cents.

Gary Christopher

Doctoral Candidate

Dept. of Kinesiology

Texas Woman's University

_____

-----Original Message-----

From: Sian Lawson [mailto:sian.lawson@newcastle.ac.uk]

Sent: Tuesday, May 22, 2007 8:11 AM

To: Glenn Fleisig; BIOMCH-L@NIC.SURFNET.NL

Subject: RE: [BIOMCH-L] Significant digits - a Quiz for our community

Glenn,

This is a good illustration of the disparity between a statistically

significant difference and a functionally (or clinically) significant

difference. Your t-test may show a statistical difference in the

populations, but the difference is still negligibly small.

Statistics, even means, cannot create accuracy. If the accuracy of your

recording is to one decimal place, then your means etc will not be any

more accurate than that.

Best regards,

Sian

Dr. Sian E. M. Lawson

Centre for Rehabilitation and Engineering Studies (CREST)

School of Mechanical and Systems Engineering

University of Newcastle upon Tyne, U.K.

CREST: +44 (0)191 222 6170

Direct: +44 (0)191 222 8224

Fax: +44 (0)191 222 8600

_____

-----Original Message-----

From: borgbros@netti.fi [mailto:borgbros@netti.fi]

Sent: Tuesday, May 22, 2007 8:16 AM

To: Glenn Fleisig

Subject: Re: [BIOMCH-L] Significant digits - a Quiz for our community

Pax!

A quick answer is that if one is investigating the mean of x_1, ..., x_N

then the *resolution* improves as stdev{x}/sqrt{N} when the number of

data N increases. Thus, reporting mean value, enough decimals should be

included so as to resolve the group standard deviation.

Regards Frank Borg

skrev

_____

-----Original Message-----

From: Torrence Welch [mailto:twelch@neuro.gatech.edu]

Sent: Tuesday, May 22, 2007 8:50 AM

To: Glenn Fleisig

Subject: Re: [BIOMCH-L] Significant digits - a Quiz for our community

Hi Glenn,

You should report values with the same number of significant digits as

the

least precise measurement in your study. Since the equipment in your

example reported values with 3 sig figs, your average measurements

should

be reported as:

> Technique A mean: 25.3 units

>

> Technique B mean: 25.4 units

>

> Average difference : 0.0284 units

>

> Paired t-test p-value: 0.0350

Even if some of your measurements were over 100 units, you would still

report 3 sig figs. You would move to 4 sig figs only if all measurements

surpassed 100 units and you maintained 0.1 precision on your equipment.

Happy researching,

--

Torrence D.J. Welch, MSE

Graduate Research Assistant / PhD Candidate

Laboratory for Neuroengineering

Wallace H. Coulter Department of Biomedical Engineering

Georgia Institute of Technology / Emory University

313 Ferst Drive

Atlanta, GA 30332-0535

(404) 385-4340 [lab]

_____

From: D. Gordon E. Robertson, PhD [mailto:dger@uOttawa.ca]

Sent: Tuesday, May 22, 2007 9:29 AM

To: Glenn Fleisig

Subject: Re: [BIOMCH-L] Significant digits - a Quiz for our community

For biomechanics courses and elsewhere I use a method originally

outlined

in an engineering mechanics textbook by Beer & Johnston. It is

ultimately based

on the accuracy of slide rules but it works quite well in biomechanics

where we

can't get highly accurate measures of most body segment parameters.

The concept is to make numbers accurate to 0.5%. That means all numbers

are rounded to three significant figures unless the first number is a

"1" in which

case use four. Students like it because it is definitive and easy to

remember.

Note that for a final answer to be accurate to 0.5% you must carry an

extra digit

in intermediate results.

Cheers

_____

-----Original Message-----

From: David Stodden [mailto:dfstodd@bgnet.bgsu.edu]

Sent: Tuesday, May 22, 2007 10:31 AM

To: Glenn Fleisig

Subject: Re: [BIOMCH-L] Significant digits - a Quiz for our community

Greetings Glenn,

I seem to remember from some distant measurement or statistics class

that

you report one decimal place more than your measurement device. Thus, if

you are measuring degrees (e.g., 0-360 degrees) then you provide your

statistic to the tenths of degrees - 90.8. If your calculations provide

you

with an extra 3 or 4 decimal places, you simply round to the nearest

decimal that is one decimal from your designated measurement. Thus,

90.842222 degrees would still be reported as 90.8 degrees. Ultimately, I

believe it depends not only what you are measuring and the specific

instrumentation device, but also what type of sensitivity you demand.

That

is my best recollection.

I agree with the practical significance information that was presented

by

the other two respondents, but the practical significance is related to

the

effect size statistics, which are based on your power estimates and

previous literature effect sizes, that should be reported in your

manuscripts.

Dave

_____

-----Original Message-----

From: David Smith [mailto:david.smithpodiatrist@tesco.net]

Sent: Tuesday, May 22, 2007 12:45 PM

To: Glenn Fleisig; BIOMCH-L@NIC.SURFNET.NL

Subject: Re: [BIOMCH-L] Significant digits - a Quiz for our community

Dear Glenn

I studied statistics for a while but I still don't really understand

them.

For me they are a tautology in that they use their own rules to prove

themselves. If you use different rules to prove something different its

not

valid because you didn't use the rules of statistics.

So if your two different groups had intervention =A and no intervention

=B

and A and B averages had no significant difference but A had a highly

variable sequence of scores and B was fairly even across the board.

Surley

this might intuitively indicate that there was a significant clinical

difference in the out comes, which is what we are interested in, isn't

it?

Also what if the scores recorded had an exponential relationship with

the

baseline value how would that effect the statistical significance. Or if

you

where measuring mass for instance then 0.01grams in a range of 0-2grams

may

be significant but in a range of 1000-10,000 grams it may not be

What if your sensor has poor linearity or your sample rate was to low

etc

etc ie if your recording is to several decimal places then you must be

sure

that the input data is reliable and accurate enough to support that

standard.

At the end of the day is it not down to the reader to decide if the

research

and statistics are suitable for the conclusions that they may want to

take

from a set of results.

Cheers Dave

_____

-----Original Message-----

From: David Smith [mailto:david.smithpodiatrist@tesco.net]

Sent: Tuesday, May 22, 2007 1:12 PM

To: Glenn Fleisig; BIOMCH-L@NIC.SURFNET.NL

Subject: Re: [BIOMCH-L] Significant digits - a Quiz for our community

Hello again Glenn

Just another point

The P value is the indication of the confidence in rejecting the null

hypothesis.

If your P value is 0.035 (alpha =

Hi everyone,

This is general question relevant to a lot of our research. It should

be simple, but I don't recall the answer.

How many digits should be used in reporting data?

Here is an example. A biomechanist is trying to show a difference

between Technique A and Technique B. He/she recruits 36 subjects and

records each one of them performing both techniques, using equipment

that measures each person to the nearest 0.1 unit. How many decimal

places should be used in reporting the data? I can make a "common

sense" determination, but is there some accepted procedure or guideline?

Example data:

TECHNIQUE A

Subject 1: 13.3 units

Subject 2: 35.0 units

Subject 3: 22.2 units

Subject 36: 18.4 units

TECHNIQUE B

Subject 1: 12.9 units

Subject 2: 35.0 units

Subject 3: 21.9 units

Subject 36: 18.5 units

Even though the mean within-subject difference is less than 0.1 units, a

paired t-test reveals a statistically significant difference.

Technique A mean: 25.33333 units

Technique B mean: 25.36170 units

Average difference : 0.02837 units

Paired t-test p-value: 0.035

How many decimal places would you use in reporting these mean values and

difference in your paper? Why?

Thanks in advance,

- Glenn S. Fleisig, Ph.D.

Glenn S. Fleisig, Ph.D., Smith & Nephew Chair of Research

American Sports Medicine Institute

833 St. Vincent's Drive, Suite 100

Birmingham, AL 35205

(email) glennf@asmi.org

(tel) 205-918-2139

www.asmi.org

_____

*** THE ANSWER ***

Thank you to all who responded. Based on the responses it is clearly a

community that is bright and polite. :-)

There were many well-written, insightful responses, which I've posted

below. From these responses, my own experience, and conversations with

others (biostatisticians, etc.), I've concluded that there is no hard

scientific rule for significant digits. Therefore, I am please to share

my conclusions from all of these thoughts, named (with tongue-in-cheek):

Fleisig's Four Steps for Significant Digits

1. In general, do not present individual values, mean values,

standard deviations, or other calculated values in smaller units than

the accuracy of your equipment.

2. If you find a statistical significance with a magnitude less

than the accuracy of the measurement, then question whether strict

enough statistics were used. For example, if you find a small

difference to be significant with a t-test, then perhaps the alpha-level

was set too high.

3. If you decide the statistical test was appropriate, then report

the values with the added decimal place.

4. If you report such a statistical difference, then explain

whether or not you believe the difference has practical significance.

In most cases, the researcher will decide the small difference has no

practical significance. If you feel the difference is important, then

you probably should have used more accurate equipment.

_____

*** THE RESPONSES ***

_____

From: Michael Cinelli [mailto:Michael_Cinelli@brown.edu]

Sent: Tuesday, May 22, 2007 7:25 AM

To: Glenn Fleisig

Subject: Re: [BIOMCH-L] Significant digits - a Quiz for our community

Hi Glenn,

You can only use the number of decimal places that is equivalent to the

precision of your instrumentation.

Michael Cinelli

Postdoctoral Fellow

Dept. of Cognitive and Linguistic Sciences

Brown University

190 Thayer St. Rm. 102F

Providence, RI

02912

Michael_Cinelli@brown.edu

Every body continues in its state of rest, or of uniform motion in a

right line, unless it is compelled to change that state by forces

impressed upon it - Sir Isaac Newton

_____

From: Chris Kirtley [mailto:ckirtley@gmail.com]

Sent: Tuesday, May 22, 2007 7:32 AM

To: Glenn Fleisig

Cc: BIOMCH-L@nic.surfnet.nl

Subject: Re: [BIOMCH-L] Significant digits - a Quiz for our community

Dear Glenn,

That's a great example - wish I'd had it to hand when I used to nag my

students about this.

The key point to understand is of course the accuracy of the measurement

- just because there is a readout with 3 significant figures doesn't

mean that the measurement is accurate to 1 part in a thousand.

The t-test also illustrates the difference between statistical

signifance and practical significance. Clearly the measured difference

is practically insignificant even if it is statistically significant

(even assuming that the t-test was appropriate, which it may not be). It

would be interesting to recalculate with the input data rounded off to

integers, though even then it really is the practical signifiacnce of

the difference that's most important.

BTW, there's nothing special about decimal places. We should also round

off integers according to the measurement accuracy. For example, it

makes no sense to record blood pressure to the nearest mmHg - it should

be rounded to the nearest 5 mmHg at least (10 is probably more

realistic).

Chris

_____

-----Original Message-----

From: garyalan59@peoplepc.com [mailto:garyalan59@peoplepc.com]

Sent: Tuesday, May 22, 2007 7:45 AM

To: Glenn Fleisig

Subject: Re: [BIOMCH-L] Significant digits - a Quiz for our community

In your example, the program calculates a difference, but that

difference is

smaller than your ability to differentiate (measure). The general rule

(from

my chemistry background) is that you cannot report values that are more

"accurate" than your original measure, because the reporting implies you

can

actually measure what you are reporting to the degree you are reporting.

Additionally, just because something is statistically significant does

not

mean it is clinically significant.

Just my two cents.

Gary Christopher

Doctoral Candidate

Dept. of Kinesiology

Texas Woman's University

_____

-----Original Message-----

From: Sian Lawson [mailto:sian.lawson@newcastle.ac.uk]

Sent: Tuesday, May 22, 2007 8:11 AM

To: Glenn Fleisig; BIOMCH-L@NIC.SURFNET.NL

Subject: RE: [BIOMCH-L] Significant digits - a Quiz for our community

Glenn,

This is a good illustration of the disparity between a statistically

significant difference and a functionally (or clinically) significant

difference. Your t-test may show a statistical difference in the

populations, but the difference is still negligibly small.

Statistics, even means, cannot create accuracy. If the accuracy of your

recording is to one decimal place, then your means etc will not be any

more accurate than that.

Best regards,

Sian

Dr. Sian E. M. Lawson

Centre for Rehabilitation and Engineering Studies (CREST)

School of Mechanical and Systems Engineering

University of Newcastle upon Tyne, U.K.

CREST: +44 (0)191 222 6170

Direct: +44 (0)191 222 8224

Fax: +44 (0)191 222 8600

_____

-----Original Message-----

From: borgbros@netti.fi [mailto:borgbros@netti.fi]

Sent: Tuesday, May 22, 2007 8:16 AM

To: Glenn Fleisig

Subject: Re: [BIOMCH-L] Significant digits - a Quiz for our community

Pax!

A quick answer is that if one is investigating the mean of x_1, ..., x_N

then the *resolution* improves as stdev{x}/sqrt{N} when the number of

data N increases. Thus, reporting mean value, enough decimals should be

included so as to resolve the group standard deviation.

Regards Frank Borg

skrev

_____

-----Original Message-----

From: Torrence Welch [mailto:twelch@neuro.gatech.edu]

Sent: Tuesday, May 22, 2007 8:50 AM

To: Glenn Fleisig

Subject: Re: [BIOMCH-L] Significant digits - a Quiz for our community

Hi Glenn,

You should report values with the same number of significant digits as

the

least precise measurement in your study. Since the equipment in your

example reported values with 3 sig figs, your average measurements

should

be reported as:

> Technique A mean: 25.3 units

>

> Technique B mean: 25.4 units

>

> Average difference : 0.0284 units

>

> Paired t-test p-value: 0.0350

Even if some of your measurements were over 100 units, you would still

report 3 sig figs. You would move to 4 sig figs only if all measurements

surpassed 100 units and you maintained 0.1 precision on your equipment.

Happy researching,

--

Torrence D.J. Welch, MSE

Graduate Research Assistant / PhD Candidate

Laboratory for Neuroengineering

Wallace H. Coulter Department of Biomedical Engineering

Georgia Institute of Technology / Emory University

313 Ferst Drive

Atlanta, GA 30332-0535

(404) 385-4340 [lab]

_____

From: D. Gordon E. Robertson, PhD [mailto:dger@uOttawa.ca]

Sent: Tuesday, May 22, 2007 9:29 AM

To: Glenn Fleisig

Subject: Re: [BIOMCH-L] Significant digits - a Quiz for our community

For biomechanics courses and elsewhere I use a method originally

outlined

in an engineering mechanics textbook by Beer & Johnston. It is

ultimately based

on the accuracy of slide rules but it works quite well in biomechanics

where we

can't get highly accurate measures of most body segment parameters.

The concept is to make numbers accurate to 0.5%. That means all numbers

are rounded to three significant figures unless the first number is a

"1" in which

case use four. Students like it because it is definitive and easy to

remember.

Note that for a final answer to be accurate to 0.5% you must carry an

extra digit

in intermediate results.

Cheers

_____

-----Original Message-----

From: David Stodden [mailto:dfstodd@bgnet.bgsu.edu]

Sent: Tuesday, May 22, 2007 10:31 AM

To: Glenn Fleisig

Subject: Re: [BIOMCH-L] Significant digits - a Quiz for our community

Greetings Glenn,

I seem to remember from some distant measurement or statistics class

that

you report one decimal place more than your measurement device. Thus, if

you are measuring degrees (e.g., 0-360 degrees) then you provide your

statistic to the tenths of degrees - 90.8. If your calculations provide

you

with an extra 3 or 4 decimal places, you simply round to the nearest

decimal that is one decimal from your designated measurement. Thus,

90.842222 degrees would still be reported as 90.8 degrees. Ultimately, I

believe it depends not only what you are measuring and the specific

instrumentation device, but also what type of sensitivity you demand.

That

is my best recollection.

I agree with the practical significance information that was presented

by

the other two respondents, but the practical significance is related to

the

effect size statistics, which are based on your power estimates and

previous literature effect sizes, that should be reported in your

manuscripts.

Dave

_____

-----Original Message-----

From: David Smith [mailto:david.smithpodiatrist@tesco.net]

Sent: Tuesday, May 22, 2007 12:45 PM

To: Glenn Fleisig; BIOMCH-L@NIC.SURFNET.NL

Subject: Re: [BIOMCH-L] Significant digits - a Quiz for our community

Dear Glenn

I studied statistics for a while but I still don't really understand

them.

For me they are a tautology in that they use their own rules to prove

themselves. If you use different rules to prove something different its

not

valid because you didn't use the rules of statistics.

So if your two different groups had intervention =A and no intervention

=B

and A and B averages had no significant difference but A had a highly

variable sequence of scores and B was fairly even across the board.

Surley

this might intuitively indicate that there was a significant clinical

difference in the out comes, which is what we are interested in, isn't

it?

Also what if the scores recorded had an exponential relationship with

the

baseline value how would that effect the statistical significance. Or if

you

where measuring mass for instance then 0.01grams in a range of 0-2grams

may

be significant but in a range of 1000-10,000 grams it may not be

What if your sensor has poor linearity or your sample rate was to low

etc

etc ie if your recording is to several decimal places then you must be

sure

that the input data is reliable and accurate enough to support that

standard.

At the end of the day is it not down to the reader to decide if the

research

and statistics are suitable for the conclusions that they may want to

take

from a set of results.

Cheers Dave

_____

-----Original Message-----

From: David Smith [mailto:david.smithpodiatrist@tesco.net]

Sent: Tuesday, May 22, 2007 1:12 PM

To: Glenn Fleisig; BIOMCH-L@NIC.SURFNET.NL

Subject: Re: [BIOMCH-L] Significant digits - a Quiz for our community

Hello again Glenn

Just another point

The P value is the indication of the confidence in rejecting the null

hypothesis.

If your P value is 0.035 (alpha =