Garry T Allison

01-24-2005, 12:00 PM

My understanding of the arbitrary "line in the sand" of 0.05 was

originally due to the choice of the original tables (pre computer)

used to calculated the specific confidence limits (someone told me

this originated back to Fisher's original tables).

This is now the "standard Scientific threshold" but in reality is is

just a threshold that people tend to agree that they are willing to

accept that there is a possibility of 1 in 20 chance that they are

reporting a type I statistical error. Of course if the result is to

be acted upon and there is a serious implication of a significant

outcome (i.e leg amputation - or the finding goes against 20 other

findings) then there has to be some consideration as to setting the

alpha level to a more stringent level of confidence. Done prior to

the actual testing of course.

The beta level or Power reflects the line in the sand which is draw

to accept the possibility that there is insufficient statistical

power to detect a difference - when there is actually one. this

type II statistical error is easier to live with (supposedly due to

the nature of the conservatism of assuming the null hypothesis is

true) since the general value is usually set at 80% level of

confidence.

The p value reflects the probability of the observed change happening

by chance. It says little as to the magnitude of the change since the

p value reflects the effect / change relative to the variations in

the data. Therefore a mean change of 4 degrees could reflect p

=.054 and a change in another parameter of 2 degrees could be p

=.0001. The magnitude of these changes is best interpreted by

reporting the confidence limits of the change. If the confidence

limits includes zero then the change is not significant at that level

of confidence.

In many aspects of biomechanics the instrumentation and processing

techniques and the ability (power) to derive data with small levels

of random error allow very small systematic difference to be detected

(at the level of statistical significance).

This is probably why (and rightly so) there is so much importance

placed in understanding the assumptions of the biomechanical models

and the "black box approach" in various automatic instrumentation

system.

Clinically.

The real challenge is being able to report the confidence limits of

the magnitude of the changes.

Inspite of observing statistically significant changes (say P

originally due to the choice of the original tables (pre computer)

used to calculated the specific confidence limits (someone told me

this originated back to Fisher's original tables).

This is now the "standard Scientific threshold" but in reality is is

just a threshold that people tend to agree that they are willing to

accept that there is a possibility of 1 in 20 chance that they are

reporting a type I statistical error. Of course if the result is to

be acted upon and there is a serious implication of a significant

outcome (i.e leg amputation - or the finding goes against 20 other

findings) then there has to be some consideration as to setting the

alpha level to a more stringent level of confidence. Done prior to

the actual testing of course.

The beta level or Power reflects the line in the sand which is draw

to accept the possibility that there is insufficient statistical

power to detect a difference - when there is actually one. this

type II statistical error is easier to live with (supposedly due to

the nature of the conservatism of assuming the null hypothesis is

true) since the general value is usually set at 80% level of

confidence.

The p value reflects the probability of the observed change happening

by chance. It says little as to the magnitude of the change since the

p value reflects the effect / change relative to the variations in

the data. Therefore a mean change of 4 degrees could reflect p

=.054 and a change in another parameter of 2 degrees could be p

=.0001. The magnitude of these changes is best interpreted by

reporting the confidence limits of the change. If the confidence

limits includes zero then the change is not significant at that level

of confidence.

In many aspects of biomechanics the instrumentation and processing

techniques and the ability (power) to derive data with small levels

of random error allow very small systematic difference to be detected

(at the level of statistical significance).

This is probably why (and rightly so) there is so much importance

placed in understanding the assumptions of the biomechanical models

and the "black box approach" in various automatic instrumentation

system.

Clinically.

The real challenge is being able to report the confidence limits of

the magnitude of the changes.

Inspite of observing statistically significant changes (say P