PDA

View Full Version : Re: Stats Power. Report Confidence Limits - p values



Garry T Allison
01-24-2005, 12:00 PM
My understanding of the arbitrary "line in the sand" of 0.05 was
originally due to the choice of the original tables (pre computer)
used to calculated the specific confidence limits (someone told me
this originated back to Fisher's original tables).

This is now the "standard Scientific threshold" but in reality is is
just a threshold that people tend to agree that they are willing to
accept that there is a possibility of 1 in 20 chance that they are
reporting a type I statistical error. Of course if the result is to
be acted upon and there is a serious implication of a significant
outcome (i.e leg amputation - or the finding goes against 20 other
findings) then there has to be some consideration as to setting the
alpha level to a more stringent level of confidence. Done prior to
the actual testing of course.
The beta level or Power reflects the line in the sand which is draw
to accept the possibility that there is insufficient statistical
power to detect a difference - when there is actually one. this
type II statistical error is easier to live with (supposedly due to
the nature of the conservatism of assuming the null hypothesis is
true) since the general value is usually set at 80% level of
confidence.

The p value reflects the probability of the observed change happening
by chance. It says little as to the magnitude of the change since the
p value reflects the effect / change relative to the variations in
the data. Therefore a mean change of 4 degrees could reflect p
=.054 and a change in another parameter of 2 degrees could be p
=.0001. The magnitude of these changes is best interpreted by
reporting the confidence limits of the change. If the confidence
limits includes zero then the change is not significant at that level
of confidence.

In many aspects of biomechanics the instrumentation and processing
techniques and the ability (power) to derive data with small levels
of random error allow very small systematic difference to be detected
(at the level of statistical significance).

This is probably why (and rightly so) there is so much importance
placed in understanding the assumptions of the biomechanical models
and the "black box approach" in various automatic instrumentation
system.

Clinically.
The real challenge is being able to report the confidence limits of
the magnitude of the changes.
Inspite of observing statistically significant changes (say P