Tweet
I agree with Mr. DeWitt that NNT might be a more useful clinical
assessment. I think of greater impact is seeing the potential affect of
interventions using a freeware piece of software called Visual Rx. This
can be downloaded from http://www.nntonline.net/
As I recall from my stats classes, using the t-test to test the
hypothesis that the experimental treatment is different than the control
as described would require doing 4 separate t-tests. One would need to
show that the groups are the same at the start of the study and
different at the end. Thus, t-tests would need to be calculated to
compare within each group, one pre and post and one t-test each to
compare between groups pre and post treatment. Having thus tested ones
hypothesis with the four t-tests the true probability of Type 1 error
would now be approximately the sum of the four values calculated as the
probability of Type 1 error by each of the four t-test. Instead one
should have used an ANOVA or other model. One could "get away" with a
single t-test if one tested using one score from both groups, delta (pre
minus post) scores and then an unpaired t-test. Could someone with a
better knowledge of statistics correct me if I am wrong. (And there is
a good probability of that!)
Finally regarding clinical significance I think a better way to describe
this is that the difference matters from point of view of the
patients/subjects. As an example, an intervention that is intended to
increase knee joint range of motion post-op, needs to result in a
difference that matters to a patient not to the researcher. If one
intervention improved range of motion, to a statistically significant
degree over another by 5 degrees, such a small increase in knee range of
motion would not be considered, from a patient's viewpoint, as
clinically important. It might make walking a tad easier but not that it
would be really important to them.
Dr. Stephen Perle
DEWITT, JOHN K. (JSC-SK) (WLS) wrote:
>Dr. Etnyre provided a dental example where there was an 80% success rate. If
>the dentist were doing the treatment and wanted to determine if the results
>were statistically significant, he could set up an experiment with the
>hypotheses:
>H0 (null): there will be no difference in the condition of TMJ with the
>application of a splint compared to no treatment
>H1 (alternative): there will be a difference
>
>He would then need a measure (I am not a dentist, but lets just say it is
>the peak force generated during a biting task). To test his hypothesis, he
>could take before and after measures from the same group, or compare the
>test group to a matched group of controls. This would involve a t-test
>(paired or two sample, depending upon the study design). Since a t-test
>compares means, a large enough change in 80% of the patients could drive the
>group mean to a value that is statistically significant. The fact that there
>was a statistically significant difference, however, would not necessarily
>mean that there is a clinically significant difference. The 80% success rate
>does not figure into the calculation.
>
>In the case of this example, using an epidemiological measure like number
>needed to treat (NNT) might be better to help interpret if the treatment was
>useful. The NNT is the number of patients necessary to treat for one
>incident (in this case, TMJ improvement) to occur. You would need to compare
>the success rate of the procedure (80%), to the success rate of another
>procedure. This is good for nominal data, which is how the example was
>presented. Effect size is another good way to measure the magnitude of the
>difference to help determine how different means really are.
>
>Clinical significance, in my interpretation, is a result that makes a
>difference in normal, daily life. Sometimes results are deemed significant
>because of statistical significance that do not make much difference
>clinically because there was a large sample size. On the contrary, sometimes
>results may not show statistical significance but have clinical significance
>(like preventing a highly contagious disease or a death). While the former
>can be tested mathematically, I think that the latter is due to the
>interpretation of the researcher (in combination with the test statistics).
>
>John DeWitt, M.S., C.S.C.S.
>Biomechanist - Exercise Physiology Laboratory
>Space Physiology & Countermeasures
>Johnson Space Center
>Houston, TX 77058
>281-483-8939 / 281-483-4181 (fax)
>
>
>-----------------------------------------------------------------
>To unsubscribe send SIGNOFF BIOMCH-L to LISTSERV@nic.surfnet.nl
>For information and archives: http://isb.ri.ccf.org/biomch-l
>-----------------------------------------------------------------
>
>
--
Stephen M. Perle, D.C., M.S.
Associate Professor of Clinical Sciences
Adjunct Professor of Mechanical Engineering
University of Bridgeport
Bridgeport, CT 06601 USA
www.bridgeport.edu/~perle
Ethics Articles www.chiroweb.com/columnist/perle
Speaker's Bureau www.ncmic.com/6026/speakers.htm
------------------------------------------------------------------------
Real knowledge is to know the extent of one's ignorance.
- Confucius
-----------------------------------------------------------------
To unsubscribe send SIGNOFF BIOMCH-L to LISTSERV@nic.surfnet.nl
For information and archives: http://isb.ri.ccf.org/biomch-l
-----------------------------------------------------------------
assessment. I think of greater impact is seeing the potential affect of
interventions using a freeware piece of software called Visual Rx. This
can be downloaded from http://www.nntonline.net/
As I recall from my stats classes, using the t-test to test the
hypothesis that the experimental treatment is different than the control
as described would require doing 4 separate t-tests. One would need to
show that the groups are the same at the start of the study and
different at the end. Thus, t-tests would need to be calculated to
compare within each group, one pre and post and one t-test each to
compare between groups pre and post treatment. Having thus tested ones
hypothesis with the four t-tests the true probability of Type 1 error
would now be approximately the sum of the four values calculated as the
probability of Type 1 error by each of the four t-test. Instead one
should have used an ANOVA or other model. One could "get away" with a
single t-test if one tested using one score from both groups, delta (pre
minus post) scores and then an unpaired t-test. Could someone with a
better knowledge of statistics correct me if I am wrong. (And there is
a good probability of that!)
Finally regarding clinical significance I think a better way to describe
this is that the difference matters from point of view of the
patients/subjects. As an example, an intervention that is intended to
increase knee joint range of motion post-op, needs to result in a
difference that matters to a patient not to the researcher. If one
intervention improved range of motion, to a statistically significant
degree over another by 5 degrees, such a small increase in knee range of
motion would not be considered, from a patient's viewpoint, as
clinically important. It might make walking a tad easier but not that it
would be really important to them.
Dr. Stephen Perle
DEWITT, JOHN K. (JSC-SK) (WLS) wrote:
>Dr. Etnyre provided a dental example where there was an 80% success rate. If
>the dentist were doing the treatment and wanted to determine if the results
>were statistically significant, he could set up an experiment with the
>hypotheses:
>H0 (null): there will be no difference in the condition of TMJ with the
>application of a splint compared to no treatment
>H1 (alternative): there will be a difference
>
>He would then need a measure (I am not a dentist, but lets just say it is
>the peak force generated during a biting task). To test his hypothesis, he
>could take before and after measures from the same group, or compare the
>test group to a matched group of controls. This would involve a t-test
>(paired or two sample, depending upon the study design). Since a t-test
>compares means, a large enough change in 80% of the patients could drive the
>group mean to a value that is statistically significant. The fact that there
>was a statistically significant difference, however, would not necessarily
>mean that there is a clinically significant difference. The 80% success rate
>does not figure into the calculation.
>
>In the case of this example, using an epidemiological measure like number
>needed to treat (NNT) might be better to help interpret if the treatment was
>useful. The NNT is the number of patients necessary to treat for one
>incident (in this case, TMJ improvement) to occur. You would need to compare
>the success rate of the procedure (80%), to the success rate of another
>procedure. This is good for nominal data, which is how the example was
>presented. Effect size is another good way to measure the magnitude of the
>difference to help determine how different means really are.
>
>Clinical significance, in my interpretation, is a result that makes a
>difference in normal, daily life. Sometimes results are deemed significant
>because of statistical significance that do not make much difference
>clinically because there was a large sample size. On the contrary, sometimes
>results may not show statistical significance but have clinical significance
>(like preventing a highly contagious disease or a death). While the former
>can be tested mathematically, I think that the latter is due to the
>interpretation of the researcher (in combination with the test statistics).
>
>John DeWitt, M.S., C.S.C.S.
>Biomechanist - Exercise Physiology Laboratory
>Space Physiology & Countermeasures
>Johnson Space Center
>Houston, TX 77058
>281-483-8939 / 281-483-4181 (fax)
>
>
>-----------------------------------------------------------------
>To unsubscribe send SIGNOFF BIOMCH-L to LISTSERV@nic.surfnet.nl
>For information and archives: http://isb.ri.ccf.org/biomch-l
>-----------------------------------------------------------------
>
>
--
Stephen M. Perle, D.C., M.S.
Associate Professor of Clinical Sciences
Adjunct Professor of Mechanical Engineering
University of Bridgeport
Bridgeport, CT 06601 USA
www.bridgeport.edu/~perle
Ethics Articles www.chiroweb.com/columnist/perle
Speaker's Bureau www.ncmic.com/6026/speakers.htm
------------------------------------------------------------------------
Real knowledge is to know the extent of one's ignorance.
- Confucius
-----------------------------------------------------------------
To unsubscribe send SIGNOFF BIOMCH-L to LISTSERV@nic.surfnet.nl
For information and archives: http://isb.ri.ccf.org/biomch-l
-----------------------------------------------------------------