sperle33

01-26-2005, 11:34 PM

I agree with Mr. DeWitt that NNT might be a more useful clinical

assessment. I think of greater impact is seeing the potential affect of

interventions using a freeware piece of software called Visual Rx. This

can be downloaded from http://www.nntonline.net/

As I recall from my stats classes, using the t-test to test the

hypothesis that the experimental treatment is different than the control

as described would require doing 4 separate t-tests. One would need to

show that the groups are the same at the start of the study and

different at the end. Thus, t-tests would need to be calculated to

compare within each group, one pre and post and one t-test each to

compare between groups pre and post treatment. Having thus tested ones

hypothesis with the four t-tests the true probability of Type 1 error

would now be approximately the sum of the four values calculated as the

probability of Type 1 error by each of the four t-test. Instead one

should have used an ANOVA or other model. One could "get away" with a

single t-test if one tested using one score from both groups, delta (pre

minus post) scores and then an unpaired t-test. Could someone with a

better knowledge of statistics correct me if I am wrong. (And there is

a good probability of that!)

Finally regarding clinical significance I think a better way to describe

this is that the difference matters from point of view of the

patients/subjects. As an example, an intervention that is intended to

increase knee joint range of motion post-op, needs to result in a

difference that matters to a patient not to the researcher. If one

intervention improved range of motion, to a statistically significant

degree over another by 5 degrees, such a small increase in knee range of

motion would not be considered, from a patient's viewpoint, as

clinically important. It might make walking a tad easier but not that it

would be really important to them.

Dr. Stephen Perle

DEWITT, JOHN K. (JSC-SK) (WLS) wrote:

>Dr. Etnyre provided a dental example where there was an 80% success rate. If

>the dentist were doing the treatment and wanted to determine if the results

>were statistically significant, he could set up an experiment with the

>hypotheses:

>H0 (null): there will be no difference in the condition of TMJ with the

>application of a splint compared to no treatment

>H1 (alternative): there will be a difference

>

>He would then need a measure (I am not a dentist, but lets just say it is

>the peak force generated during a biting task). To test his hypothesis, he

>could take before and after measures from the same group, or compare the

>test group to a matched group of controls. This would involve a t-test

>(paired or two sample, depending upon the study design). Since a t-test

>compares means, a large enough change in 80% of the patients could drive the

>group mean to a value that is statistically significant. The fact that there

>was a statistically significant difference, however, would not necessarily

>mean that there is a clinically significant difference. The 80% success rate

>does not figure into the calculation.

>

>In the case of this example, using an epidemiological measure like number

>needed to treat (NNT) might be better to help interpret if the treatment was

>useful. The NNT is the number of patients necessary to treat for one

>incident (in this case, TMJ improvement) to occur. You would need to compare

>the success rate of the procedure (80%), to the success rate of another

>procedure. This is good for nominal data, which is how the example was

>presented. Effect size is another good way to measure the magnitude of the

>difference to help determine how different means really are.

>

>Clinical significance, in my interpretation, is a result that makes a

>difference in normal, daily life. Sometimes results are deemed significant

>because of statistical significance that do not make much difference

>clinically because there was a large sample size. On the contrary, sometimes

>results may not show statistical significance but have clinical significance

>(like preventing a highly contagious disease or a death). While the former

>can be tested mathematically, I think that the latter is due to the

>interpretation of the researcher (in combination with the test statistics).

>

>John DeWitt, M.S., C.S.C.S.

>Biomechanist - Exercise Physiology Laboratory

>Space Physiology & Countermeasures

>Johnson Space Center

>Houston, TX 77058

>281-483-8939 / 281-483-4181 (fax)

>

>

>-----------------------------------------------------------------

>To unsubscribe send SIGNOFF BIOMCH-L to LISTSERV@nic.surfnet.nl

>For information and archives: http://isb.ri.ccf.org/biomch-l

>-----------------------------------------------------------------

>

>

--

Stephen M. Perle, D.C., M.S.

Associate Professor of Clinical Sciences

Adjunct Professor of Mechanical Engineering

University of Bridgeport

Bridgeport, CT 06601 USA

www.bridgeport.edu/~perle

Ethics Articles www.chiroweb.com/columnist/perle

Speaker's Bureau www.ncmic.com/6026/speakers.htm

------------------------------------------------------------------------

Real knowledge is to know the extent of one's ignorance.

- Confucius

-----------------------------------------------------------------

To unsubscribe send SIGNOFF BIOMCH-L to LISTSERV@nic.surfnet.nl

For information and archives: http://isb.ri.ccf.org/biomch-l

-----------------------------------------------------------------

assessment. I think of greater impact is seeing the potential affect of

interventions using a freeware piece of software called Visual Rx. This

can be downloaded from http://www.nntonline.net/

As I recall from my stats classes, using the t-test to test the

hypothesis that the experimental treatment is different than the control

as described would require doing 4 separate t-tests. One would need to

show that the groups are the same at the start of the study and

different at the end. Thus, t-tests would need to be calculated to

compare within each group, one pre and post and one t-test each to

compare between groups pre and post treatment. Having thus tested ones

hypothesis with the four t-tests the true probability of Type 1 error

would now be approximately the sum of the four values calculated as the

probability of Type 1 error by each of the four t-test. Instead one

should have used an ANOVA or other model. One could "get away" with a

single t-test if one tested using one score from both groups, delta (pre

minus post) scores and then an unpaired t-test. Could someone with a

better knowledge of statistics correct me if I am wrong. (And there is

a good probability of that!)

Finally regarding clinical significance I think a better way to describe

this is that the difference matters from point of view of the

patients/subjects. As an example, an intervention that is intended to

increase knee joint range of motion post-op, needs to result in a

difference that matters to a patient not to the researcher. If one

intervention improved range of motion, to a statistically significant

degree over another by 5 degrees, such a small increase in knee range of

motion would not be considered, from a patient's viewpoint, as

clinically important. It might make walking a tad easier but not that it

would be really important to them.

Dr. Stephen Perle

DEWITT, JOHN K. (JSC-SK) (WLS) wrote:

>Dr. Etnyre provided a dental example where there was an 80% success rate. If

>the dentist were doing the treatment and wanted to determine if the results

>were statistically significant, he could set up an experiment with the

>hypotheses:

>H0 (null): there will be no difference in the condition of TMJ with the

>application of a splint compared to no treatment

>H1 (alternative): there will be a difference

>

>He would then need a measure (I am not a dentist, but lets just say it is

>the peak force generated during a biting task). To test his hypothesis, he

>could take before and after measures from the same group, or compare the

>test group to a matched group of controls. This would involve a t-test

>(paired or two sample, depending upon the study design). Since a t-test

>compares means, a large enough change in 80% of the patients could drive the

>group mean to a value that is statistically significant. The fact that there

>was a statistically significant difference, however, would not necessarily

>mean that there is a clinically significant difference. The 80% success rate

>does not figure into the calculation.

>

>In the case of this example, using an epidemiological measure like number

>needed to treat (NNT) might be better to help interpret if the treatment was

>useful. The NNT is the number of patients necessary to treat for one

>incident (in this case, TMJ improvement) to occur. You would need to compare

>the success rate of the procedure (80%), to the success rate of another

>procedure. This is good for nominal data, which is how the example was

>presented. Effect size is another good way to measure the magnitude of the

>difference to help determine how different means really are.

>

>Clinical significance, in my interpretation, is a result that makes a

>difference in normal, daily life. Sometimes results are deemed significant

>because of statistical significance that do not make much difference

>clinically because there was a large sample size. On the contrary, sometimes

>results may not show statistical significance but have clinical significance

>(like preventing a highly contagious disease or a death). While the former

>can be tested mathematically, I think that the latter is due to the

>interpretation of the researcher (in combination with the test statistics).

>

>John DeWitt, M.S., C.S.C.S.

>Biomechanist - Exercise Physiology Laboratory

>Space Physiology & Countermeasures

>Johnson Space Center

>Houston, TX 77058

>281-483-8939 / 281-483-4181 (fax)

>

>

>-----------------------------------------------------------------

>To unsubscribe send SIGNOFF BIOMCH-L to LISTSERV@nic.surfnet.nl

>For information and archives: http://isb.ri.ccf.org/biomch-l

>-----------------------------------------------------------------

>

>

--

Stephen M. Perle, D.C., M.S.

Associate Professor of Clinical Sciences

Adjunct Professor of Mechanical Engineering

University of Bridgeport

Bridgeport, CT 06601 USA

www.bridgeport.edu/~perle

Ethics Articles www.chiroweb.com/columnist/perle

Speaker's Bureau www.ncmic.com/6026/speakers.htm

------------------------------------------------------------------------

Real knowledge is to know the extent of one's ignorance.

- Confucius

-----------------------------------------------------------------

To unsubscribe send SIGNOFF BIOMCH-L to LISTSERV@nic.surfnet.nl

For information and archives: http://isb.ri.ccf.org/biomch-l

-----------------------------------------------------------------