Tweet
Very interesting, Adrian!
For those interested, the original article is at:
http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=1182327
I have often thought that research papers have following weaknesses:
1. In the introduction or background (which should build the case for the
question being answered), reference is often made to statements by others in
a similar section of their paper, or speculative comments in the discussion.
In my view, reference should only be made to the conclusions previous
papers.
2. Particularly in biomechanics (and even more sports biomechanics) I
suspect, scientists often rely on their intuitive feelings about what is
true, refusing to believe results which have been confirmed over and over
again.
3. The numbers of subjects and tests are usually so small as to make the
application of statistics absurd.
Great food for discussion, Adrian & David!
Chris
On Feb 20, 2008 7:15 PM, Adrian Smith wrote:
>
> May be relevant:
>
> Jim Giles interviews John Ioannidis for "New Scientist"
>
> 16th Feb 2008, pp.44-45 (Vol. 197)
>
> e-man-who-would-prove-all-studies-wrong.html>
>
> "When the clinical epidemiologist John Ioannidis published a paper
> entitled "Why most published research findings are false" in 2005, he
> made a lot of scientists very uncomfortable. The study was the result of
> 15 years' work cataloguing the factors that plague the interpretation of
> scientific results, such as the misuse of statistics or poor
> experimental design"
>
> Title: Why most published research findings are false
> Author(s): Ioannidis, JPA
> Source: PLOS MEDICINE Volume: 2 Issue: 8 Pages: 696-701
> Published: AUG 2005
>
> "Some colleagues and I have looked at high-profile papers, with over
> 1000 citations each, that were later completely contradicted by large,
> well-conducted studies. One example is the finding that beta-carotene
> protects against cancer. It doesn't, but we found a sizeable component
> of literature where these original beliefs were still supported. It's
> hard to believe the researchers had never heard they had been refuted.
>
> Jim Giles: "How should we promote the studies that produce more credible
> results, rather than those that are simply statistically significant?"
>
> "There are several ways to do this. One: do larger, well-designed
> studies. Two: instead of having 10 teams of researchers, each working
> behind closed doors, investigators should collaborate and study the same
> questions. All the data should be made publicly available. If one team
> comes up with an interesting result then the whole consortium should try
> to replicate it. Much of the work I've been doing for the past 10 years
> has been about creating consortia to carry out research. The experience
> has been very positive.
>
> From issue 2643 of "New Scientist" magazine, 16 February 2008, page
> 44-45
>
> Title: Why most published research findings are false: Author's reply to
> Goodman and Greenland
> Author(s): Ioannidis, JPA
> Source: PLOS MEDICINE Volume: 4 Pages: 1132-1133 Published: 2007
>
> Title: Limitations are not properly acknowledged in the scientific
> literature
> Author(s): Ioannidis, JPA
> Source: JOURNAL OF CLINICAL EPIDEMIOLOGY Volume: 60 Issue: 4
> Pages: 324-329 Published: APR 2007
>
> Title: Why most published research findings are false: Problems in the
> analysis
> Author(s): Goodman, S; Greenland, S
> Source: PLOS MEDICINE Volume: 4 Issue: 4 Pages: 773-773
> Published: APR 2007
>
> Title: Most published research findings are false- but a little
> replication goes a long way
> Author(s): Moonesinghe, R; Khoury, MJ; Janssens, ACJW
> Source: PLOS MEDICINE Volume: 4 Issue: 2 Pages: 218-221
> Published: FEB 2007
>
> Title: THE DISSEMINATION OF FALSE DATA THROUGH INADEQUATE CITATION
> Author(s): LARSSON, KS
> Source: JOURNAL OF INTERNAL MEDICINE Volume: 238 Issue: 5 Pages:
> 445-450 Published: NOV 1995
>
> --------
>
> Adrian Smith
> Leeds University Library
> +44 (0)113 3435531
>
>
> -----Original Message-----
> From: * Biomechanics and Movement Science listserver
> [mailto:BIOMCH-L@NIC.SURFNET.NL] On Behalf Of McFarlane, David
> Sent: 19 February 2008 22:36
> To: BIOMCH-L@NIC.SURFNET.NL
> Subject: [BIOMCH-L] What criteria should we use for interpreting
> epidemiology?
>
> Dear all,
>
> Lately in my professional reading I have noticed that the standards of
> evidence varies greatly between different fields. For instance, the
> mode of establishing causality used in science (which was devised by
> Jakob Henle and Robert Koch for research in bacteriology) requires that
> in every single instance the effect follows the cause. By contrast
> epidemiology uses a system of causal inference that is based on the
> philosophies of David Hume and John Stuart Mill (Morabia, 2005). The
> question of identifying causes in epidemiology has always been a subject
> of controversy. For instance, the controversy over the interpretation of
> the statistical relationship between smoking and lung cancer caused a
> landmark debate in that field in the second half of the twentieth
> century. It led Bradford Hill to formulate the "pragmatics" of risk
> factor epidemiology in 1965 (Berlivet, 2005). His model for establishing
> causation ("The Hill causation model") is well known in the public
> health field and widely used. For example Hill's nine proposed
> "criteria" for determining causation were used was used to classify
> Chrysotile asbestos as a cause of mesothelioma (Lemen, 2004); it met all
> nine of them. However, "multiple causation" is the canon of contemporary
> epidemiology and its "web of causation" is widely accepted though it is
> a very poorly elaborated model (Krieger, 1994). Sadly public heath
> debates these days are often based on evidence that use it in
> questionable ways and this topic clearly deserves further debate. Has
> anyone found a good review of criteria currently in use for interpreting
> causation?
>
> Regards,
>
> David McFarlane MAppSc (Ergonomics)
> Ergonomist, WorkCover NSW
>
> Disclaimer
>
> Any recommendation concerning the use or representation of a particular
> brand of product in this document or any mention of them whatsoever
> (whether this appears in the text, illustrations, photographs or in any
> other form) is not to be taken to imply that WorkCover NSW approves or
> endorses the product or the brand.
>
> ************************************************** **********************
> ************************************************** **************
>
> This message, including any attached files, is intended solely for the
> addressee named and may contain confidential information. If you are not
> the intended recipient, please delete it and notify the sender. Any
> views expressed in this message are those of the individual sender and
> are not necessarily the views of WorkCover NSW. Please consider the
> environment before printing this E-mail.
>
> ************************************************** **********************
> ************************************************** **************
>
>
> ---------------------------------------------------------------
> Information about BIOMCH-L: http://www.Biomch-L.org
> Archives: http://listserv.surfnet.nl/archives/Biomch-L.html
> ---------------------------------------------------------------
>
> ---------------------------------------------------------------
> Information about BIOMCH-L: http://www.Biomch-L.org
> Archives: http://listserv.surfnet.nl/archives/Biomch-L.html
> ---------------------------------------------------------------
>
--
Dr. Chris Kirtley MB ChB, PhD
608 Dockside
44 Ferry St.
Kangaroo Point
Queensland 4169
Australia
Tel. (07) 3891 6644 x 1608
Fax 3891 6900
West End Family Medical Centre
Mondays, Tuesdays & Thursday afternoon (07) 3844 4111
Clinical Gait Analysis: http://www.univie.ac.at/cga
Book:
http://www.amazon.co.uk/exec/obidos/ASIN/0443100098/203-6674734-4427132
For those interested, the original article is at:
http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=1182327
I have often thought that research papers have following weaknesses:
1. In the introduction or background (which should build the case for the
question being answered), reference is often made to statements by others in
a similar section of their paper, or speculative comments in the discussion.
In my view, reference should only be made to the conclusions previous
papers.
2. Particularly in biomechanics (and even more sports biomechanics) I
suspect, scientists often rely on their intuitive feelings about what is
true, refusing to believe results which have been confirmed over and over
again.
3. The numbers of subjects and tests are usually so small as to make the
application of statistics absurd.
Great food for discussion, Adrian & David!
Chris
On Feb 20, 2008 7:15 PM, Adrian Smith wrote:
>
> May be relevant:
>
> Jim Giles interviews John Ioannidis for "New Scientist"
>
> 16th Feb 2008, pp.44-45 (Vol. 197)
>
> e-man-who-would-prove-all-studies-wrong.html>
>
> "When the clinical epidemiologist John Ioannidis published a paper
> entitled "Why most published research findings are false" in 2005, he
> made a lot of scientists very uncomfortable. The study was the result of
> 15 years' work cataloguing the factors that plague the interpretation of
> scientific results, such as the misuse of statistics or poor
> experimental design"
>
> Title: Why most published research findings are false
> Author(s): Ioannidis, JPA
> Source: PLOS MEDICINE Volume: 2 Issue: 8 Pages: 696-701
> Published: AUG 2005
>
> "Some colleagues and I have looked at high-profile papers, with over
> 1000 citations each, that were later completely contradicted by large,
> well-conducted studies. One example is the finding that beta-carotene
> protects against cancer. It doesn't, but we found a sizeable component
> of literature where these original beliefs were still supported. It's
> hard to believe the researchers had never heard they had been refuted.
>
> Jim Giles: "How should we promote the studies that produce more credible
> results, rather than those that are simply statistically significant?"
>
> "There are several ways to do this. One: do larger, well-designed
> studies. Two: instead of having 10 teams of researchers, each working
> behind closed doors, investigators should collaborate and study the same
> questions. All the data should be made publicly available. If one team
> comes up with an interesting result then the whole consortium should try
> to replicate it. Much of the work I've been doing for the past 10 years
> has been about creating consortia to carry out research. The experience
> has been very positive.
>
> From issue 2643 of "New Scientist" magazine, 16 February 2008, page
> 44-45
>
> Title: Why most published research findings are false: Author's reply to
> Goodman and Greenland
> Author(s): Ioannidis, JPA
> Source: PLOS MEDICINE Volume: 4 Pages: 1132-1133 Published: 2007
>
> Title: Limitations are not properly acknowledged in the scientific
> literature
> Author(s): Ioannidis, JPA
> Source: JOURNAL OF CLINICAL EPIDEMIOLOGY Volume: 60 Issue: 4
> Pages: 324-329 Published: APR 2007
>
> Title: Why most published research findings are false: Problems in the
> analysis
> Author(s): Goodman, S; Greenland, S
> Source: PLOS MEDICINE Volume: 4 Issue: 4 Pages: 773-773
> Published: APR 2007
>
> Title: Most published research findings are false- but a little
> replication goes a long way
> Author(s): Moonesinghe, R; Khoury, MJ; Janssens, ACJW
> Source: PLOS MEDICINE Volume: 4 Issue: 2 Pages: 218-221
> Published: FEB 2007
>
> Title: THE DISSEMINATION OF FALSE DATA THROUGH INADEQUATE CITATION
> Author(s): LARSSON, KS
> Source: JOURNAL OF INTERNAL MEDICINE Volume: 238 Issue: 5 Pages:
> 445-450 Published: NOV 1995
>
> --------
>
> Adrian Smith
> Leeds University Library
> +44 (0)113 3435531
>
>
> -----Original Message-----
> From: * Biomechanics and Movement Science listserver
> [mailto:BIOMCH-L@NIC.SURFNET.NL] On Behalf Of McFarlane, David
> Sent: 19 February 2008 22:36
> To: BIOMCH-L@NIC.SURFNET.NL
> Subject: [BIOMCH-L] What criteria should we use for interpreting
> epidemiology?
>
> Dear all,
>
> Lately in my professional reading I have noticed that the standards of
> evidence varies greatly between different fields. For instance, the
> mode of establishing causality used in science (which was devised by
> Jakob Henle and Robert Koch for research in bacteriology) requires that
> in every single instance the effect follows the cause. By contrast
> epidemiology uses a system of causal inference that is based on the
> philosophies of David Hume and John Stuart Mill (Morabia, 2005). The
> question of identifying causes in epidemiology has always been a subject
> of controversy. For instance, the controversy over the interpretation of
> the statistical relationship between smoking and lung cancer caused a
> landmark debate in that field in the second half of the twentieth
> century. It led Bradford Hill to formulate the "pragmatics" of risk
> factor epidemiology in 1965 (Berlivet, 2005). His model for establishing
> causation ("The Hill causation model") is well known in the public
> health field and widely used. For example Hill's nine proposed
> "criteria" for determining causation were used was used to classify
> Chrysotile asbestos as a cause of mesothelioma (Lemen, 2004); it met all
> nine of them. However, "multiple causation" is the canon of contemporary
> epidemiology and its "web of causation" is widely accepted though it is
> a very poorly elaborated model (Krieger, 1994). Sadly public heath
> debates these days are often based on evidence that use it in
> questionable ways and this topic clearly deserves further debate. Has
> anyone found a good review of criteria currently in use for interpreting
> causation?
>
> Regards,
>
> David McFarlane MAppSc (Ergonomics)
> Ergonomist, WorkCover NSW
>
> Disclaimer
>
> Any recommendation concerning the use or representation of a particular
> brand of product in this document or any mention of them whatsoever
> (whether this appears in the text, illustrations, photographs or in any
> other form) is not to be taken to imply that WorkCover NSW approves or
> endorses the product or the brand.
>
> ************************************************** **********************
> ************************************************** **************
>
> This message, including any attached files, is intended solely for the
> addressee named and may contain confidential information. If you are not
> the intended recipient, please delete it and notify the sender. Any
> views expressed in this message are those of the individual sender and
> are not necessarily the views of WorkCover NSW. Please consider the
> environment before printing this E-mail.
>
> ************************************************** **********************
> ************************************************** **************
>
>
> ---------------------------------------------------------------
> Information about BIOMCH-L: http://www.Biomch-L.org
> Archives: http://listserv.surfnet.nl/archives/Biomch-L.html
> ---------------------------------------------------------------
>
> ---------------------------------------------------------------
> Information about BIOMCH-L: http://www.Biomch-L.org
> Archives: http://listserv.surfnet.nl/archives/Biomch-L.html
> ---------------------------------------------------------------
>
--
Dr. Chris Kirtley MB ChB, PhD
608 Dockside
44 Ferry St.
Kangaroo Point
Queensland 4169
Australia
Tel. (07) 3891 6644 x 1608
Fax 3891 6900
West End Family Medical Centre
Mondays, Tuesdays & Thursday afternoon (07) 3844 4111
Clinical Gait Analysis: http://www.univie.ac.at/cga
Book:
http://www.amazon.co.uk/exec/obidos/ASIN/0443100098/203-6674734-4427132