Tweet
As you may recall, I posted the following inquiry a few weeks
back:
"I recently had the opportunity to attend the NIH Midwest
Regional Seminar at Northwestern University. Of particular
interest was a discussion focusing upon science misconduct and
science fraud. Apparently, Congress is beginning to exert a
great deal of pressure on NIH to develop guidelines for dealing
with cases of science misconduct and science fraud. I won't
attempt to present the entire discussion but I would like to
share the thoughts I presented in hopes of getting feedback from
the group at large. The discussion began with a case study in
which a junior faculty member accused an established researcher
of 'sloppy data collection.'
The discussion then moved into whether it was the responsibility
of the university or NIH to deal with accusations of science
fraud/misconduct. I began by asking if there was a universally
accepted operational definition of "sloppy data". The
answer given was no and that point has been brought home in court
cases won by persons accused of science misconduct who
subsequently sued. Given that answer, I pointed out that it is
highly unlikely that anyone within my university has the
expertise to evaluate my data collection procedures or is aware
of what may be considered standard practice within my area of
expertise. I am extremely apprehensive about allowing someone
not expert in my field make judgements about my procedures.
Similarly, I believe several conflict of interest issues are
raised by leaving the responsibility with the university. On the
other hand, giving NIH the responsibility to conduct such
investigations will require additional administration,
administration which would most likely suck money out of research
funds which are already difficult to obtain.
My suggestion was to allow either the university or NIH oversee
or coordinate any investigation but to leave the ultimate
investigation up to the professional organizations to which the
accused belongs. In fact, either of those bodies should be
responsible for compendensating and coordinating the
investigators. What do you think? While the thought of such
instances is uncomfortable and highly unlikely, would it be to
the advantage of ISB, ASB, CSB, etc. to formulate policies for
dealing with questions of science misconduct within our area of
expertise? It seems to me that if the professional societies
have a vested interest, it is in keeping the field pure. How
would we, as a field, distinguish between idiosyncracies in data
collection and sloppy data collection? Would standardized data
collection/analysis methods provide an adequate base for making
such a distinction?"
While there were relatively few responses, the responses received
raise many important points. The issues raised require a great
deal of thought and discussion. The replies I received accompany
this note in their entirety. While I am not sure that I am the
best person to moderate a discussion of the issues raised, I am
firmly convinced that such a discussion should occur and I will
follow up after the list has had time to consider the following:
>> "I think you have raised an important issue. Also it is very
controversial. It reminds me of investigations of incompetance by
the engineering professional association that I am licensed by.
A professional society may very well be given the responsibilty
for judging whether data is fraudulently obtained or just
sloppily gathered. You then need some legal legislation to make
the society responsible for the credentials of the members.
My wife is involved with a new area, Art Therapy. This is
psychotherapy using art as part of the communication method and
there are professional organizations that set up and monitor the
members qualifications, degrees, diplomas AND internship hours.
In my own case the Province of Ontario has made the Association
of Professional Engineers on Ontario responsible for licensing
engineers. You must have a degree in engineering from an
accredited school and two years of experience and now may even
have to write an exam. This is only to work for others. If you
want to consult as an expert you need another licence and futher
qualifications.
I also belong to the Canadian Medical and Biological Engineering
Society. They have special criteria for CLINICAL ENGINEERS that
also include special exams, degrees, and experience.
You have opened up a serious matter. Let us know what you find
out from replies to your posting.
Good luck,
Marvin Sherebrin P. Eng.
Ph. D.
M. H. Sherebrin
Associate Professor
Dept. of Medical Biophysics
Univ. of Western Ontario
London, Ontario, Canada N6A 5C1
email: sherebrin@uwovax.uwo.ca"
************************************************** ***************
>> "Two points in response to your posting:
1) Universities and NIH have, so far, shown themselves to be
unable, or at least unwilling, to address misconduct issues.
Blundering response by MIT permitted the Baltimore affair to
become a circus event in the Congress, although prompt unbiased
investigation might have resolved the issue completely. An NIH
case concerning claims of misconduct by V.C. Mow by NIH reseacher
C. McCutchen, was an example of NIH's own leaden feet in dealing
with researchers of established reputation and power base in the
academic/research system. The biomedical engineering field seems
particularly vulnerable to bad practice, perhaps because there
are so few competing authorities in any one specific area and the
system fosters an atmosphere where hard questions are not
welcomed.
2) On the same theme as my last point, any researcher with
experience in instrumentation and measurement, even if unrelated
to your particular area, should be able to make informed
assessments of your techniques. The fundamental principles of
instrumentation and measurement are the same. Someone with
experience would also be able to pose questions and/or respond to
your defense of practice specific to your field in judging the
quality of data or presence of deliberate negligence and
misconduct. Otherwise you seem to imply that no one is capable of
judging your work. A researcher's job is to be able to defend not
just the results of his work, but the methods used to obtain
them. If you are comfortable with your approach, then honest
inquiry should not be a problem and may even stimulate a new
avenue of thought.
I will concede that in the political climate of today, honest
assessment may be hard to find. But it also may be more likely
that you would obtain an unbiased assessment from someone who is
not and will not be a competitor for funds or publications.
later
Mike Murphy
Mechanical Engineering
LSU
memcm@lsuvax.sncc.lsu.edu"
************************************************** ***************
>> "I just read a hardcopy of your posting on BIOMECH-L, and I
would like to thank you for bringing this issue up. This is just
one of a number of politically driven issues that the
biomechanics community should be preparing to deal with
proactively. I have two specific comments:
1) It is possible to develop community standards for data
collection and documentation. In areas that have been more
politically contentious and litigious, such as occupational
epidemiology, codes of "good practice" have been developed. I
recall seeing such a guideline in the Journal of Occupational
Medicine. I think I can find a copy for you if you would like.
2) We need to think about the problem of whether electronic
data storage is archival. As the David Baltimore case points
out, great weight is placed on the dating and forensic analysis
of laboratory notebooks. Too many biomechanics labs do not
insist on keeping lab notebooks, and much of the data is left in
softcopy form in poorly organized fashion. In such an
environment researchers leave themselves wide open to allegations
of "sloppy data collection," because they cannot prove otherwise
with archival materials. It one occured to me that what we need
is a central repository of biomechanics data where all
biomechanics researchers would E-mail their data sets immediately
after data collection. The repository would make tape backups
and document the date the data were received. Thus,
participating researchers would be able prove that they didn't
fudge the data during data reduction and analysis. Although this
wouldn't address allegations of improper lab procedure, it would
provide some protection. I have absolutely no idea who would
fund this service, how it would be administered, or what legal
status it would have, but it's a random thought.
Again, thanks for bringing this subject up. I hope the ASB,
ISB, etc. decide to do something about it. I would be willing
to contribute if it does.
Richard Hughes
Research Fellow
Orthopedic Biomechanics Lab
Mayo Foundation
hughr2@Mayo.EDU"
************************************************** ***************
>> "I've just read your recent posting to BIOMCH-L, and I'd like
to contribute my ten cents to your discussion.
Although I believe your suggestion of using our professional
organizations (ISB, ASB, etc) as mediators/judge/jury/whatever in
cases of professional misconduct is well intended, I'm afraid I
strongly disagree with your proposal. The "old boy" network is
far too strong in such organizations--not just in biomechanics,
but in most fields of endeavor. Surely you can see the potential
conflicts of interest in this approach ... "Oh! I've known him
for years, he wouldn't do a thing like that!" ... "Oh, him.
Yes, he's been doing that for years!" ... and so on. Frankly, I
don't think you can use people who know each other as a jury, you
need unbiased, independent thinkers who stand to gain/lose
nothing by their decision. Just imagine upsetting the "old boy"
network. You don't have to be a genius to identify the possible
reprisals!
Let me suggest an alternative. There are several large
professional organizations of national standing ... the American
Association for the Advancement of Science (AAAS), The National
Academy of Sciences (NAS), Sigma Chi, etc., all with members
well-versed in fundamental scientific principles. Why not use
them as independent arbitrators? Sigma Chi, for example, has
expressed considerable interest in scientific fraud.
No matter where you go for arbitration, it will cost you. It's
naive to think that ISB, ASB, ... would be able to perform a
thorough investigation without an additional source of funding.
Frankly, I'd rather see government support for an independent
body, than membership support for local investigations which,
frankly, could destroy the organization with in-fighting. I
think our professional organizations should stand back and watch
the chips fall as they may. If an independent jury needed
field-specific advice, it could be provided, but the independent
body would lead the investigation and (hopefully) eliminate
conflicts of interest among scientists with common interests--and
personal agendas.
I appreciate the forthright manner in which you have brought this
topic to the attention of our membership. It needs discussing.
I look forward to seeing the other responses you receive.
Sincerely,
Jim Walton
************************************************** ***************
JAMES S. WALTON, Ph.D.
President
4D VIDEO
3136 Pauline Drive
SEBASTOPOL, CA 95472
PHONE: 707/829-8883
FAX : 707/829-3527
INTERNET : FourD@crl.com --or--
Jim.Walton@Forsythe.Stanford.Edu
BITNET : Jim.Walton@Stanford.Bitnet
COMPUSERVE : 72644,2773"
back:
"I recently had the opportunity to attend the NIH Midwest
Regional Seminar at Northwestern University. Of particular
interest was a discussion focusing upon science misconduct and
science fraud. Apparently, Congress is beginning to exert a
great deal of pressure on NIH to develop guidelines for dealing
with cases of science misconduct and science fraud. I won't
attempt to present the entire discussion but I would like to
share the thoughts I presented in hopes of getting feedback from
the group at large. The discussion began with a case study in
which a junior faculty member accused an established researcher
of 'sloppy data collection.'
The discussion then moved into whether it was the responsibility
of the university or NIH to deal with accusations of science
fraud/misconduct. I began by asking if there was a universally
accepted operational definition of "sloppy data". The
answer given was no and that point has been brought home in court
cases won by persons accused of science misconduct who
subsequently sued. Given that answer, I pointed out that it is
highly unlikely that anyone within my university has the
expertise to evaluate my data collection procedures or is aware
of what may be considered standard practice within my area of
expertise. I am extremely apprehensive about allowing someone
not expert in my field make judgements about my procedures.
Similarly, I believe several conflict of interest issues are
raised by leaving the responsibility with the university. On the
other hand, giving NIH the responsibility to conduct such
investigations will require additional administration,
administration which would most likely suck money out of research
funds which are already difficult to obtain.
My suggestion was to allow either the university or NIH oversee
or coordinate any investigation but to leave the ultimate
investigation up to the professional organizations to which the
accused belongs. In fact, either of those bodies should be
responsible for compendensating and coordinating the
investigators. What do you think? While the thought of such
instances is uncomfortable and highly unlikely, would it be to
the advantage of ISB, ASB, CSB, etc. to formulate policies for
dealing with questions of science misconduct within our area of
expertise? It seems to me that if the professional societies
have a vested interest, it is in keeping the field pure. How
would we, as a field, distinguish between idiosyncracies in data
collection and sloppy data collection? Would standardized data
collection/analysis methods provide an adequate base for making
such a distinction?"
While there were relatively few responses, the responses received
raise many important points. The issues raised require a great
deal of thought and discussion. The replies I received accompany
this note in their entirety. While I am not sure that I am the
best person to moderate a discussion of the issues raised, I am
firmly convinced that such a discussion should occur and I will
follow up after the list has had time to consider the following:
>> "I think you have raised an important issue. Also it is very
controversial. It reminds me of investigations of incompetance by
the engineering professional association that I am licensed by.
A professional society may very well be given the responsibilty
for judging whether data is fraudulently obtained or just
sloppily gathered. You then need some legal legislation to make
the society responsible for the credentials of the members.
My wife is involved with a new area, Art Therapy. This is
psychotherapy using art as part of the communication method and
there are professional organizations that set up and monitor the
members qualifications, degrees, diplomas AND internship hours.
In my own case the Province of Ontario has made the Association
of Professional Engineers on Ontario responsible for licensing
engineers. You must have a degree in engineering from an
accredited school and two years of experience and now may even
have to write an exam. This is only to work for others. If you
want to consult as an expert you need another licence and futher
qualifications.
I also belong to the Canadian Medical and Biological Engineering
Society. They have special criteria for CLINICAL ENGINEERS that
also include special exams, degrees, and experience.
You have opened up a serious matter. Let us know what you find
out from replies to your posting.
Good luck,
Marvin Sherebrin P. Eng.
Ph. D.
M. H. Sherebrin
Associate Professor
Dept. of Medical Biophysics
Univ. of Western Ontario
London, Ontario, Canada N6A 5C1
email: sherebrin@uwovax.uwo.ca"
************************************************** ***************
>> "Two points in response to your posting:
1) Universities and NIH have, so far, shown themselves to be
unable, or at least unwilling, to address misconduct issues.
Blundering response by MIT permitted the Baltimore affair to
become a circus event in the Congress, although prompt unbiased
investigation might have resolved the issue completely. An NIH
case concerning claims of misconduct by V.C. Mow by NIH reseacher
C. McCutchen, was an example of NIH's own leaden feet in dealing
with researchers of established reputation and power base in the
academic/research system. The biomedical engineering field seems
particularly vulnerable to bad practice, perhaps because there
are so few competing authorities in any one specific area and the
system fosters an atmosphere where hard questions are not
welcomed.
2) On the same theme as my last point, any researcher with
experience in instrumentation and measurement, even if unrelated
to your particular area, should be able to make informed
assessments of your techniques. The fundamental principles of
instrumentation and measurement are the same. Someone with
experience would also be able to pose questions and/or respond to
your defense of practice specific to your field in judging the
quality of data or presence of deliberate negligence and
misconduct. Otherwise you seem to imply that no one is capable of
judging your work. A researcher's job is to be able to defend not
just the results of his work, but the methods used to obtain
them. If you are comfortable with your approach, then honest
inquiry should not be a problem and may even stimulate a new
avenue of thought.
I will concede that in the political climate of today, honest
assessment may be hard to find. But it also may be more likely
that you would obtain an unbiased assessment from someone who is
not and will not be a competitor for funds or publications.
later
Mike Murphy
Mechanical Engineering
LSU
memcm@lsuvax.sncc.lsu.edu"
************************************************** ***************
>> "I just read a hardcopy of your posting on BIOMECH-L, and I
would like to thank you for bringing this issue up. This is just
one of a number of politically driven issues that the
biomechanics community should be preparing to deal with
proactively. I have two specific comments:
1) It is possible to develop community standards for data
collection and documentation. In areas that have been more
politically contentious and litigious, such as occupational
epidemiology, codes of "good practice" have been developed. I
recall seeing such a guideline in the Journal of Occupational
Medicine. I think I can find a copy for you if you would like.
2) We need to think about the problem of whether electronic
data storage is archival. As the David Baltimore case points
out, great weight is placed on the dating and forensic analysis
of laboratory notebooks. Too many biomechanics labs do not
insist on keeping lab notebooks, and much of the data is left in
softcopy form in poorly organized fashion. In such an
environment researchers leave themselves wide open to allegations
of "sloppy data collection," because they cannot prove otherwise
with archival materials. It one occured to me that what we need
is a central repository of biomechanics data where all
biomechanics researchers would E-mail their data sets immediately
after data collection. The repository would make tape backups
and document the date the data were received. Thus,
participating researchers would be able prove that they didn't
fudge the data during data reduction and analysis. Although this
wouldn't address allegations of improper lab procedure, it would
provide some protection. I have absolutely no idea who would
fund this service, how it would be administered, or what legal
status it would have, but it's a random thought.
Again, thanks for bringing this subject up. I hope the ASB,
ISB, etc. decide to do something about it. I would be willing
to contribute if it does.
Richard Hughes
Research Fellow
Orthopedic Biomechanics Lab
Mayo Foundation
hughr2@Mayo.EDU"
************************************************** ***************
>> "I've just read your recent posting to BIOMCH-L, and I'd like
to contribute my ten cents to your discussion.
Although I believe your suggestion of using our professional
organizations (ISB, ASB, etc) as mediators/judge/jury/whatever in
cases of professional misconduct is well intended, I'm afraid I
strongly disagree with your proposal. The "old boy" network is
far too strong in such organizations--not just in biomechanics,
but in most fields of endeavor. Surely you can see the potential
conflicts of interest in this approach ... "Oh! I've known him
for years, he wouldn't do a thing like that!" ... "Oh, him.
Yes, he's been doing that for years!" ... and so on. Frankly, I
don't think you can use people who know each other as a jury, you
need unbiased, independent thinkers who stand to gain/lose
nothing by their decision. Just imagine upsetting the "old boy"
network. You don't have to be a genius to identify the possible
reprisals!
Let me suggest an alternative. There are several large
professional organizations of national standing ... the American
Association for the Advancement of Science (AAAS), The National
Academy of Sciences (NAS), Sigma Chi, etc., all with members
well-versed in fundamental scientific principles. Why not use
them as independent arbitrators? Sigma Chi, for example, has
expressed considerable interest in scientific fraud.
No matter where you go for arbitration, it will cost you. It's
naive to think that ISB, ASB, ... would be able to perform a
thorough investigation without an additional source of funding.
Frankly, I'd rather see government support for an independent
body, than membership support for local investigations which,
frankly, could destroy the organization with in-fighting. I
think our professional organizations should stand back and watch
the chips fall as they may. If an independent jury needed
field-specific advice, it could be provided, but the independent
body would lead the investigation and (hopefully) eliminate
conflicts of interest among scientists with common interests--and
personal agendas.
I appreciate the forthright manner in which you have brought this
topic to the attention of our membership. It needs discussing.
I look forward to seeing the other responses you receive.
Sincerely,
Jim Walton
************************************************** ***************
JAMES S. WALTON, Ph.D.
President
4D VIDEO
3136 Pauline Drive
SEBASTOPOL, CA 95472
PHONE: 707/829-8883
FAX : 707/829-3527
INTERNET : FourD@crl.com --or--
Jim.Walton@Forsythe.Stanford.Edu
BITNET : Jim.Walton@Stanford.Bitnet
COMPUSERVE : 72644,2773"