PDA

View Full Version : Summary: Numerical models and clinical recommendations



mviceconti63
07-11-2004, 05:42 PM
Dear Colleagues:
What follows may be one of the longest
messages ever posted on BIOMCH-L. Thus, before I
challenge your patience, let me summarise the
content of this essay. Some time ago I posted a
question on which criteria we should use when
reviewing papers that draw clinically relevant
conclusions using the results of numerical
models. My original question is reported below
in the last section, where also all the replies I
got are reported.

On the basis of the replies I collected, plus
some further reading, plus some useful
discussions I had during the ESB meeting in Den
Bosch, I wrote a brief essay entitled 'The use of
finite element analysis to produce
recommendations for the clinical practice', that
you find here below. Such essay can be
summarised as follow. I recommend four levels of
validation for a numerical model, and I suggest
for each type of journal which of them should be
considered mandatory or publication:

- Stage #1: verification and proper parameters
identification. This stage only ensures that the
model is numerically correct. This level should
be mandatory for any published paper.

- Stage #2: sensitivity analysis and
inter-subject variability. This level of
validation should be present in papers published
in applied biomechanics journals.

- Stage #3: validation against in vitro
controlled experiments: This level of validation
should be present in papers published in those
journals aiming at the space between biomechanics
research and clinical research.

- Stage #4: risk-benefit analysis and
retrospective clinical studies: This level of
validation should be present in papers published
in clinical research journals and should be
available before clinical trials are started.
This stage requires that the whole research line
is mature enough that we can identify a clear
clinical question to which we can answer using
our numerical model.

- Stage #5: prospective studies: before any
technology can be used routinely in the clinical
practice its validity should be proved
conclusively by a prospective trial.

I hope this draft will create a lively discussion
so that we can translate this draft essay into a
consensus paper on this important topic.

Regards

Marco Viceconti


----
Essay: The use of finite element analysis to
produce recommendations for the clinical practice


What is finite element model?
There was consensus among those who responded to
my original query that a finite element model is
conceptually equivalent to a theory. Thus,
presenting predictions of a non-validated finite
element model is very much like report personal
opinions. Any finite element model should be
verified and validated before we can consider its
predictions of any scientific value.


Verification and validation
We usually indicate the term verification to
indicate the process that ensures that a
numerical model accurately predicts the results
of the theoretical model it is based on. In
other words we verify a model by assessing its
numerical accuracy. On the contrary, the term
validation indicated the process that ensures
that our numerical model is accurately predicting
the physical phenomenon it was designed to
replicate. Thus, we validate a model by assessing
its accuracy tout court, i.e. the accuracy with
which it predicts the reality.

However, as for a theory, also for a numerical
model it is in general impossible to prove its
validity in a complete way. This is why the
formation of a scientific truth, which is a fully
validated theory, is a slow and sedimentary
process. In science we frequently use the
expression 'it is generally believed' to indicate
this consensus process.

It is evident that while the verification is a
process fully internal to the work of the single
scientist, and thus we can consider it mandatory
for publication, the validation process involves
the whole scientific community and in this sense
there is a sort of relativism in it.


Validation of numerical models to be used for clinical purposes
In my opinion the biggest difference between
physicists and engineers is that the physicists
can wait. Newton kept it universal gravitation
theory 30 years in a drawer, until he was able to
clarify all mathematical implications of his
work. Obviously, Newton was not in the need to
use that theory to do something practical, such
as building a house or an engine. Clinical
medicine faces a similar problem. The medical
professional must do something when is faced with
a suffering patient, even if he or she is not
100% sure that what is planning to do will truly
help the patient.

In contemporary medicine one way to formalise
this decisional process is the so-called
risk-benefit analysis. I propose that we should
start to consider this instrument for the
evaluation of the degree of validation of a
numerical model to be used in the clinical
practice. We accept the fact that no numerical
model can be totally validated. Thus, there is
always a risk associated to the use of the
predictions made by this model in the clinical
practice. So the first question we should try to
answer is: what is the level of risk to which we
are exposing the patient when we include the
results of the numerical model into the clinical
process? This is not an easy process. Firstly,
the more the model has been validated by means of
controlled experiments and/or clinical efficacy
studies, the lower is the risk that it provides
results that a re completely wrong. But the
other question we need to ask ourselves is: what
is the risk for the patient if the model is wrong
(within the limits of its validation)?

Once we have an idea of the risk associated with
the uncertainty of our model results, we have to
balance it with the benefits that the use of this
numerical model would bring to the patient, in
order to decide whether this is worth or not.
Obviously the researcher alone cannot conduct the
risk-benefit analysis; the whole
research-clinical team that is considering using
some numerical results into the clinical practice
should be responsible for it.


So, what we do on our journals?
The original motivation of this work was related
to the doubts we have to accept certain papers
entirely or mostly based on numerical models in
journals with a clinical target. On the basis of
the comments I received to my original query, and
because of the line of thoughts I reported above,
which derived form these comments, my conclusions
are:
- zero tolerance for verification: in no case we
should allow the publication of a paper where the
numerical model in use has not been fully
verified. For linear models we need to have
information of the convergence of the mesh
refinement, or even better on any post-hoc
indicator related to mesh refinement. If an
iterative solver is used, this should be made
clear and the convergence tolerance used should
be reported. When the model is non-linear the
verification should be specific for the type of
non-linearity present. If you are including
non-linear frictional contact it is appropriate
to report the peak penetration or equivalent
tolerance, and so on. Also, convergence
tolerance for the Newton-Raphson or other similar
schemes needs to be reported. Please notice that
the proper way to report these convergence
details is to write the convergence variable
(i.e. the force), the norm on which we check the
convergence, and the tolerance itself.

- Proper parameter identification: another
mandatory required for any numerical paper in
order to be published is the proper
identification of its parameters. Parameters
should be associated to physical/physiological
quantities and be independent by the time or the
repetition in the experiment used to identify
them. When one pretends to validate a model by
showing its ability to fit experimental results
over an interval of a given parameter, i.e. time,
we should check that the number of independent
parameters in the model is significantly lower
than the 'order' of the of the experimental
event. In this stage it may be reasonable to
investigate the sensitivity of a few critical
parameters, and eventually perform a 'design of
experiment' study in order to explore the
sensitivity of the solution to the uncertainty of
the parameters.

When a paper presents these features it should be
published in those journals that are interested
to the theoretical speculation, and the deductive
reasoning. In practice with a fully verified
model, we put forward a theory, and we can use
the numerical model to explore all its
implications. At this stage, when deductions are
made with respect to the clinical practice, it
should be made clear that these are theoretical
speculations, in need of further support.

- Sensitivity analysis: the second vital step in
the verification and validation process should be
the sensitivity analysis. Recent statistical
finite element analysis tools made this process
much simpler than in the past. Sensitivity
studies are essential in biomedical research,
where frequently the few available experimental
measurements are affected by large uncertainties.
Rather than 'using' these uncertainty to say that
the predictions of our model fall with the range
of the predicted experiment, it is necessary to
do a full sensitivity study that shows how this
uncertainty impacts on our deductions.

- Inter-subject variability: this is another type
of sensitivity analysis. We know that human been
are very different one to each other, from
anatomical and functional point of views. On the
other hand we usually create our models targeting
a specific subject or an idealistic average
subject. Thus, in principle, we should assess
how much our conclusions are affected by
inter-subject variability. It is very difficult
to provide strict guidelines here. In some cases
the inter-subject variability can be
parameterised, and thus included in the
sensitivity analysis. In these cases I would
consider this mandatory. In other cases you
simply need to build a new model for each new
subject you intend to describe, and this prevents
you from doing a systematic exploration on the
effects of inter-subject variability. In any case
the authors should consider this issue somehow.
One option may be to investigate a few subjects
that are representative of the extremes of
variability of the population of interest. This
is very similar to a design of experiments
approach, and it gives you a gross indication of
the level of variability you may expect in your
results.

At this stage of the validation process, we have
theoretical model that is robustly linked to the
experiments that are used to identify it. I
suggest that this second level of validation
should be mandatory to publish in those journals
aiming to applied biomechanics research.

- Validation against in vitro controlled
experiments: this is the first step in the true
validation process. It is usually very difficult
to perform them, and when you succeed they
usually show you a lot of unexpected weaknesses
in your model. Thus, I consider this a highly
valuable form of scientific result. One word
about how to report the results of this
comparison between numerical and experimental
results. Most authors use a linear regression
between measured and predicted value, and report
the regression parameters and coefficients.
However, I propose that we should always ask also
to report on the residuals of such regression.
One way I like is provide a root mean square
error as an indication of the average residual,
and the peak error, as indication of the maximum
residual.

My suggestion is that we should consider this
third level mandatory for all those journals
aiming at the space between biomechanics research
and clinical research.

- Risk-benefit analysis: as I wrote at the
beginning of this report, I believe that before
we can use results obtained from numerical models
in the clinical practice we need to report the
results of a risk-benefit analysis. To do so we
need a fully verified model on which a complete
sensitivity study and an in vitro validation
study has been conducted. This would provide us a
quantitative basis to estimate the risk
associated with the use of the model. Of course
these studies should be conducted in
collaboration with experts in clinical research,
which should have the necessary knowledge to
estimate the expected benefits.

- Retrospective studies: along the same lines,
but with greater level of confidence, there are
the retrospective studies. If you can use your
model to answer a clinical question over a
population for which the answer to this question
is known, you may get a good insight on the
validity of your model. In many cases the best
way to report these results is in term of
specificity and sensitivity, using the R.O.C.
curves. A key issue here is that the clinical
question must accept a yes-no answer.

When this type of validation studies is available
my suggestion is that we should allow the
publication of numerical studies also on the most
clinical journals. The clinical audience would
have in general clear enough the difference
between a retrospective and a prospective study.
Numerical models at this stage of validation
could also, in my opinion, be used in controlled
clinical trials.

- Prospective studies: as for any other method,
whenever possible the conclusive word on the
clinical use of a numerical model comes from
prospective clinical trials.


Code reliability
Some among those who replied to my original query
pointed out the issue of reliability of
commercial numerical analysis software, which is
a black box that may hide some problems. Some
others reversed the issue warning about the
danger of using software developed in-house.
After some thinking I decided that this has to do
with the general quality control we should apply
to all our laboratory instruments, including
software. Nobody is reporting in a scientific
paper when he calibrated last time the load cell
used in the study. This is left to our
consciences. For sure in our group we shall
start soon to develop a quality control system
for numerical analyses, very much like the one we
have already in place for the experimental
biomechanics unit. We expect to rely a lot on the
NAFEMS independent benchmarks for finite element
analysis codes.

----------------------------------------------------------------------------------------------------
Original posting:

I serve as a reviewer for a few biomechanics or orthopaedics
journals, some of which have a clear clinical perspective. In
particular I frequently revise papers where finite element models are
used as the main research tool.

These models are becoming more and more effective and powerful, and
it is not rare to find papers where the authors, on the basis on the
results obtained with the model, draw conclusions that may have a
clinical relevance, i.e. affect clinical decisions.

The problem I have is methodological: are we allowed to draw
clinically relevant conclusions from the predictions obtained by a
numerical model? Or, more appropriately, what are the conditions a
numerical model must fulfill in order to considered so reliable that
we can reasonably use it to draw clinically relevant conclusions?

In order to foster the discussion let me bring in my two cents.

In my understanding a numerical model is a particular instance of a
theoretical model. Once we are sure that the theoretical model has
been solved with sufficient numerical accuracy (and this is in
general possible with post-hoc indicators) what remains to be proved
is the adequacy of the theoretical model.

A theoretical model (a theory) in principle can be assumed to be true
(in the sense of a scientific truth, i.e. as far as we, as a
community we know, and within the limits of validity of the theory),
if with this model we can predict the outcomes of independent
experiments (independent in the sense that they are not those
measurements that were used to identify the model) and/or if starting
from the model we can deduct derivative conditions that are proved to
be true.

How do we translate these general rules in the specific of our
research domain, biomechanics? Can we say that once a model predicts
with sufficient accuracy the results of an in vitro experiment, we
can consider it valid and use it to draw clinically relevant
conclusions? Or is it sufficient to prove that the model is
numerically accurate and all the model assumptions and parameters are
well supported by experimental observations?

As usual I am ready to post a summery of the comments I shall receive.


----------------------------------------------------------------------------------------------------
Replies:

From: "Daniel P. Nicolella"

Marco,
You have raised an excellent point. Model simulation verification and
validation (V&V - essentially what you have eloquently described) is an
ongoing research topic within several disciplines. My colleague, Ben
Thacker, is actively involved in this area and has published several papers
on the topic of V&V. I have attached one such paper for your information
that should give you some background on the current thinking in this area.

I personally believe this is an important issue that should be addressed
within the biomechanics community and would like to see a consensus
developed on this topic.

Best Regards,
Dan Nicolella
----------------------------------------------------------------------------------------------------
From: "Mahar, Andrew"

Hi Marco,
You raise a very important issue in the world of orthopedic biomechanics.
We are conducting in-vitro experiments and numerical tests in parallel
across a variety of applications (spine/trauma) to optimize the numerical
theories, with the understanding that even in-vitro experiments can't
duplicate the in-vivo situation. At this point, the surgeons I work with
are hesitant to accept surgical interpretation based on the numerical data
and the engineers are hesitant to provide it. At this point, for our
research group, the greatest value numerical tests have are to better
understand implant behavior under a wide variety of testing
scenarios/positions/materials. These data may be used to better understand
the clinical situation, but that is after the fact. Anyway, that is where
our group stands on the issue.

I look forward to your summary.

Andrew
----------------------------------------------------------------------------------------------------
From: "Warren G. Darling"

Dear Dr. Viceconti, I would say that a model must
predict results of in vivo experiments (in a
human or in a close animal model) before it
should be used to draw any clinically relevant
conclusions. In vitro experiments often produce
quite different results from in vivo experiments.
I definitely do not think it is sufficient to
simply show a model is numerically accurate and
that model assumptions and parameters are well
supported by experimental observations. A
numerical model with valid assumptions and
parameters can be used as a guide to design
experiments to test clinically relevant issues,
but should not guide treatment until its
predictive value has been tested.

Sincerely,
Warren Darling
----------------------------------------------------------------------------------------------------
From: Bjorn.Olsen@MEMcenter.unibe.ch



Dear Dr. Viceconti,



Like yourself, I am often faced with the same
questions regarding numerical simulation; from
both surgeons and editors.

Like many on this list I have used various
numerical models in biomechanics, and I find it
useful to answer specific research questions.
However, in my opinion numerical modeling is
*just* a method. There is nothing special about
numerical simulation, and the manner in which
this particular method is employed must be
scrutinized like any other (eg. experimental)
method. Here I think the biomechanics community
has failed to some degree. When compared to most
other engineering disciplines, numerical
simulation is in its infancy when employed in
bio-engineering. The range of published
(modeling) material range from absolutely cutting
edge, to the - to say the least - not so great.
Very often, in the papers I review, I find a lot
of *very* basic mistakes (boundary conditions
applied incorrectly, linear analysis for large
deformation, material property/units wrong,
etc.). Furthermore, often the models do not
correspond to the problem the authors are trying
to solve.

The rise of numerical simulation over the last 10
years has seen a drop in experimental
biomechanics (which I personally find a bit
disturbing). This is most likely also related to
cost. Simulation is, most often, a cheaper and
quicker solution. Whether or not experimental
methods should always be used depends on the
specific problem at hand. As all of you would
know, quite often simulation is the only method
available to study certain problems in our field.
The engineer should always review the existing
methods (experimental/numerical) and chose the
one which is best suited to the answer the
research question.

It is mistaken to think that numerical analysis
(like FEA) is a simple, out-of-the-box method. It
is not. This is an engineering discipline, and
great care should be taken when using it. My
message is: We, as authors and reviewers, must
ensure that these methods are used correctly.
This is the only way it is possible to maintain
and transfer the credibility of numerical
simulation.

With kind regards,

Björn Olsen
----------------------------------------------------------------------------------------------------
From: "Anders Eriksson"

Dear Marco,
I think you have raised a very important issue, which deserves
an extensive discussion in the ongoing quick
development of numerical modelling and
simulations in biomechanics. My viewpoint, being
primarily in computational mechanics but with
several applications to biomechanical problems,
is that there is often/always a too high
credibility given to numerical, e.g. FEM,
calculations -- "the experiments were verified by
FEM calculations" -- without stating anything
about the used elements: nuermical, geometric,
kinematic and kinetic assumptions.

As you say, numerical simulations are always
based on some underlying assumptions and theory:
the 'model'. They can therefore, at best, be as
good as these theories and assumptions. But, and
this is important, they also have their own sets
of assumptions, approximations and numerical
error sources. These must be kept under strict
control, for any conclusions of practical
importance to be drawn. This is as well known by
people within the computational research areas as
is the fact that physical theories are only the
best possible explanation, within a set of
assumptions, is to the physics people. The
general science philosophical conclusion is
probably that we always have a tendency to
consider everyone else's scientific area as
simple and well-defined, whereas all complexities
occur in our own area.

Regarding your underlying question on the
applicability of numerical simulations, I am,
however, not only pessimistic. But, care is
needed. I think that the basic requirement on a
numerical study, aiming to have application
relevance, is a careful documentation (by you or
someone reliable) of the underlying assumption
and theories used in the modelling. And, this is
perhaps even more the case when black box
software is used. In particular, this is true
when the general FEM software is used for
biological problems, which are very often far off
from the parametric ranges where the basic
numerics have been developed and verified. As
strict theories seldom exist for general
problems, testing of limited setups should always
be documented, as a basis for the larger models.
It is also, as you mention, important to state
the problem in fundamental principles, and not
just base parameters and assumptions on
regressions from very similar experiments. This
done, I think that conclusions drawn from
numerical modelling and simulations can be
helpful in the clinical understanding.

In the same context, I would also like to point
to a common type of problem statement in
engineering, and maybe as useful in other
scientific branches, namely the 'inverse problem'
solving, or perhaps a very systematic 'what-if'
research. This can be used for understanding many
types of behaviour qualitatively and (at least
relatively) quantitatively. Assuming that you
have good knowledge of the basic building blocks
inccluded in your simulation package, you can
very easily vary assumptions and parameters in
the simulation model, to see how results are
affected by these assumptions, thereby allowing
some conclusions to be drawn, when these
simulations are compared to experiments (or
general knowledge of behaviour). This is, but
only when you have the underlying knowledge about
the modelling assumptions, something very
different and much more powerful than regression
of parameters in a fixed basic model, as it
allows a much wider space of assumptions to be
tested. Good basic knowledge and a critical view
on the interpretation of the results are the key
ingredients in this method, and the possibility
for independent critical assessment of
assumptions the strength.

As I said, I think that this area needs an
extensive discussion to avoid that many incorrect
clinical conclusions are drawn, based on bad
numerical models, using irrelevant theories and
incompletely known parameters.

Best regards

Anders Eriksson
----------------------------------------------------------------------------------------------------
From: "linping zhao"


Dr. Venceconti:
You brought a very important topic for
discussing. I am expecting such discussion for a
long time. It seems it's time now. I'd like to
see various responses from people with either
engineering or clinical background or from other
point of views.

Here are more concerns regarding to the validation of a FE model:

If a FE model is validated with a mechanical
model or even a cadaver test, is it clinical
relevant? In what sense and what degree?

When we build up a FE model based upon in vivo
CT/MRI data, is it possible to validate a FE
model in vivo? If so, how? If not, how can we say
the FE model is validated?

If the prediction of a FE model is in agreement
with clinical observation, the model can be
considered as validated in some sense. Is this
statement true? If so, in what sense the model is
validated?

Looking forward to reading more.

Linping Zhao, Ph.D.
Plastic and CranioMaxillofacial Surgery
Shriners Hospitals for Children, Chicago
----------------------------------------------------------------------------------------------------
From: Apache

Here is my comment: I think it would be too rigorous to require
every modeller to do the whole job at once, i.e. to find a parameter
fit to an experimental outcome (identifying all unknown parameters
of the numerical model), use the predictive power of the model by parameter
variation and perform an adequate testing experiment.
Now, here, by trying to define what "adequate" means, we find that there
are some prerequisites to be fulfilled in order for a model to increase
the probability of its validity. The probability can never really be 1 (valid)
but 0 (invalid).

1. The parameters of the theoretical model must be independent of time
or experiment (i.e. they may in no way be fitted along time or across
experiments!) AND can be mapped to physical / physiological quantities
which can at least in principle be determined experimentally.

2. The more parameters the model has (in the fitting process) the lower is
its value for identifying fundamental, underlying principles.
In my opinion this statement should be equivalent to saying that for every
structure or phenomenon that can be identified in nature you only need
a finite number of essential parameters describing it. It's just a personal
belief that this number is always small enough to get the chance
of understanding. It is the art of research to identify these essential
parameters.
In FEM models there are heaps of parameters that additionally may be
hidden to the normal user. I.e. even requiring tests of sensitivity
with respect to these parameters (physical or numerical) seems to be
an endless job. I think providers of these software packages should be
urged much more by the scientific community to pass fundamental physical
tests e.g. such as conservation of energy or of moment of momentum in simple
test models that are claimed to be conservative or free of external
forces / torques.

3. More generally: The lower the ratio between the number of fitted parameters
and the number of independent experimental findings that may be used for the
fitting procedure the better the model meaning that the probability
of the model to have predictive power is higher.
The lower the number of model parameters the higher its potential for
gaining insight. Thus, the effort of researches to REDUCE the number of
parameters should be favoured by referees over the apparent success
to provide a perfect multi-parameter fit. Here, it is very useful to
have an estimation of the accuracy of the EXPERIMENTAL findings. Simpler
models (and there potential of gaining deep insight) are discredited
by an inappropriate trust into measured data.

4. Additionally to 2. good referees ask the researches for the sensitivity of
their models with respect to predictions. Here, 'sensitivity' means that
the FITTED parameters should be varied and the deviation of the results
with respect to the a priori experiments should be shown. On the other hand,
a good referee also identifies very few but CRITICAL parameters but does
not make the researcher do a complete new study.

> How do we translate these general rules in the specific of our
> research domain, biomechanics? Can we say that once a model predicts
> with sufficient accuracy the results of an in vitro experiment, we
> can consider it valid and use it to draw clinically relevant
> conclusions?
I tried to speak out explicitly some criteria (1. to 4. above) that I
find essential when dealing with the interplay of theory and experiment.
I personally 'love' extremely reduced models more and more. My very personal
view is that an FEM-model can be considered a little bit more valid if it has
passed one predictive test. Full validity in biomechanics can not be shown
within one study as long as the MINIMAL number of parameters that might be
necessary for predicting underlying processes is not at least estimated.
If one compares to the many years that e.g. the
spring-mass model for locomotion
was and still is tested and developed around the world one can get a feeling
for the long way to high validity of a model. I personally would not draw
clinically relevant conclusions based on an FEM-model before knowing many
things about the set of parameters, sensitivities etc. (see above).

> Or is it sufficient to prove that the model is
> numerically accurate and all the model assumptions and parameters are
> well supported by experimental observations?
As a summary: I think, validity can only be given in terms of probability.
Unfortunately, one can only prove a model to be INVALID if it clearly fails
with a predection. Otherwise, we can only check accuracies, minimize the number
of parameters and do as many tests (some mentioned in 1. to 4.) as possible to
enhance the probability of model validity.

Cheers,
Michael.

--
Michael Gunther
----------------------------------------------------------------------------------------------------
From: "Ben Thacker"

Marco,

The points you raise are exactly those that must
be addressed if numerical simulation is
to play an active role in design, i.e., produce
credible predictions with quantified
accuracy. Requiring this of numerical models
means the stakes have been raised -- over the
next few years it will be interesting (and
exciting) to see if the modeling community can
address these questions to the satisfaction of
the decision makers and customers. It will
not be easy.

I'd like to refer you to our ASME V&V Standards Committee website at

www.usacm.org/vnvcsm

There are several reports there, etc. that may be
of interest to you. You can also
subscribe to our email list from the web site.

I hope this helps.

--Ben
----------------------------------------------------------------------------------------------------
From: Ton van den Bogert

Dear Marco,

This is a great question, and very important.

I had hoped for some public replies, maybe when
you post your summary you can ask if people want
to post additional public responses.

>Or, more appropriately, what are the conditions a
>numerical model must fulfill in order to considered so reliable that
>we can reasonably use it to draw clinically relevant conclusions?

It depends, I suppose. If a model depends only
on known laws of physics, and not on assumptions,
one should have no problem with clinical
conclusions. But usually there are assumptions
and/or simplifications, and this is where your
question applies. Here are my ideas on how to
alleviate such concerns.

First, it is often not known to what extent the
results depend on assumptions. Usually it is
possible to do sensitivity analysis or
perturbation analysis. Simply change the assumption and see how much
difference it makes. If there is any question on the influence of
assumptions, such a sensitivity analysis should always be attempted.

Second, computatonal studies often use just one model, and if you think
of it, this is absurd. You would never use just one human subject, one
animal, or one cadaver specimen. There is biological variation among
humans and when a model represents a sample of N=1, conclusions may be
quite unreliable. Why not make a group of models to reflect variability
within the human population, do the model study
with all of them, and analyze the results
statistically? This also takes care of the
sensitivity analysis to some extent. If a result
is obtained with one model, and not with another,
based on a different human subject, the result is
not reproducible and should not be reported. If
you have N=1, how do you know?

Computers get faster all the time. N=1 with no sensitivity analysis may
have been OK ten years ago. Now, computers are 20 times faster, so we
can do N=10 and additionally use one of those models to quantify ten
sensitivities. But instead, we have built models that take longer and
longer to compute and are supposedly more realistic. But still we often
do N=1 and no sensitivity analysis. I think this is not the right way
to proceed.

This N=1 practice may be more of a problem in my
own field of movement dynamics than in tissue
mechanics. In movement, there is nonlinear
dynamics with potential for self-regulating
mechanisms. See Wright et al (Clin Biomech,
1998) for a nice example. The question was the
effect of shoe hardness on impact forces in
running. Ten models were used. In some, the
harder shoe increased the impact, in others it
decreased the impact. This was unexpected and
forced us to examine the results carefully. A
self-regulating mechanism was found: a harder
shoe causes the impact force to rise faster,
which causes earlier knee flexion, which then
slows the further rise in impact force. Whether
the peak force ends up being higher or lower than
with the other shoe, depends on subtle
differences between the models. This is an
example where N=1 would have been a lottery,
rather than a scientific study. With ten models,
the statistically correct conclusion was
obtained: no significant effect, but also there
was an understanding of the underlying mechanism.

There is also the problem of quality control. If
there is a bug in your code, multiple models and
sensitivity analyses will all be influenced in
the same way. You may think that results are
consistent, and therefore correct. This is where
commercial software has the advantage of a wide
user base, and a better likelihood that errors
have already been found. For those of us who
want to go beyond commercial tools and do our own
programming, sound software engineering practices
and a healthy dose of self-criticism are
essential.

Finally, models are mainly useful in basic research. We can use a model
to generate and test hypotheses, but before
giving clinical recommendations, I think that a
clinical trial should be done to see if
conclusions are also valid in the real world.
This can be expensive, but it is nevertheless the
standard procedure for drugs, after doing animal
studies. For the same reason it may be a good
idea in biomechanics also, after numerical
studies have been done as a first test of
efficacy and safety.

It seems that in biomechanics we are not always
as rigorous as in other disciplines, perhaps
because biomechanics usually does not deal with
life-threatening problems. By the way, the
problem is not limited to numerical models. In
vitro models are heavily used in orthopaedic
biomechanics, to study a joint or bone in isolation. Results can depend
very much on the mechanical boundary conditions, and this is not always
sufficiently recognized.

Ton van den Bogert

--
--------------------------------------------------
MARCO VICECONTI, PhD (viceconti@tecno.ior.it)
Laboratorio di Tecnologia Medica tel. 39-051-6366865
Istituti Ortopedici Rizzoli fax. 39-051-6366863
via di barbiano 1/10, 40136 - Bologna, Italy

Tiger! Tiger! Burning bright in the forest of the night,
what immortal hand or eye could frame thy fearful symmetry?
--------------------------------------------------
Opinions expressed here do not necessarily reflect those of my employer

-----------------------------------------------------------------
To unsubscribe send SIGNOFF BIOMCH-L to LISTSERV@nic.surfnet.nl
For information and archives: http://isb.ri.ccf.org/biomch-l
Please consider posting your message to the Biomch-L Web-based
Discussion Forum: http://movement-analysis.com/biomch_l
-----------------------------------------------------------------