Coefficient of Multiple Determination/Correlation (CMD/CMC) Troubleshooting

[Amanda Stone]

Hello Biomechanists,

I am trying to calculate coefficient of multiple determination (CMD) and correlation (CMC) but I keep getting negative numbers when values should fall between 0 and 1. I have based my calculations off of the original Kadaba 1989 article (attached) and it seems to me my code follows the equations provided in the article, however I still get some negative numbers for CMD. Does anyone have experience with CMD and CMC calculations? I am happy to provide sample data and code.

Amanda Stone
stonaman@ufl.edu

[Allan Carman]

Re: Coefficient of Multiple Determination/Correlation (CMD/CMC) Troubleshooting

Amanda,

The CMC and the adjusted CMC (aCMC, Cadaba 1989) are not the same. Worked examples of CMC, aCMC, sampling error and the #CMC with more detailed explanations are given the excel file.

CMC #CMC workings.xls

https://drive.google.com/drive/folders/1W9SxnZO5WK7J-Nqfo8MCFSJtH5vx225p?usp=sharing

The file starts with the basic calculation of CMC and aCMC. Next, sampling error, the effect on CMC and its correction in the #CMC. Followed by estimating sampling and #CMC from a sample of unknown population size. Finally an application to knee abd/add in gait.

The main points:

• CMC and the adjusted CMC (aCMC, Cadaba 1989) are not the same.
• aCMC does not give a correlation coefficient between multiple trials and should not be described as such.
• The gait reliability literature often uses the aCMC to describe the reliability of multiple gait curves but incorrectly presents it as a correlation coefficient or as the CMC.
• aCMC substantially reduces or penalizes the CMC depending on the magnitude of the original CMC, subjects or frames (N) and trials (T).

The smaller the CMC magnitude the larger the reduction in aCMC
The greater the number of subjects or time points (N) the larger the reduction in aCMC
The greater the number of trials (T) the smaller the reduction in aCMC.

• aCMD can be negative and aCMC undefined. For gait data with two trials per subject normalized to 100 time points will result in a negative aCMC if CMC < 0.5. See excel file for calculation.
• The CMC calculated from a sample of 2 trials per subject suffers from a sampling error that under estimates SSTotal. This overestimates CMC when compared to the CMC obtained when a large sample size or the whole population of trials is used. As each set of trials for a given subject or examiner suffers from sampling error, pooling results across multiple subjects or examiners does not reduce this error.
• Increasing the sample size (3, 4, 5… trials per subject) will successively decrease sampling error, reducing error in CMC to converge to the CMC obtained if a large or total population of trials were used.
• The aCMC will substantially underestimate CMC values, no matter how many trials are used.
• The aCMC will not converge to the CMC when larger numbers or the total population of trials is used.
• CMC can be corrected for sampling error, the #CMC. This will give an estimation of CMC obtained if a larger number or population of trials had been used. However this requires a good estimation of SSE(sampling), see worked examples in excel file.

Recommendations:

• Use the CMC, but be aware of the sampling error and overestimation of CMC that comes from a small number of trials (sample of size S).
• A corrected #CMC is preferred which accounts for sampling error when drawing a small sample from a larger, usually unknown, pool of trials (see Table 1)
  
• Use three or more trials instead of two trials (days or examiners). It will substantial improve estimations and reduce variability in CMC, estimation of SSE(sampling) and #CMC.
• If aCMC is presented then it should be clearly labelled as such and should not be presented as the CMC or as a correlation coefficient.
  

Table 1) CMC, aCMC and #CMC calculated from varying numbers (sets) of trials drawn from a larger pool of 5 trials
CMC, aCMC and #CMC calculated from 10 sets of 2 trials, 10 sets of 3 trials, 5 sets of 4 and 1 set of all 5 trials (CMC = 0.840).

	S = 2	S = 3	S = 4
CMC	0.902	0.867	0.850
aCMC	0.795	0.797	0.797
#CMC	0.840	0.840	0.840

Allan