Measuring Agreement In Method Comparison Studies. Stat Methods Med Res

For M different methods, we have M!/2! (M – 2)! different comparisons in pairs. However, in the absence of negligible data or data for any of the different methods, there are linear dependencies in these differences on all kinds of comparisons of different pairs and only M-1 comparisons are needed to determine the rest. For example, if we have three methods J, R and S, then we have 3!/2! (3 – 2)! 3 different pairs of comparisons, namely: J -R, J-S and R-S. For example, the R-S pair comparison depends on the J-R and J-S pair comparisons and therefore does not need to be explicitly modeled. If the three pair comparisons were simulated numerically simultaneously, convergence could be expected to be poor and selfcorrelation to be high in the relevant parameters. Where there is significant data available for one of the methods, it is important to ensure that paired comparisons can be modelled and related dependencies taken into account. If a method systematically provides missing data that is too negligible compared to other methods, that would probably be reason enough to call into question the usefulness of this method. D.G.A. had encountered a similar problem in a study of variation in changes related to the coil and knee. The publication of this study contained a brief footnote on this subject: “It is wrong to use the correlation coefficient to compare sets of measurements of the same variables. Under these conditions, the correlation largely reflects the variability of the measured subjects. For our least reliable measurement at 15 cm above the Patella, the correlation between the measurements of two observers was 0.99. These are the differences between the measures that should be considered.

3 To this end, D.G.A. reviewed methodological comparative studies in an article in the British Medical Journal and stressed the importance of examining differences between methods and non-correlations.4 In 2004, Carstensen described more general methods of regression and variance components for the analysis of these data [8]. Although these methods are conceptual, they can be difficult to implement, which limits their usefulness. The latest energies of Carstensen and colleagues have been to signal simplified versions of his methods and to develop new techniques with more practical utility [11].