Interobserver Agreement Scores

Another variant of interval reliability is proportional reliability (Bailey-Burch, 2002). For the first time described by Bostow and Bailey (1969), proportional reliability includes the division of the smaller frequency by the largest at each observation interval and the average of the resulting fractions over all intervals. Since proportional reliability is based on the number of recorded responses, and not only if a response has been recorded, this is a stricter measure than the reliability of the interval. At the same time, it allows “partial credits” for each response recorded in the meantime, while the reliability of the accuracy does not satisfy. Thus, proportional reliability seems to strike a balance between the less stringent interval method and the stricter method of exact tuning. Background assessment for each observer for low, medium and high responses during meetings with a single response point (left) and during meetings where a second medium-rate reaction occurred (right) in study 1. This article reports on inter- and intra-observe reliability tests of the Coimbra method for entheseal fibrocartilaginious modifications and discusses the results in the context of reliability for other qualitative methods that assess skeletal variations. Four interobserver reliability tests and an intraobserver reliability test were conducted. Three of the Interobserver tests were conducted in person (Geneva, Coimbra A, Coimbra B) and one online. Before each test, the observers trained together and refined the method. Reliability between and within the observer was assessed using a percentage of match, Kappa scores and Krippendorffs Alpha scores. Online tests had the lowest reliability (60% match, n -0.292, α – 0.369).

The final results of the Coimbra B trial increased from the first Test in Geneva (68.6 to 80.0% match, n . . . 0.522 to 0.589, α – 0.604 to 0.666). Factors such as training, observation conditions, visual acuity and differences in interpretation of the evaluation criteria have an influence on reliability. Proposals are being made to address the general problem of the reliability of qualitative methods. Mudford et al. (2009) compares exact and proportional reliability (described as “block by block”) with time slot analysis, where the recordings of both observers contain an inside response ±t. Twelve observers recorded data from six video samples of clientelistic and therapeutic interactions and focused on a target response at each session, which changed in response rate (three samples) or duration (three samples). Response rates were 4.8, 11.3 and 23.5 per minute for responses at a low, medium and high rate. The results showed that the exact and proportionate reliability of the low-interest reaction was similar (Ms- 78.3% and 85.3% respectively). However, the reliability of the agreement`s accuracy was significantly less than the proportional reliability of the average interest rate (Ms – 59.5% and 76.8% respectively).

and high responses (Ms – 50.3% and 88%). These results suggest that reliability calculations are influenced by the rate of a response variable, but they did not determine whether lower results of exact matching were a function of the response rate per se or another high-rate reaction characteristic, such as. B the periodic glow. Repp, A.C., Deitz, D. E., Boles, S.M., Deitz, S.M., and Repp, C. F. The differences between the methods of calculating the Interobserver agreement. Journal of Applied Behavior Analysis 1976,9, 109-113 Figure 2 shows individual matching results for each of the six types of meetings, and Table 1 summarizes the results as average percentages for observers.