MerQuR+Statistics Glossary

Statistics Glossary → Absolute agreement and consistency (ICC types)

Absolute agreement and consistency (ICC types)concept

Two definitions of the intraclass correlation coefficient: one measuring whether raters give the same values (absolute agreement), the other whether they merely rank subjects in the same order with the same spacing (consistency).

The ICC is the share of total variance due to genuine differences between subjects. Under the consistency definition, systematic differences between raters (for example, one rater giving everyone 5 points more) are not counted as error; only the consistency of the subjects' relative positions matters. Under the absolute-agreement definition, that systematic difference is added to the denominator as well, so the ICC falls unless the raters give the same numbers. For this reason the absolute-agreement ICC is in practice equal to or smaller than the consistency ICC.

The choice depends on the research question: if the measurements must be interchangeable (e.g. one device will replace the other), absolute agreement is appropriate; if only ranking and relative differences matter (e.g. scores will be standardized separately for each rater), consistency is. When reporting an ICC, the model (one-way, two-way random, two-way mixed), the type (absolute agreement or consistency) and the unit (single or average measures) should all be stated; McGraw and Wong (1996) describe this classification in detail.

An everyday example

Two scales weigh the same ten people. The second scale shows everyone exactly 2 kg heavier, but the order of the people and the differences between them are identical. The consistency ICC comes out very close to 1, because the scales agree completely about who is heavier. The absolute-agreement ICC is lower, because the reading from one scale cannot be used in place of the other.

In MerQur

It is chosen with the 'ICC type (Absolute agreement, Consistency)' parameter in the Intraclass Correlation Coefficient (ICC) analysis.

Analyses where this concept appears

Intraclass Correlation Coefficient (ICC)

Related concepts

Intraclass correlation (ICC)Spearman-Brown formula (single and average measures)