Development of the COSMIN checklist
The COSMIN checklist was developed in an international Delphi study. The aims of this study were:
A Delphi study was performed consisting of four written rounds in 2006-2007. Ninety-one international experts were invited to participate in a Delphi panel, of whom 57 (63%) agreed to participate. These were experts in the field of psychology, epidemiology, statistics and clinical medicine. People from different parts of the world participated.
The names of all panel members who completed at least one round can be found here.
As preparation for the Delphi study a literature search was performed to determine how measurement properties are generally defined and evaluated. A systematic review was performed to identify all systematic reviews of measurement properties (open access). Additional searches were performed to identify relevant methodological articles and textbooks that presented standards for evaluating measurement properties.
Each Delphi round consisted of a series of questions. Questions were asked about which measurement properties should be included when evaluating HR-PROs, how they should be called and defined, and how they relate to each other in a taxonomy. In addition, questions were asked about design requirements and preferred statistical methods for assessing the measurement properties. Preferred statistical methods were asked separately for studies using Classical Test Theory (CTT) and Item Response Theory (IRT). The results of previous rounds were presented in a feedback report, containing all results of the previous round, including arguments provided by the panel members. Consensus was considered to be reached when at least 67% of the panel members agreed with a proposal.
Consensus was reached on terminology (74% to 88%, except for structural validity (56%)), and definitions of measurement properties (68% to 88%) and on the position of each measurement property in the COSMIN taxonomy (68% to 84%). Consensus was also reached on design requirements (68-94%) and preferred statistical methods (68-100%).
The results of the consensus reached in the Delphi rounds were used to construct the COSMIN checklist. The COSMIN checklist consists of 12 boxes. Ten boxes can be used to assess whether a study meets the standard for good methodological quality. Nine boxes are used to evaluate the quality of the assessment of different measurement properties: internal consistency, reliability, measurement error, content validity (including face validity), construct validity (subdivided into three aspects, i.e., structural validity, hypotheses testing, and cross-cultural validity, criterion validity, and responsiveness (boxes A to I). One box is used to evaluate the quality of a study on interpretability of a HR-PRO (box J). Interpretability is not considered a measurement property, but nevertheless an important characteristic of a HR-PRO instrument and is therefore included in the checklist. In addition, two boxes are included in the checklist that contain general requirements for articles in which IRT methods are applied (IRT box), and general requirements for the generalisability of the results (Generalisability box), respectively.
The following persons have participated as a panel member in the COSMIN study:
Neil Aaronson, Linda Abetz, Elena Andresen, Dorcas Beaton, Martijn Berger, Giorgio Bertolotti, Monika Bullinger, David Cella, Joost Dekker, Dominique Dubois, Arne Evers, Diane Fairclough, David Feeny, Raymond Fitzpatrick, Andrew Garratt, Francis Guillemin, Dennis Hart, Graeme Hawthorne, Ron Hays, Elizabeth Juniper, Robert Kane, Donna Lamping, Marissa Lassere, Matthew Liang, Kathleen Lohr, Patrick Marquis, Chris McCarthy, Elaine McColl, Ian McDowell, Don Mellenbergh, Mauro Niero, Geoffrey Norman, Manoj Pandey, Luis Rajmil, Bryce Reeve, Dennis Revicki, Margaret Rothman, Mirjam Sprangers, David Streiner, Gerold Stucki, Giulio Vidotto, Sharon Wood-Dauphinee, Albert Wu.
The COSMIN Delphi study was financially supported by the EMGO Institute for Health and Care Research of the VU University Medical Center, Amsterdam, and the Anna Foundation, Leiden, the Netherlands.
The development of the COSMIN checklist has been published in:
Mokkink LB, Terwee CB, Patrick DL, Alonso J, Stratford PW, Knol DL, Bouter LM, de Vet HCW. The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments. Quality of Life Research 2010;19:539-549. (open access).