AGREE II Instrument

Follow us on Twitter

Scoring the AGREE II

A quality score is calculated for each of the six AGREE II domains. The six domain scores are independent and should not be aggregated into a single quality score.

i)    Calculating Domain Scores

Domain scores are calculated by summing up all the scores of the individual items in a domain and by scaling the total as a percentage of the maximum possible score for that domain.

Example:
If 4 appraisers give the following scores for Domain 1 (Scope & Purpose):

Item 1 Item 2 Item 3 Total
Appraiser 1 5 6 6 17
Appraiser 2 6 6 7 19
Appraiser 3 2 4 3 9
Appraiser 4 3 3 2 8
Total 16 19 18 53

 

Maximum possible score = 7 (strongly agree) x 3 (items) x 4 (appraisers) = 84
Minimum possible score = 1 (strongly disagree) x 3 (items) x 4 (appraisers) = 12

The scaled domain score will be:

(Obtained score – Minimum possible score)
(Maximum possible score – Minimum possible score)

(53 – 12) / (84 – 12)  X 100 =
41 / 72    X 100 =    0.5694 x 100 =     57 %

 
If items are not included, appropriate modifications to the calculations of maximum and minimum possible scores are required.

ii)    Interpreting Domain Scores

Although the domain scores are useful for comparing guidelines and will inform whether a guideline should be recommended for use, the Consortium has not set minimum domain scores or patterns of scores across domains to differentiate between high quality and poor quality guidelines.  These decisions should be made by the user and guided by the context in which AGREE II is being used.