AGREE II Instrument

Follow us on Twitter

AGREE A3: Application, Appropriateness, Action

Innovations to Enhance the Capacity of Practice Guidelines to Improve Health and Health Care Systems: AGREE Application, Appropriateness, Action (2009-2013)


The AGREE A3 Project was comprised of two Streams of research:

Stream 1, AGREE Application & Action:
The objective was to develop, execute, and evaluate the impact of two interventions designed to accelerate the capacity of stakeholders to use the AGREE II: i) an on-line training video with a virtual coach and ii) an on-line AGREE II training tool with an interactive appraisal practice exercise activity.

The online training tools can be found here:
The following publications describe the research work and results:

Brouwers, M., Makarski, J., & Levinson, A. (2010). Study protocol A randomized trial to evaluate e-learning interventions designed to improve learner’s performance, satisfaction, and self-efficacy with the AGREE II. Implementation Science, 5(29). Doi:10.1186/1748-5908-5-29.

Brouwers MC., Makarski J., Durocher L., & Levinson A. (2011). E-learning interventions are comparable to user’s manual in a randomized trail of training strategies for the AGREE II.  Implement Sci. 2011; 6:81. Doi: 10.1186/1748-5908-6-81.

Stream 2, AGREE Appropriateness & Action:
For this Stream the objective was to design, test, and refine a measure of practice guideline clinical appropriateness (validity) to serve as a companion tool to the AGREE II.

AGREE II focuses on methodological issues relevant to practice guideline development and reporting but does not address the “clinical validity/appropriateness” of their recommendations, which is an important precursor to enable their uptake (or implementability).   The quality of reported guideline processes (as articulated in AGREE II) may or may not lead to recommendations that are clinically “appropriate” or “valid”. Currently, no tool to toolkit resource exists that adequately addresses the gap by providing guidance to inform the development, reporting, or evaluation of this important concept.

To address this gap, we sought to undertake a systematic review of the literature and to ultimately develop a toolkit for practice guideline developers to facilitate explicit and transparent judgments (and reporting) during guideline development and formulation of guideline recommendations.

A Collaboration of Project Teams
During the course of Stream 2, we formed a collaboration with a KT Canada Research Team who were undertaking a review in a related area (lead by Dr. Onil Bhattacharyya).  Together, we completed a Realist Review of the literature to identify which characteristics intrinsic to practice guidelines impact their uptake in practice, under what circumstances, and why.  Intrinsic characteristics refer to those features of guidelines that can be manipulated by guideline developers (e.g., aspects of formatting, use of action-oriented language, compiling a relevant guideline development group, etc.).  From that Realist Review, we developed a framework entitled, GUideline Implementablity for Decision Excellence Model or GUIDE-M.

GUIDE-M is a comprehensive model comprised of 7 domains of guideline features that can facilitate their uptake and organized under three broad categories of practice guideline development: 1. Developers of Content (Comprehensive, Knowledgeable and Credible, and Competing Interests), 2. Creating Content (Evidence Synthesis and Deliberations and Contextualizations) and 3. Communicating Content (Language and Format).   The model contains information taken from the realist review literature.

The purpose of GUIDE-M is to i) serve as a comprehensive resource of characteristics intrinsic to guidelines that can facilitate their uptake (including the “how” and “why”) and ii) to serve as a resource guideline developers may wish to consult during development activities to perhaps identify and incorporate into their guidelines some characteristics that might optimize their guideline’s implementability.

Our recent publication in BMC Medical Research Methodology describes our mapping process for developing GUIDE-M.

We have completed the validity assessment of GUIDE-M and released the final version. To view the completed model, click here.

Using relevant components of GUIDE-M as a foundation, the AGREE Enterprise has launched a new research project called AGREE-REX (Recommendation EXcellence), which will develop a toolkit to guide the development, reporting, and evaluation of clinically credible practice guideline recommendations. Click here for more information.


Research Team:

AGREE A3 Research Team
Melissa Brouwers (PI), George Browman, Jako Burgers, Francoise Cluzeau, Lisa Durocher (Research Assistant), Beatrice Fervers, Anna Gagliardi, Jeremy Grimshaw, Steve Hanna, Kate Kerkvliet (Research Assistant), Anthony Levinson (Stream 1), Julie Makarski (Project Manager),  Holger Schunemann, Louise Zitzelsberger

KT Canada Team
Onil Bhattacharyya, Monika Kastner, Leigh Hayden, Ananda Chaterjee, Judith Versloot