The Toolbox of Assessment Methods

OVERVIEW

[Note: The following material is based in part on an introduction to the Outcome Project found on the ACGME Website at " www.acgme.org/outcome/project/proHome.asp ".]

The second key aspect of the ACGME General Competencies requirements and the second major component of the Outcome Project is the identification and development of tools to measure the progress of residents and programs in meeting the objectives defined. The "evaluation" clause of the requirements states that "t he residency program must demonstrate that it has an effective plan for assessing resident performance throughout the program and for utilizing assessment results to improve resident performance" and furthermore that "the residency program should have in place a process for using resident and performance assessment results together with other program evaluation results to improve the residency program." Toward this end, the ACGME's Susan Swing collaborated with the American Board of Medical Specialties' (ABMS) Phillip Bashook to develop an inventory of descriptions and examples of instruments that programs could use to assess the outcomes of their educational efforts. This inventory, known as the Assessment Toolbox, includes:

• The results of an extensive survey of residency evaluation practices and resources conducted by the ACGME
• The results of an extensive review of current literature on assessment in residency programs
• The results of inquiry into "best practices" in evaluation of residency programs
• Input from the Outcome Project Advisory Group, and
• Input from programs and experts in residency program evaluation

Included in the Toolbox of Assessment Methods are brief descriptions of 13 assessment methods and references to articles where more complete and in-depth information about each method can be found. The Toolbox will be revised periodically to reflect the ongoing work of the joint initiative between the ACGME and the ABMS. The list of methods currently includes:

• 360-Degree Evaluation Instrument
• Chart Stimulated Recall Oral Examination (CSR)
• Checklist Evaluation of Live or Recorded Performance
• Global Rating of Live or Recorded Performance
• Objective Structured Clinical Examination (OSCE)
• Procedure, Operative, or Case Logs
• Patient Surveys
• Portfolios
• Record Review
• Simulations and Models
• Standardized Oral Examination
• Standardized Patient Examination (SP)
• Written Examination (MCQ)

Some of the tools are more relevant to certain of the General Competencies and some to others. During the course of your residency, you are likely to encounter a number of these various types of instruments and methods. A well-developed program will combine the use of several methodologies to form a multi-dimensional picture of the program's effectiveness and a resident's competencies.

To review the Toolbox itself and learn more about the assessment methods listed click the following link:

http://www.acgme.org/outcome/assess/toolbox.asp