WHAT does calibration look like?

The Steps of the Calibration Process

The ideal way to use the calibration process is through a structured protocol as a part of a PLC that meets regularly, so that you can build trust and collective knowledge over the course of your efforts, and everyone gets a chance to learn from each other's work and feedback.  The exact format and tools used to support the calibration process can vary slightly depending on the group.  For example, a small team of 4 teachers working together with common planning time will most likely operate a bit differently than a team of 12 teachers.  No matter what your group looks like, there are strategies that can support your team in doing this important work.   

The calibration process typically runs as follows:

Step 1: Prework

The group identifies a "presenter", who is the teacher that has brought work to calibrate. The presenter brings sufficient copies of the prompt/task, the task rubric, and student work for a full class of learners, or provides a digital folder for all to access. The group determines how they will record scores, either on a score sheet or task rubric, and makes sure there are sufficient copies for the recorder to document the scores for the whole group.  Some teams find that it is helpful to use a digital option to collect scores.  Digital options could include a google document, a google form or OSCAR Classroom which will be introduced in Module 12.  The advantage to a digital option is that it allows the team to quickly identify where scoring is consistent and where work needs to be done to be more aligned.  Either way, the presenter needs to do some planning and create that tool ahead of time.  Usually, this tool can be created using the task rubric. 

Feel free to use the following examples as templates:

1.       This first document would be used during the calibration session itself for individual evaluators to score work from each of the 4 students.

2.       The second document would then be used for each evaluator to share their scores with the team.  

Step 2: The Protocol

The group selects a "facilitator" to guide the team through a calibration protocol (shared in the HOW section of this module).  This model is based on collective evidence that the task aligns to the Design Elements. At the beginning of the protocol, the facilitator overviews the process norms, then gives the group several minutes to do an initial examination of the prompt, student work, rubric, and score sheet. Once group members have familiarized themselves with these materials, they have the chance to ask clarifying questions. Participants then independently score the student work based on the rubric. The facilitator asks group members to share their scores without explaining their rationale while the recorder documents all scores. The group then examines areas where there were differences in scores for rubric criterion to try to understand why those differences occurred. The image below is an example of what this might look like using template above.  The final step of the protocol is a debrief to consolidate key learnings, consider revisions to the task and rubric, and examine implications for instructional practice.

Step 3: Revision

The presenter revises the performance assessment and rubric based on feedback from colleagues and makes any relevant updates to the instructional strategy. 

Want to see it in action?

To see the Calibration Protocol in action, watch Teachers Conducting the Calibration Protocol or feel free to watch a prerecorded session from the December 2020 Colorado Performance Assessment PLC.  You can access the recording here.

Readings About Examining Student Work

• Principles of Scoring Student Work - Guidelines for examining student work. 
• Calibration Uses - Brief overview on why and how to use the calibration process.