The Problem With Asking Observers to Memorize a Rubric

Ask an experienced administrator about the district's teacher evaluation rubric, and chances are they know it well. The domains are familiar. The indicators have been discussed in training. The performance levels have been reviewed, practiced, and probably revisited more than once.

Then the administrator walks into a classroom.

Suddenly, knowing the rubric is only part of the job. The observer listens to teacher questions, watches how students respond, notices who is participating, looks for evidence of understanding, follows transitions, considers differentiation, and pays attention to dozens of other things happening in real time.

At the same time, the observer is expected to connect all of that activity to a complex professional framework (e.g., Danielson, 2013; Marzano & Toth, 2013; LoTi Connection, n.d.).

That's a lot to ask someone to hold in their head at once.

Knowing the Rubric Isn't the Same as Seeing the Practice

Observer training often focuses, understandably, on the evaluation instrument. Administrators study indicators, discuss performance levels, examine sample lessons, and practice calibration.

That knowledge matters. But classroom observation requires something more.

Research on classroom observation has found that raters can struggle to keep multiple dimensions of instructional quality in mind simultaneously (Cohen and Goldhaber, 2016), and it also points to the observer as a key potential source of variation in observation scores, even when established observation instruments are used.

That makes sense when we consider what an observer is being asked to do. A rubric may contain numerous indicators, each describing a different aspect of effective practice. Meanwhile, teaching isn't unfolding one indicator at a time.

A student makes an unexpected connection. The teacher changes a question. A group begins solving a problem in a way the teacher hadn't anticipated. Another student struggles and receives additional support. The observer has seconds—not minutes—to notice what happened and decide whether it matters.

The Classroom Doesn't Follow the Order of the Rubric

Rubrics are organized because they need to be. Classrooms aren't.

Evidence of questioning might appear during a discussion and again during independent work. Differentiation may become visible in the resources students use, the questions a teacher asks, or the choices students are given. Evidence of engagement may appear in what students say and not simply in whether they appear attentive.

None of those moments announces itself by saying, This is evidence for Indicator 2.3. The observer has to recognize it.

That's where asking someone to simply “know the rubric” can fall short. The challenge isn't recalling an indicator's wording. It's recognizing the evidence of that practice while instruction is happening.

Research reinforces why that distinction matters. Studies of classroom observation have documented differences in how raters interpret and score teaching practice, including variation in scoring severity and changes in how raters apply observation criteria over time (Casabianca et al., 2015; Jones & Bergin, 2019; White, 2018). These findings suggest that even trained observers can introduce variation in observation scores, reinforcing the importance of helping observers develop a consistent understanding of the evidence they see in classrooms. 

More memorization isn't necessarily the answer.

Give Observers Something More Useful to Look For

What if observer preparation started with a different question?

Instead of asking only, “Do our observers know the rubric?” we could also ask:

“Do our observers know what evidence of these practices looks and sounds like in a classroom?”

That's an important shift.

It doesn't make the rubric less important. And it doesn't replace professional judgment. It gives observers a clearer bridge between the professional expectations described in the rubric and the classroom practice unfolding in front of them.

That is part of the thinking behind Evidence-First™. Rather than requiring organizations to replace the frameworks they already use, Evidence-First strengthens the connection between those frameworks and observable classroom evidence. The goal is to help observers recognize meaningful evidence, connect it to professional practice, and make more informed judgments about what they observed.

For schools, districts, and teacher preparation programs, that means observer development can become about more than remembering what the rubric says. It can focus on helping observers recognize what the rubric looks like in practice.

References

Casabianca, J. M., Lockwood, J. R., & McCaffrey, D. F. (2015). Trends in classroom observation scores. Educational and Psychological Measurement, 75(2), 311–337. https://doi.org/10.1177/0013164414539163

Cohen, J., & Goldhaber, D. (2016). Building a more complete understanding of teacher evaluation using classroom observations. Educational Researcher, 45(6), 378–387. https://doi.org/10.3102/0013189X16659442

Danielson, C. (2013). The framework for teaching evaluation instrument. The Danielson Group. 

Jones, E. T., & Bergin, C. (2019). Evaluating teacher effectiveness using classroom observations: A Rasch analysis of the rater effects of principals. Educational Assessment, 24(2), 91–118. https://doi.org/10.1080/10627197.2018.1564272

LoTi Connection. (n.d.). Transform teacher evaluation with Evidence-First™. https://www.loticonnection.com/ 

Marzano, R. J., & Toth, M. D. (2013). Teacher evaluation that makes a difference: A new model for teacher growth and student achievement. ASCD.

White, M. C. (2018). Rater performance standards for classroom observation instruments. Educational Researcher, 47(8), 492–501. https://doi.org/10.3102/0013189X18785623