Part 1: Best practices in evaluation design | Chalkboard Project

Part 1: Best practices in evaluation design

Wednesday, October 3, 2012 Melissa Tooley

Thinking back on our own experiences as students, I think most of us would agree that our strongest teachers had a powerful and lasting impact on us. At The Education Trust, we regularly cite research showing that teachers are the biggest in-school factor influencing student achievement. Highly effective teachers are especially important for struggling students: Multiple studies have found that a series of strong teachers can help previously low-achieving students soar academically, while a series of weak ones can lead such students to stagnate. Informed by this research, many states, including Oregon, have begun to create more rigorous systems for measuring teachers’ effectiveness. Such systems can help teachers improve their practice, while helping schools and districts make smarter staffing decisions.

Design
Strong evaluation systems start with a strong design. They also rely on several different pieces of information about each teacher—never just one test score or a single observation—to provide for a more complete picture from which to identify teachers’ strengths and areas for growth. Teacher assessment measures do not need to be exactly the same across districts; however, good systems will share certain vital measures: detailed classroom observations and robust measures of teacher impact on student learning.

Observations
Today, most evaluation systems are built entirely upon one or two “drive-by” observations, often based on a checklist of criteria that are not clearly connected to student learning. Worse, they rarely result in meaningful feedback that teachers can actually use to improve instruction. Classroom observations remain a key element of well-designed evaluation systems but look quite different from these traditional observations: strong observation frameworks are research-based and offer consistent expectations as well as a common language for instructional practice across schools and districts. They also lead to meaningful follow-up conversations and useful feedback that result in clear, measurable goals for the teacher.

In conjunction with observation data, a measure of student learning growth helps teachers (and their supervisors) know whether what they are doing in their classrooms is positively impacting their students’ knowledge and capabilities — which is why most teachers chose this profession in the first place.

Student Growth Measures
For state-tested grades and subjects, student growth measures should be consistent across all districts so that statewide comparisons are possible. And for teachers in non-tested grades and subjects, these measures should at least be comparable within each district. Some student growth measures, such as value-added, can take into account students’ prior performance so that teachers aren’t punished for students who come into their classroom behind, as long as the students make progress consistent with how they and similar students have progressed in the past. And while these measures aren’t perfect, they are much more meaningful than traditional evaluation measures, which haven’t distinguished different levels of teacher effectiveness.

Performance Levels
Speaking of different levels of effectiveness, another key design element is having (and using) more than two performance level designations. Systems with at least four performance levels allow supervisors to see who needs the most support and who may be able to support others in a mentoring or collaborating role. Finally, all educators should be evaluated annually so that they receive timely and consistent feedback that helps them develop and perform to their fullest ability.

Oregon Framework
Oregon’s new teaching standards aim to inform the performance evaluation of its teachers, which, by state law, must take into account multiple measures of teacher effectiveness, including evidence of student learning and observations. Oregon has also developed a framework to guide districts in developing new local evaluation systems that meet these guidelines. These are steps in the right direction, but many details are still unclear or in development—and in working them out, it may at times be tempting to pull back or take the easiest available path. For example, districts may choose a measure of student progress that fails to differentiate between the actual levels of student academic growth being realized. Or they may opt to include a student progress measure as only a minute portion of a teacher’s overall evaluation rating. Oregon should allow districts the flexibility to design systems that reflect their local context without straying from the original intent of the law.

Benefits for Teachers and Students
Designed well, new educator performance evaluation systems could be a big win for Oregon’s students and educators. A seminal study by TNTP found that only 42 percent of teachers agree that traditional evaluations allow an accurate assessment of performance. Like other professionals, teachers and school leaders deserve adequate feedback on their performance. And students, especially those who start off behind, deserve effective teachers who can help them learn and grow.

But let’s remember that a high-quality design is just the start of a good evaluation system. How state, district, and school administrators implement and use results from these systems ultimately will determine how much teaching and learning improves. In the coming weeks, I’ll offer recommendations for how states and districts can make sure these critical areas also are strong.  See you then.

 

Melissa Tooley is a teacher quality and data analyst at The Education Trust. Melissa’s work at the Ed Trust focuses on evaluating and influencing policy to ensure that all students receive the effective teachers they need and deserve. This is part one in a series of blog posts discussing educator evaluations.

  • Evaluation

Chalkboard on Twitter

Follow Us for Useful Information About Education in Oregon