How should teacher effectiveness be assessed? The answer to that is simple, and it is the same answer now as it was 10 years ago and 50 years ago: teachers need to be assessed on whether or not their students are learning.

Simple to answer, perhaps, but much harder to do. Or is it? Walk into any classroom and what do you see? Are hands eagerly up eagerly, or are heads resignedly down? Are answers and suppositions flying back and forth, or only spitballs? And is the little girl scribbling furiously in her notebook there in the corner responding with passion to the lesson, or is she simply preparing to pass a note about a birthday party to a friend across the class? Most of us would agree that if children are engaged in their own learning, then they are more likely to be learning. Student engagement is only one observable indicator of whether or not students are learning in a teacher’s class, and there are others. Does the teacher know his subject well and focus the material on state standards? Does he design lessons that address the specific needs of his students? Does he exhibit an understanding of child development, or have a specific goal in mind for the day, the unit, and the year? This is not a complete or well-thought out list, but my point is that when teachers know their subjects, engage their students, and design their work around short and long term goals, then students learn. All of these are characteristics of good teachers that we can observe by watching classes, talking to teachers, and looking at the contexts in which they teach.

Of course the problem that arises is immediately obvious. Supervisor observations seem to vary from class to class, school to school, district to district. That has always been a problem for teachers, and their response has been to build strong due process systems to protect themselves from what they perceive (rightly or wrongly) to be the incompetence, vindictiveness or arbitrariness of their supervisors. But in more recent years the inconsistency in observations has also become an urgent problem for communities because too many kids have been failing for too long. Communities (again rightly or wrongly) blame teachers for the bulk of that failure, and believe that better evaluations – and quicker termination – will lead to more effective teachers overall.

The community’s solution (by which I mean the general national solution) has been to try to fall back on the simplicity of numbers, and thus value-added formulas (VA) have become a popular proxy for assessing student learning in a teacher’s class. These numbers may not be as useful as what we’d learn from direct observation, and certainly they cannot begin to capture the full range of what students learn from a given teacher. Those arguments carry little weight, however, in the policy circles and in the press where these questions are most often debated. The fact is that numbers are appealing, and that there is something terribly appealing about being able to say a teacher ranks in the 76th percentile based on her students’ standardized test scores, and leaving it at that. Never mind that the numbers may be systemically biased, prone to high rates of error, or just plain wrong. They answer to an almost primal urge in us for simple answers. Ultimately, VA may have a place in evaluation systems but it is not the panacea we all seek.

What then, is the solution? Specifically, if teachers believe that the current observation system is arbitrary and capricious, and if VA scores are a poor proxy for student learning, then what improvements need to be made to ensure that a child’s opportunity to learn is not hindered by the quality of his or her teacher?

There are many changes we need (teachers need better support systems for example) but in terms of evaluation, I suggest two:

1. That observers (supervisors, peers, etc) be better trained using consistent teaching standards. That won’t be enough, but it is a start.

1. That perhaps decisions about teacher effectiveness cease to result from information generatede by exactly one source (for example, an immediate supervisor). VA may be a possiblity as an additional (not sole) measure for a limited number of teachers (only a fraction of teachers have students with standardized test scores), if the right protections are in place to guard teachers against the inherent bias and random error of the formulas. (A quick example: NYC currently creates VA values for its teachers based upon the scores of as few as six students over three years. The rates of error with such small samples must be significant, and clearly teachers cannot be evaluated on those results.) But, again, even with protections, these formulas may tell us less about student learning than we think.

Another possibility is to employ the use of an independent observer whose judgment would carry weight in addition to the weight of the immediate supervisor. In some districts this could be developed through a peer system, and in others through a completely external evaluator. In any case, these additional evaluators would assess the learning that is happening in a teacher’s class through direct observation and context (feedback from the teacher, for example). The key to the success of such a program would be the independence of the evaluators as well as their adherence to agreed-upon teaching standards. This independent evaluator would not usurp the supervisory responsibility of the district’s usual evaluator, but he would lend a second eye – a kind of inter-rater reliability – to what the supervisor sees in class.

These two changes (better training to common standards, and the use of a second evaluator) would be good for everybody. They would be good for the community because they would ensure that teachers were meeting the standards they must meet if children are to learn, as well as giving arbitrators a clearer picture of a teacher’s ability at due process hearings. And they would be good for teachers too. Teachers deserve a system in which their livelihoods are not wholly in the hands of a single supervisor who can, in teacher parlance, “get you in the classroom if he wants to.” Supervisors may not abuse their power very often (I hasten to add that the vast majority of supervisors I know are conscientious and sincere), but the fact remains that a single individual has inordinate (omnipotent) power over what his teachers say and do, and that has an alarming ability to change teacher behavior, sometimes for the worse. Right now, due process is the only counterbalance to that power, but few teachers believe due process can protect them from a single administrator’s claim that they can’t teach.

My thoughts about effective teaching crystallized as I contemplated the death of Ted Sizer and Gerald Bracey. Ted was Dean of the Harvard Graduate School of Education when I attended and along with Harold Howe influenced a generation of us to consider learning outcomes as something that couldn’t be measured as precisely as professors like Sandy Jencks and Mike Smith taught us as we analyzed and reanalyzed the data in the Coleman Report. Gerald Bracey kept reminding us that that our choices for measuring outcomes moved us away from public education’s enduring commitment to educating citizens for a democratic society. He also had such a delightful way of noting that various emperors of education expertise had few if any clothes.

So here’s my take on effective teaching. As some really smart people struggle to measure effectiveness as something more than raising achievement scores, I want to share a few stories.

At six, my son Dan could not manage his boundless energy. This led to many frustrating conversations with his kindergarten teacher who viewed him as a problem. Upon meeting his first grade teacher Mrs. B, I steeled myself for the usual diatribe against his big mouth and hyperkinetic activity. Instead Mrs. B smiled and said, “Oh, Mrs. Tompkins I have had many little boys like Daniel. Be patient. By Spring he will be able to sit in his chair and do his work and still be happy and full of energy.” And it was true. Dan’s doctor suggested drugs. My neighbors tried to get Mrs. B fired and then decamped to private schools in Northwest DC because her grammar occasionally contained the African American construction of “to be” verbs. Dan and I revere Mrs. B.

My daughter struggled with the strictures of formal schooling. A voracious reader, a knee jerk challenger of authority, a gifted writer, she now makes her living in communications. She hated math from the first grade on. Mr. S, her algebra teacher, cautioned us not to worry about an occasional C or D. One of the original geeky guys, Mr. S smiled and insisted that she was learning algebra and that was the most important thing. Later when she was a junior having completed two years of algebra that brought her cumulative grade average down, a savvy assistant principal waived the rule about GPA to allow her into college English courses where she excelled in creative writing under a published author professor and stopped talking about dropping out of high school.

Over the past ten years I have had the opportunity to visit rural schools across America. Many of them impressed me but some second grade teachers are at the top of my list. In Colorado, one of them had her class doing a year-long study of the aspen grove outside their window. On the Fall day I visited, they went outside and studied twigs. They were asked to ponder several questions. Would the trees have leaves next year? How could you tell? Students spent time collecting fallen twigs and leaves, came in to study them and write and draw in their journals about what they observed, responded to questions about which books and on line resources might give them more answers to the questions. I have imagined the dinner conversations in homes of seven year olds that evening: What did you do in school today? We pondered. Ah yes, pondering.

A final story is from Anderson Valley in northern California. Here the children of winery owners and the children of those who work in the fields all go to the public schools. The dominant language is Spanish. In the second grade computer lab that I visited, the students were learning to keyboard by typing up favorite recipes they had collected from their visits to the local nursing home. There was a buzz of English and Spanish and all mixtures in between as they contemplated measures and tastes that were unfamiliar. Later that year the recipes would be collected and illustrated in a small cookbook and the final visit of the year would be one in which parents and nursing home staff prepared some of the recipes for a celebration with the families and the elderly. I should mention that most of the nursing home residents were Anglo; most of the students were Latino. I can’t begin to tote up all the learning outcomes not only for the children but for their families and the community.

My only hope as really smart people try to measure effectiveness that they consider that context matters not as an excuse for lack of success in learning but as a rich tapestry on which the best teachers build lessons to last.

At the end of the week, I like to start my day reading every post about the current issue in front of us. Either I have too much time on my hands or I want to hold up a finger and figure out which way the wind is blowing. Let’s hope it’s the latter because today I’m missing some good college football.

From what I read this week about teacher evaluation, three prominent threads seem to be knitting themselves together into one of those cute teacher sweaters we all know and love:

1. Test data will be used for teacher evaluations, though not exclusively.

2. Evaluation instruments will become increasingly complex.

3. Most people are in favor of tough evaluations but few want to think out loud right now about what the implications of tough evaluations might be.

Think about Campbell’s Law and other possible unintended consequences, not to shoot these evals down but to make them more robust before they are taken to scale. Also, before we think about firing too many people, let’s keep future demographics in mind. We’re projected to be losing 1,000,000 teachers in the next five years as Boomers age out of the system. At present, our nation only produces about 1,000,000 new teachers every five years, a third of whom quit after two. Yes, alternative certification programs could fill the gap. But no one has yet created or even proposed an approach that could produce the high number of really good teachers we need – much less how to keep these talented people in the profession long after they start.

---------------------

I was a little disappointed this week that I didn’t see more talk about teacher evaluation methodology. But maybe that’s just too geeky for this group. However, I really do believe that the process by which new teacher evaluation instruments are created, and the guidelines that govern their use, will turn out to be the straw we weave into gold or that breaks the camel’s back.

Here are two thoughts on the matter:

1. IF I WERE CREATING AN EVALUATION INSTRUMENT, I WOULD BASE IT ON REAL TEACHERS AND REAL TEACHING.

Here’s one possible approach:

(1) Find a small number of teachers who have reputations for being successful, and for whom we perhaps have a bit of data to back that up.

(2) Observe them teach on several occasions.

(3) Describe what they do and enhance the descriptions by reviewing videotapes.

(4) Interview all participants and ask them questions about their teaching while reviewing videotapes to learn WHY good teachers do what they do.

(5) Identify key commonalities in their practice and base the language of an evaluation instrument on this evidence. Edit the videos for viewing by those who will be evaluated and to use as future training tools.

(6) Interview teachers one more time to discover their “developmental progression” over the course of their careers. For example, “How did you get as good as you got? In what order did your most important gains occur? And what, if any special events (like certain trainings, for example) catalyzed quick growth?” This information would tell us how best to help struggling teachers progress toward the defined ideal.

I can attest to the fact that a process like this works very well. I can also attest to the fact that not doing some of these things leaves evaluation systems extremely vulnerable to criticism, simple gaming, and the general feeling among those being evaluated that something foreign, random, and punitive is being perpetrated against them.

The other cool thing about this approach is that it’s really easy to improve the evaluation instrument over time by simply adding more successful teachers to the sample.

I’m sure social scientists have a nifty name for this kind of approach but I don’t know what it is. So I’ve always just called it “Finding out what talented people do and how they do it.” Malcolm Gladwell just did something similar in his extremely popular book, “Outliers”. (We need to study outliers in teaching, too, because they represent the true innovators and legitimate geniuses in our system.)

I realize that this approach is not something most organizations will want to bother with. But it sure does work and, given the sensitivity of the issue and what we have at stake, I think we can all afford to be a little bothered. For me, this sort of “epidemiological” approach is well worth the effort because as long as you identify a reasonable sample size of reasonably talented teachers, it can’t fail too badly. Basing an evaluation instrument on the real work of real teachers brings reality into the process. And reality is often the best tool we have for catching what often feels to me like yet another runaway cigarette boat of education reform.

Another thing I like about this is the “local flavor” angle. Assuming that a district is large enough to have enough talented teachers, it could successfully create its own instrument based on its own norms and values. This can obviously backfire for many reasons. But it might be a smart first step in what will surely be at least a decade-long process of national scale and scope to arrive at the best ways of evaluating teachers.


2. WHY NOT JUST USE THE TOOLS CREATED BY THE NATIONAL BOARD OF PROFESSIONAL TEACHING STANDARDS?

Off the top of my admittedly-sometimes-addled head, I see eleven exceedingly reasonable reasons why this makes sense:

1. The NBPTS approach is widely used. I’m sure there are national board certified teachers in all 50 states, the District of Columbia, and maybe even Puerto Rico, too.

2. NBPTS certification covers both teaching and teacher professionalism.

3. There are different forms of certification (along with appropriate criteria) for different types of teachers. Let’s not underestimate how important this might be, especially to secondary folks.

4. NBPTS certification includes both teacher observation (via videotape) and testing of content knowledge.

5. Teachers really seem to like the process, and feel a great sense of pride when they complete the program.

6. A national infrastructure of trainers and evaluators already exists.

7. A large number of teachers have gone through the program.

8. It has been thoroughly debugged over a reasonable period of time.

9. It is respected by teachers, administrators, pundits, pols, and the general public.

10. It requires a periodic re-certification process.

11. It represents a high but achievable goal for all teachers.

Frankly, the more I go over this, the better it seems. But perhaps I’m missing something here. I will admit this came to me in a flash late last night while switching back and forth between re-runs of Seinfeld and Miami Vice (which no doubt inspired my earlier reference to “cigarette boats”).

But seriously, wouldn’t it be great to have a nation of NBPTS teachers? And wouldn’t the NBPTS process be a nifty way of helping teachers who weren’t quite making the grade improve their skills until they did. Perhaps best of all, this idea offers something for all the quants out there: the NBPTS program provides teachers with numeric scores representing their degree of success.

So, anyone wanna whack me over the head and tell me I’m speaking gibberish?

National Journal Experts Blog: Education

Deborah A. Gist, R.I. Commissioner of Education

October 21, 2009

How should teacher effectiveness be assessed? What role should student performance and standardized testing have in this equation?

Since becoming Commissioner of Education in Rhode Island, I have said repeatedly, in many forums across the state, that the single most important factor in the education of our students is the effectiveness of our classroom teachers.

One of my first steps as commissioner has been to share what I consider to be five priorities for transforming Rhode Island education in order to ensure that our students are ready for success in college, careers, and life. At the top of the list is: ensure educator excellence. We will do everything we can in Rhode Island to make sure that every student has a highly effective teacher in every class every year. This work will engage us in all facets of the teaching profession, from preparation to recruitment, selection, development, and throughout the career ladder. As part of this work, we will ensure that districts conduct annual educator evaluations that emphasize teacher effectiveness, including indicators of student achievement.

You can get a sense of how we will go about this work from the draft Education Evaluation System Standards, which the Board of Regents approved for public comment in August. These standards are based on six components, which are:

· a common vision of expectations for educator quality and effectiveness;

· an emphasis on continuous growth and improvement for each educator;

· an organized approach to growth and improvement of groups of educators;

· a differentiated evaluation process, with clear expectations for exiting educators from the profession as needed;

· an assurance of fair, accurate and consistent performance assessment that includes evidence of student achievement and input of feedback from many voices, including peers, parents, and student voices; and

· a process for reviewing and revising the evaluation system.

Two aspects of the evaluation-system standards have provoked considerable comment: the call for evidence of student achievement and for feedback from many voices.

It seems obvious to me that we must include student achievement and feedback from many sources as vital elements in any meaningful educator-evaluation system. In what other professional field would we not look at feedback from various sources, including those who are closest to the experience—in this case our students? In what other profession would we not look at the results for those whom we serve, whether customers, clients, patients, or students and their families?

Principles of validity and clarity will be important as we incorporate feedback from peers, parents, and students into our evaluation system. During our public hearings on the new evaluation system, for example, some teachers asked: If I say something in a conference that might anger a parent, am I in danger of receiving a negative evaluation? We have to ensure teachers that the evaluation process will be fair and all evaluators will receive thorough training in the evaluation process. I believe that students and parents must have a voice in this process, but there’s a difference between feedback and peer evaluation or supervisor evaluation, so we will weigh the components of the evaluation system appropriately.

Developing the right tool(s) and the right system is complicated and critical. We believe a system of evaluation should be based on multiple measures and processes. Videotaped lessons, peer support, parent and student feedback, formal and informal (i.e. “walk throughs”) classroom observation, samples of classroom tasks and student work, as well as measures of student achievement from multiple sources should all be considered as parts of a comprehensive system.

We need to see evaluation as a system, and it must be transparent and consistent for all. That means we must support and train our school leaders and our evaluators, so that they have an excellent understanding of the evaluation tools. Their judgments, after all, affect people’s lives significantly, and they must be accurate and consistent across departments and over time. The system needs to be a “living” system: we should review it regularly and revise and improve it as necessary.

Recently, a member of my team at the R.I. Department of Education shared his views on educator evaluation by saying: “While many have tried to persuade me that many other things matter, I have come to the conclusion that the primary measure is student achievement or growth in student learning over time.” I agree. However, determining how to factor in student-achievement data is a complicated area. Most would agree that we should not look only at a single year of data but should evaluate teacher effectiveness by measuring the growth in student learning over time, but there are other factors as well.

One big decision point is whether individual teachers or teams of teachers should be evaluated using student-achievement data. Most systems do not have strong enough data systems to be able to evaluate at the teacher level with confidence, although those systems are rapidly improving. Also, there are factors such as collaboration at the school that should be considered. We want a system that fosters teamwork throughout our schools, where our teachers are working together to drive student achievement rather than feeling as if they are competing with one another. We should use caution before rushing into measures of student achievement until we have a strong and thoughtful systems. Yet, we must make strides toward putting those systems into place.

I have asked students from early elementary to high school all across our state what makes a great teacher, and the consistency in their answers is astounding. Students want teachers who challenge them and help them learn while making learning active, relevant, and fun. They also want a teacher who cares about them as an individual and will do whatever is necessary to make sure they are successful. Students know whether they are receiving rigorous, quality instruction, and they know when they are respected by their teachers. Their descriptions of excellent teachers are spot-on every single time. At the end of each of these conversations, I make a pledge to those students that I will work tirelessly and fearlessly to ensure that they have a teacher like the ones they described in every single classroom, every year. They deserve nothing less than that.

How to assess teachers? I’m not sure—but here’s an idea. Let each community choose its approach. With one caveat: the existence of a review process on a District or State level that can pass judgments regarding the reasonableness of each plan. Such a more “distanced” panel of judges representing a broader constituency can negotiate with a local community over paricular provisions in their idiosyncratic plans.

At Mission Hill—a public K-8 “pilot” schools in Boston—we designed such a plan, which met the approval of both the local union and the local administration. It involved a probationary period of up to three years during which faculty on a rotating basis engaged in a serious peer review process. The school’s principal was one voice in the process, carrying no special power. Candidates could appeal the committee’s decisions to the full staff or school’s governing parent/teacher governing body, and eventually to the superintendent and union President—and of course, to the courts. All steps are well documented.

It has worked for more than 10 years—and is still working. It is difficult but enhances the entire school’s focus on doing our best for our students, while simultaneously acting as a kind o continuous staff development project. But all this happens within a school in which the faculty have open doors, meet frequently to discuss every aspect of the school and make most decisions collegially. Since the faculty also do the hiring it makes sense that they take responsibility for “firing.” Since it’s painful to fire someone, it puts a high premium on the selection of staff and on helping each other to constantly learn from our shared work.

Everyone knows that effective teachers are one of the most important factors in student success. And yet only four states require that student learning be the primary criterion in teacher evaluations, and only two states require that teacher effectiveness be considered as part of tenure decisions, according to the National Council on Teacher Quality. As a nation we spend hundreds of billions of dollars each year on teacher salaries, benefits, and professional development, but do very little to ensure that we attract, prepare, support, and retain effective teachers who help students to make the most progress, and remove those who do not improve.

In 2007, the Commission called for a major shift in the way we measure teacher quality—from evaluations based on qualifications to those based significantly on classroom results. Specifically, the Commission recommended that value-added student learning gains count for at least fifty percent of a teacher’s effectiveness determination, with the remainder to include a principal evaluation or teacher peer reviews (via a state- or district-approved process). Equally important, we recommended better supporting teachers in reaching effectiveness standards by using data and information from principal/peer reviews to target quality professional development to their needs.

Since then, more states have developed the sophisticated data systems needed to measure teacher and principal effectiveness, and now, American Recovery and Reinvestment Act (ARRA) funds may fuel additional progress in improving teacher and principal effectiveness. The potential for dramatic improvements in schools is great—but many questions remain about how to help schools get the best principals and teachers for their students.

To help answer these questions, the Commission is holding a public hearing next Wednesday morning (October 28th) at the Colorado State Capitol in Denver to examine how teacher and principal effectiveness can be measured; how we can attract, prepare, better support, and retain effective teachers and principals and remove those who do not improve; how to ensure disadvantaged students have access to effective educators and school leaders; and how NCLB can more effectively support teacher and principal effectiveness.

Commissioner Mike Johnston, a Colorado State Senator and former principal, will lead fellow members of the Commission’s Committee on Teacher and Principal Effectiveness in hearing testimony and engaging in in-depth dialogue with diverse witnesses to gain insight for improving NCLB. Participants include: Hon. Barbara O’Brien, Lieutenant Governor of Colorado; Mr. Tom Boasberg, Superintendent, Denver Public Schools; Mr. Dennis Van Roekel, President, National Education Association; Dr. F. King Alexander, President, California State University, Long Beach; and Dr. Dan Goldhaber, Research Professor, Center on Reinventing Public Education, University of Washington.

For more information, go here. Anyone can attend the hearing—just RSVP to nclb.commission@aspeninstitute.org.

I want, sadly, to announce here the death of researcher, writer and advocate, Jerry Bracey, who died quietly in his sleep last night, at age 69. His death has stunned many of us, particulary those of us who knew him personally and the many more who relied on his critical analyses and careful interpretations of data. His most recent book (of 8 published) is "Education Hell" (Educational Research Service), he blogged regularly on Huffington Post, he wrote the research column for Phi Delta Kappan, and he moderated for years the unusual "EDDRA" listserv, aimed at exposing and debunking educational misinformation. This list was heavily trafficked with spirited debates among informed people.

I will miss him greatly, and I send my and FairTest's condolences to his widow, Iris.

Sometimes it is worth weighing in late in the development of a discussion because you can see a wide array of interesting posts. I will avoid repeating them, but I want to give special kudos to Bob Peterson for reminding us that unless the nation is willing to pay enough to provide good schools and ensure skilled professionals (which we do not in most urban, some suburban, and many rural areas), we cannot expect good results.

Yes, serious evaluation of teachers aimed primarily at improving their craft is fundamental to improving schools, and that means principals and other teachers having the time and knowledge to do that job well. Decisions on denying tenure, structuring pay, or removing teachers requires even more careful evidence, though good ongoing evaluation of teachers by principals and their peers is the place to start.

But we should consider all this in light of an emerging politics of charging pell mell into mandated high stakes evaluation of teachers despite the absence of good structures, the lack of resources to do it well, and a continuing propensity to rely on the same inadequate tests inflicted on children as a key part of "evaluating" teachers. (I put "evaluation in quotes, because major use of these instruments no more deserves to be called an "evaluation" than a trivial five-paragraph write to a prompt is an "essay.")

The October 5, 2009 National Academy of Sciences Board on Testing and Assessment letter on the "Race to the Top" draft guidelines should be required reading for key personnel in the Department of Education and for Education Committee members and their staffs in both Houses of Congress. It makes clear that the preponderance of evidence shows that current tests are not adequate to the job of seriously evaluating either students or teachers, and that "value added" or "growth" measures are not ready for prime time use. This is not a matter of waiting for perfection (as Sandy correctly cautions us against), but a warning not to cause further damage to education by intensifying the misuse of tests built into NCLB. In this letter, neutral scholars back up key positions not only of FairTest, but the Forum on Educational Accountability, the Forum on Education and Democracy, the NEA and the AFT, as stated in their letters on RTTT submitted to the Department.

All the good advice posed on this list won't matter if the Department insists that states rapidly impose evaluation systems and require they use current standardized tests as a "significant part" of evaluating teachers. Let's stop this one cold, then use RTTT money to start building solid evaluation systems and far better assessment systems. (This means not just a new array of standardized tests pretending under the pretense that one-shot exams can be good enough for controlling curriculum and instruction and making decisions about students, teachers, principals and schools).

We have known for years that any system is only as strong as its weakest link, and as "The Widget Effect" report demonstrated, teacher evaluation in today’s schools is inadequate. Today we have the opportunity to build more rigorous, aligned evaluation systems that can address poor instruction and build more effective teaching and learning communities that work for all learners.

School-effects research has shown clearly that teachers are the most important school-based factor in student learning, and it seems logical to evaluate teachers on just how much or how little learning they “produce” in students. However, assessing teachers’ contributions to student learning is tremendously difficult, not always practical, and more science needs to be conducted to develop valid and reliable ways to do this. Teacher evaluation systems must therefore be built using multiple measures, not only of student outcomes but also of teacher performance in the classroom.

No matter how we assess teacher effectiveness, however, it is critical that we consider the systems that support teaching and learning and assure that they are effective as well. First, for all 3.2 million public school teachers to be effective, they need high quality consistent curricula, that is tied to common high standards and aligned with high quality assessments. It should go without saying that we cannot hold teachers accountable for student test scores if their students are tested on things they did not have the opportunity to teach. Our preparation programs and professional developers also play a vital role and can more effectively target their programs toward meeting the specific needs of the 21st century teacher and bring them to more efficient and effective scale if there are common learning standards and assessments.

Second, teachers need multiple opportunities to learn and improve their skill. They need to learn how to personalize learning, to use student data and quality research evidence to make sound instructional decisions, and to analyze their own instruction when the evidence is uncertain. Third, they need to be part of a genuine professional learning community. Teaching and learning are social endeavors, and students do not learn from just one teacher in a school day or an academic year, they learn from many teachers.

Noting these needs that lie beyond evaluation alone, it is critical to have systems ready to identify teachers who are not currently effective. And teachers agree. A forthcoming national research report from Learning Point Associates and Public Agenda, will show that approximately a third of all teachers believe that removing ineffective teachers from the classroom would be a “very effective” way to improve teaching. This finding implies a couple of things: first, teachers feel the impact of colleagues on their own ability to teach well, and second, we need better systems to identify those ineffective teachers and swiftly and humanely replace them with more effective teachers. A difficult endeavor to be sure, but one that must be undertaken to ensure that all students learn what they need to be successful in the 21st Century.

The incredible number of affirmative suggestions here make the obvious moreso: There exists a great deal of experience and organizational support for any school that wants to know how to evaluate quality teaching and teachers.

I think it is silly to suggest that there is one single best way to do this, and pursuing such a belief only ends us up where Secretary Spellings suggests...everybody gets a star. Schools are different; they value different kinds of behaviors.The point is that in order to be excellent, they must engage in vigorous and ongoing self-evaluation in all sectors of the school. There is not an excellent school in the country that does not do that.

What policy makers need to ensure is that schools are not PREVENTED from doing so. Today's discussion seems blithely dismissive of the fact that we have laws and regulations that prevent a principal from using ANY test data to evaluate a teacher...even test data from measurements the school chooses itself. Secretary Duncan has called this out, and we should be grateful for the added push he gives to eliminating this serious barrier.

Schools struggling to be great will always want to evaluate all their staff in ways that reflect the school mission. We must not prevent them from doing so.

Schools that are not struggling to be great will not want to evaluate anybody, and the results for students will reflect that sad reality.

While there is a significant amount of interest in assessing teacher effectiveness, it’s not new in education and we have been struggling with it for decades. Back in Texas (and other states and districts) in the mid-80’s, policymakers were busy creating career ladders for teachers as a way to identify and reward excellence and target professional development to teachers who needed more help. After putting lots of time and effort into creating a system that would allow excellent teachers to move up the pay scale without leaving the classroom, almost all the teachers were rated as excellent. A few years later the entire system was thrown out and went the way of the dinosaur. We went back to giving across-the-board pay increases to all teachers. Since that time, we’ve started innovative practices on performance pay in some districts and states, but we’ve clearly not learned our lesson in a meaningful enough way across the country.

It’s clear that education is never going to be a high-performing enterprise with such a random and imprecise means of determining quality and effectiveness, especially when human capital is so integral to the success of our kids. Other sectors have figured this out—and it’s high time that we used those lessons in education. My colleagues at the Boston Consulting Group have broad experience in this area and have worked with school districts that want to do a better job of measuring teacher effectiveness. They have found that districts rarely use objective measures to evaluate teachers and usually don’t have clear rubrics to evaluate performance that are understood by teachers, principals, and other managers. Without those essential components, it’s no wonder that we aren’t effectively assessing the performance of teachers. They have also found that principals are often the sole evaluator of a teacher’s performance and that they have little time or training in how to do that. Without addressing these most critical issues, today’s teacher effectiveness systems will go the way of the career ladder and the dinosaur.

Bob Peterson's excellent post makes two things clear:

1. Unless evaluating teachers includes evaluating the conditions under which they work, it is of little value for improving professional practice.

2. Teachers need to be at center of the evaluation process in both its design and implementation.

Peterson describes a school where, due to an underlying lack of funding, teachers lack planning time (common or otherwise), music or art staff, administrative and other supports, in short, a school without the work environment or support system to make professional growth a serious priority. In light of the deep funding inequities in our state finance regimes, such conditions exist in many schools, especially in low wealth communities serving high concentrations of poor students and students of color. In these under-resourced schools, teacher evaluation becomes a monitoring charade -- or a contractual battle -- between employees and supervisors instead of a collaborative process of professional growth and development.

Give teachers models of good instructional practice, a healthy school culture in partnership with the community it serves, and system-wide responsibility for providing the resources essential for a quality education and you have the basis for a meaningful accountability system. Give over-extended and under-supported school personnel inadequate funding, decrepit facilities, poor working conditions, and a lack of professional supports, coupled with unrealistic expectations handed down from the state capitol or Washington, and you have the makings of more grievance procedures and pass-the-buck evasion.

Peterson’s description of the on-the-ground reality should give us pause. Efforts to tie teacher evaluation in under-funded schools to standardized test scores will only make things worse. The overuse and misuse of testing for "accountability" purposes has already contributed to the de-skilling of new teachers and the demoralization of veteran ones. It also feeds narrow notions of "rigor" and "academic achievement" on the students and teachers experiencing educational inequality, a distraction from the state officials responsible for it.

The question of how to ensure teacher effectiveness is a tricky one – especially for those of us in Washington. Everyone can agree we want to get as many effective teachers into America’s classrooms as possible. And yet most of us recognize the way to achieve that goal differs from state to state and school district to school district. This is why the federal government cannot step in and dictate a one-size-fits-all approach to measuring teacher effectiveness. As AEI researcher Rick Hess cautioned during a recent Education and Labor Committee hearing, research indicates the most effective teacher in one classroom is not necessarily the most effective teacher in another. This underscores the fact that programs to evaluate and support teachers must come from the local level and reflect local differences. As I’ve written on more than one occasion, what works in Lakeville, MN may not work in Los Angeles, CA.

Although there is no magic federal formula to assess teacher effectiveness, this discussion is certainly worthwhile. We have seen promising local models that have helped districts put the right teachers in the classrooms – examples that can be replicated, and localized, elsewhere. We know programs like the Teacher Incentive Fund, in which teachers are financially rewarded for improving student achievement in low-income schools, work because they treat teachers like professionals and allow them to be recognized for their achievements. We can also learn from initiatives like the Teacher Advancement Program or promising strategies being tested in charter schools – data-driven approaches that help teachers identify their weaknesses and implement strategies for improvement. With a program such as TAP, which provides a framework of four main tenets for assessing and promoting teacher effectiveness, local flexibility is essential.

Finally, to assess teacher effectiveness and take the next step of actually utilizing that data to improve students’ access to quality teachers, districts must be willing to tackle the tough political challenges. For instance, collective bargaining agreements sometimes require the last teachers hired to be the first fired – irrespective of merit. Similarly, a stubborn refusal to even discuss linking student achievement to performance pay will stymie even the most ambitious local efforts to improve teacher quality. Measuring teacher effectiveness is important, but what we do with that information is equally critical.

“The Widget Effect” report offered a comprehensive and crisp diagnosis of the problem with existing teacher evaluation systems. Evaluations typically do not result in sufficient gradations to reward the most exemplary educators and hold the most ineffective accountable for unacceptable performance. Even more important, they rarely provide informative feedback to help individual educators improve their practice.

How do we best define and measure teacher effectiveness? Some conceive the definition of teacher effectiveness as exclusively based upon value-added student achievement data. While value-added data can be used to reward and recognize teacher performance and should be used to inform teacher evaluations, it should not be the sole method by which teachers are evaluated and deemed “effective” or “ineffective.”

Great teachers are made – not born. Teachers need professional support and opportunities to develop their practice, including focused induction during their initial years in the profession. Those who see the firing of the least effective teachers and the rewarding of the most effective as the primary goal of value-added driven evaluation are missing the bigger picture. They ignore the vast majority of good-to-very good teachers who can achieve even greater success if given access to high-quality induction and professional development, supported by data and evidence of student learning.

There’s room to embrace value-added methodology without allowing it become an end-game. It is important to measure teacher impact on student learning, but measuring impact without providing the means to help educators strengthen their practice will ultimately fail our schools. In our induction work at the New Teacher Center, we employ a Formative Assessment System (FAS) to increase teacher effectiveness through support provided by carefully selected and trained instructional coaches and mentors. Beginning teacher formative assessment can help build a more complete picture of teacher effectiveness than student assessment data alone. Through the use of FAS, high-quality induction programs and job-embedded professional development rapidly advance teacher practice. We also feel that it has tremendous applicability to inform changes in teacher evaluation systems.

Numerous states have developed definitions of effective teaching that incorporate essential knowledge, skills and classroom practices, and which do not rely exclusively on outcome-based measures such as student achievement. Effective teaching includes aspects of teacher leadership, the ability to embrace diversity and individual learning characteristics, and content knowledge. Planning instruction to meet the needs of all students, provide relevant instruction, make connection across content areas, develop effective communication, and foster critical thinking skills are measures of teacher effectiveness as well.

Effective education reform strategies must recognize teacher development as a primary means to maximize classroom effectiveness. States, districts and schools must go beyond merely identifying the most and least effective teachers, but see that the successes of our best educators form the building blocks of a better understanding of effective teaching practice that can be replicated in classrooms across America.

In education reform, one of the few ideas nearly everyone agrees about is that effective teachers are essential to improving student achievement. In New York City, we are developing a sophisticated system for measuring teacher effectiveness that avoids the debates that have quashed many past attempts. With funding from the Gates Foundation, we have begun a collaboration with the United Federation of Teachers and independent researchers to identify and support good teaching over the course of two years—a study that will be based on our shared belief that teachers teach best when they understand what’s expected of them, know clearly how to reach their goals, and feel assured that no single snapshot measure will determine the course of their careers.

The initiative builds on existing efforts in New York City to improve teacher effectiveness. Last year, we provided “Teacher Data Reports” to principals of schools that provided instruction in grades 4-8 reading and math. The reports show the “added value” teachers brought to their students’ performance. Principals (who could share the reports with individual teachers) are using these measurements to provide targeted mentoring and professional development. In the longer term, teacher data reports and similar instruments should be part of a comprehensive approach to improving the quality of the instruction teachers provide: not as the sole determinant of how a teacher performed but as one means to improve support to teachers, assess performance, and reward effectiveness.

It is not only appropriate, but essential that we use test-based measures like New York City’s school progress reports and teacher data reports to help gauge the effectiveness of our schools and educators. I also believe, however, that we need to improve classroom observation tools and performance measurement of teacher skills to find robust approaches, validated empirically as being related to student achievement, which can be combined with quantitative measures for a rich picture of teacher strengths and development needs. A combination of validated skills measurements, school-wide achievement, and teacher value-added measures could be combined into a fair and balanced approach to assessing teacher effectiveness. I should add that this multi-faceted approach to teacher evaluation is a critical component of the agenda of the Education Equality Project (www.edequality.org), of which I am a Founder.

It amazes me that we continue to rely on subjective teacher performance management systems that proclaim 99% of the adults in our schools to be doing a satisfactory job even as we fail so many of our students. As we look to the future, we must develop a more holistic approach, which includes evidence of student learning as the central focus in our assessment of effective teaching.

I’ve been a classroom teacher for 29 years (I tell my students I flunk fifth grade every year.) The teacher evaluation systems I’ve seen over the years generally fall somewhere between inadequate and horrible.

So yes, teacher evaluation needs to be improved. But before talking about how, one reality must be addressed. No new system will work unless we also change the growing problem of inadequate time, a problem which makes it almost impossible for classroom teachers to seriously reflect, evaluate student work and collaborate with colleagues to learn best practices from one another.

Let me be specific.

The bilingual public school where I work has 413 students. We have one principal and no other administrative staff. Because of annual cutbacks in our funding we have no educational support personnel except for two who work in special education. We have no physical education or music teacher. Our head secretary position was cut to half time, and we have one assistant secretary.

This means that classroom teachers have only 45 minutes of preparation time every six days (we are on a six-day rotation) and have lunch or recess duty every other day. The consequences for teaching and learning are significant: there is virtually no time for staff to observe each other’s teaching, plan together, talk about students, collaboratively assess student work portfolios, and so forth. It also means that the principal has virtually no time to be in teachers’ classrooms to observe, evaluate, meet individually with teachers, or act as a pedagogical leader. She is too busy supervising lunch, resolving transportation mix-ups, or dealing with discipline and safety issues.

If we want serious – not superficial –principal observation to be part of a new evaluation system, we need to fund schools so principals have time to do quality observations and to meet with teachers before and after such evaluations.

If we want teachers to improve their teaching, we need to fund schools so teachers are treated like professionals and have adequate planning and collaboration time every day, not just once or twice a year during so-called professional development days.

Let’s assume that policymakers take the issue of teacher assessment seriously and devote sufficient resources. There is another, more fundamental question that also needs to be answered and that goes beyond the question of how. And that is, why should teacher effectiveness be assessed?

I see two main reasons.

The first is to identify the small percentage of teachers who are obviously ineffective, dysfunctional, or dislike children. The second is to help improve the teaching of the those who don’t fall into the first category – I’d estimate that being around 95% or more – in order to improve student learning.

I know that it is easy to identify the “bad” teachers. Virtually everyone in a school knows if there is a teacher like that and who that teacher is; teachers know, educational support staff know, the principal knows, and many parents and students know. There is no need for some test-driven evaluation system to determine this. The question is what’s going to be done about such teachers.

In the past, our school district administrators argued that due process rights of teachers prevented them from getting rid of bad teachers. In my experience, the real problem rested in either overworked or ineffective building principals. The result was that instead of doing the work of documenting the teacher’s ineffectiveness and either helping them get better or get out of teaching, they would give the teacher a satisfactory evaluation and help them transfer to another school. After several years of satisfactory evaluations, it was even harder to get rid of the teacher.

Several of my colleagues in the Milwaukee Teachers Education Association and I pushed for a very different approach, which is commonly known as peer evaluation. In Milwaukee it is known as the TEAM (Teacher Evaluation and Mentoring) Program. It is a district-wide program that provides support to struggling teachers, assists principals in evaluating such teachers, and if the teachers don’t show satisfactory improvement in one or two semesters, provides job counseling in careers other than teaching. The program is run by a joint board of administrators and teachers while a group of veteran teachers (released from classroom responsibilities) work with individual teachers. Since the TEAM program started in 1997, 65 participants successfully completed the program and are now performing effectively in the classroom, while 96 teachers resigned or retired from Milwaukee Public Schools while participating in the program. If you extrapolated those numbers to large districts such as New York, it would encompass hundreds of teachers.

If the Secretary of Education Arne Duncan wants to significantly improve the quality of the teaching profession, he would encourage and aggressively fund the expansion of such programs.

And if we want teacher evaluation to include how well students are learning, then we need to be clear about the following:

a) student assessment means students must demonstrate what they know through performance-based assessments in writing, science, math and oral presentations. Snapshot standardized tests cannot, and must not, be the backbone of student and teacher evaluation.

b) student assessments should be thoughtfully evaluated by groups of educators in such a way that fosters the growth of students as learners and teachers as teachers.

c) quality assessment systems – whether they are based on performance assessments, portfolios and rubrics or exhibitions of student work – are significantly more rigorous than standardized tests, and push teachers towards teaching practices that we know are more effective than the drill and kill approaches fostered by standardized tests.

It is time to treat teachers like the professionals our craft requires. Relying disproportionately on standardized tests reduces teaching to a data-driven, sterile game of numbers – one that negates not only the craft of teaching, but fundamentally disrespects the intelligence, creativity and life of the students we serve.

Teaching is a complex professional task and requires assessments that are designed to capture that complexity. That means we need to stop thinking simplistically about measuring teacher effectiveness and start thinking systemically.

Assessments of teacher performance should include a comprehensive and collaborative approach. And assessments and evaluation should have as their central principle the improvement of teacher knowledge, skills, and classroom practice—with the ultimate goal of enhancing student learning.

Assessments should also incorporate multiple sources and kinds of evidence. Why? Because all measures provide only a partial view of teacher performance. Student performance on standardized tests is an important part of this richer array of documentation – including classroom observations and portfolios – that can provide educators with truly useful information that informs diagnostic, curricular, and instructional decisions.

However, we shouldn’t continue the unhealthy focus on standardized tests as the primary evidence of student success. Test-based measures of teacher “effects” are all the rage despite warnings from respected researchers that these measures should not be used for individual teacher evaluation. I also worry that our present “tyranny of testing” is jeopardizing our ability to match the neediest students with the most able professionals.

That is why NEA launched its Priority Schools Initiative aimed at attracting, preparing, and supporting accomplished teaching in high-needs schools. Our report, Children of Poverty Deserve Great Teachers, outlines ideas for better ways to gauge teacher effectiveness and teacher success.

I recommend that policymakers closely watch the progress of our colleagues at AACTE, CCSSO, and Stanford University, who are partnering to create performance assessments that might provide us with a valid, national teacher assessment that will truly meet our needs. I am also hopeful that the Bill & Melinda Gates Foundation's recent initiative to support the development of new measures will yield methods we can all agree on—methods that are fair, powerful, and reliable.

I don't have much to add to the fine comments that have been made.

Rather I simply want to praise The New Teacher Project for the wonderful work they've done here. There is groundbreaking work from others as well, including the NCTQ, TFA, and TAP, that deserve tremendous credit.

I recall all too vividly my years on a school board when the only data we had on teacher performance was subjective evaluations showing virtually all ratings to be "exceeds expectations" or better. This was so despite the fact that student achievement was pretty universally well below expectations.

The new tools that these fine folks are developing will not be perfect, nor should they be expected to be. They should be fair, as precise and accurate as possible, and always subject to being improved. Mainly they should be designed and used to help teachers do better. Further, they should help those who are in a position to help teachers do better themselves. But, importantly, these tools should not be measured or dismissed because they are imperfect. Judge them not against a standard of perfection but rather against the awful "measures" they will replace. True professionalization of teaching depends upon it.

Finally, I want to express gratitude to Chairman Miller and Secretary Duncan for their leadership on this issue. I would be remiss if I didn't also remember the work on teacher effectiveness of former Secretary Spellings and the late Senator Kennedy and their staffs.

I guess I’m one of the fortunate people in this discussion because I’ve had my teaching evaluated, I’ve evaluated other teachers, and I’ve created and reviewed teacher evaluation instruments.

I’ve long held five unfashionable points of view on the issue of teacher evaluation:

1. Intent precedes instrument.

2. Evaluators are more important than evaluation.

3. The hard work has already been done.

4. 360-Degree models work best.

5. Integrity is everything.

Inside of education, I’m often told these ideas are “nutty” or “impractical” or that “they do not conform to existing research on teacher evaluation.” Sometimes people just get angry. By contrast, among the people I talk to outside of education – and especially those with evaluative responsibilities in large for-profit, non-profit, and government organizations – these ideas seem reasonable if not painfully obvious. So I’m eager to learn from all of you about where I might be making mistakes and why my intuitions about these ideas might be wrong for education. (Seriously, I’d really appreciate the feedback. Plus, it makes the blog a lot more fun.)

QUICK PS: Evaluating teachers on test scores, either in part or in whole, makes little sense for all the reasons Diane Ravitch brought up – plus the fact that most, if not all, states monkey around so much with their testing systems that scores simply aren’t reliable over time. Mr. Antonucci has a great real-world angle that, in general, I agree with and applaud. But I’ll bet if we dropped by a big city firehouse or hospital, a large military base, or even the Pentagon, we’d eventually notice that performance assessments were a normal part of how all of these types of organizations operate. In general, we don’t wait until the city burns down before the Fire Chief gets fired. Chances are good, he or she will have risen to the position over many years of service evaluated by his or her superiors and often peers (not to mention taking tests to move up to higher pay grades and levels of responsibility). Finally, teachers don’t stand alone within their school communities. Many, if not most, have significant interactions with others. A recent study here at UNC-Chapel Hill showed that “high quality” teachers can have a positive effect on less talented peers just by spending more time with them. In addition to that, I have often been surprised in my consulting work to find that a teacher of rather modest abilities in the classroom was the glue that held a school together. It is my belief that reductive views of teaching lead to reductive teaching practices – and eventually to reductions in student achievement.

-----------------------------------

1. INTENT PRECEDES INSTRUMENT. The main reason teacher evaluation is messed up now is because the intent of traditional teacher evaluation is wrong. The intent of traditional teacher evaluation is to avoid evaluating teachers. When 99% of a group receives the same evaluation – whether it’s good, bad, or in between – evaluators are not making any evaluations. The reason most evaluators avoid evaluating is because they are uncomfortable with it. They tend to be uncomfortable with it because they either don’t know how to do it well or because the resulting surface-level harmony and lack of confrontation they achieve by not evaluating are fixtures of traditional education culture that they value greatly and strive to maintain.

So what should be the intent of teacher evaluation? Should it be to help teachers improve? Or to fire the weakest among us? This is a crucial distinction to understand because achieving each goal depends on a different kind of instrument and process.

If you want to fire people, you draw up an incredibly complex evaluation with large numbers of categories and a wide array of highly technical descriptors. You also create a scoring system that is difficult for those being evaluated to understand. In this case, complexity alone is often enough to catch people not doing their job. The greater the degree of complexity, the greater advantage evaluators have over the people being evaluated.

But if your intent is to help teachers improve, a relatively simple instrument with a small set of easy-to-understand criteria that are strongly aligned with a school or district’s highest instructional values works best. The instrument will be as easy for teachers to understand as it will be for evaluators to implement. The advantage that accrues in this case is to the school or district as a whole – including teachers – because using this type of instrument to help teachers improve their teaching is simple and straightforward. In this scenario, many teachers may still lose their jobs but only if they don’t make improvements based on a small set of clearly stated goals that are aligned to organizational values.

This alignment process is probably the chink in the armor when it comes to designing new teacher evaluation systems. In 15 years of consulting, I’ve never worked in a school or district that had clearly defined instructional values. Without clear values, any instrument may rightly be deemed irrelevant, unfair, or otherwise out of integrity. In this case, high quality evaluation will be elusive.

CONCLUSION: Choose simple instruments aligned to values and focus on teacher improvement not termination. Pilot simple instruments, and the improvement processes they suggest, with those who you believe to be your strongest teachers. Apply their feedback to hone the process before “going live” with the new system. Be clear and open about your intent and proceed with a watchful eye on integrity.

-----------------------------------

2. EVALUATORS ARE MORE IMPORTANT THAN EVALUATION. Regardless of the organization’s intention in evaluating its people, the ability of evaluators to do their jobs well is more important than the instrument or the evaluation process itself. Case in point: even with today’s ridiculous (non-)evaluation systems, some schools where I’ve worked do a terrific job of evaluating – and improving – their teachers. So, even with poor instruments, or even no instruments at all, savvy evaluators can do a fabulous job of helping teachers hone their craft – and helping others out the door.

Evaluating teachers is not a mystery; it’s based on a set of traits like anything else. In general, people who possess these traits are more apt to recognize them in others and to be able to help others develop them in themselves. For example, our company “evaluates” teachers every day we’re in their classrooms – even on days when we’re the ones doing the teaching. We either produce a written summary or we use a simple numbering system to note how folks are doing. The progress teachers make can be easily tracked. And an informal report can be generated if anybody other than us wants one. But the evaluation tools we use are irrelevant. Even before we made them up, we were still good at evaluating teachers and helping them improve. We made the tools and designed the system in order to have more “data” to give to administrators -- most of whom, by the way, couldn't do anything with it because they weren't good evaluators to begin with.

Most teachers I have known have felt extremely fortunate to have had even one competent and helpful evaluator during their entire career. Some are so shocked and so excited by working with a good evaluator that they beg for more evaluations!

CONCLUSION: We’re spending too much time in our country right now arguing about what types of evaluation instruments we should be using. This puts the cart so far before the horse that I’m not even sure the horse knows what a cart looks like, let alone how to pull it.

-----------------------------------

3. THE HARD WORK HAS ALREADY BEEN DONE

We already have a fantastic knowledge base on evaluating job performance. Much of it was originally created for the business world. But even large non-profits have formalized high quality evaluation systems. Why not borrow from what already exists?

I grew up in Seattle, WA. Not surprisingly, I have many friends who work at Microsoft from low- and mid-level positions all the way up to top VPs. Through my friends’ eyes, I’ve watched Microsoft use a number of different evaluation systems. Some seem to work better than others for different parts of their business. But at least they have them, and probably have for twenty years for most of their existence. Older companies have been doing employee performance evaluation since the 1950s. And specialists in HR and organizational development have been studying these approaches to discover if and how well they work. This gives us a good research base to work from.

As an aside, since we’ve all decided that the purpose of education is to help kids get jobs, it might be nice if their teachers had some real-life experience with the kinds of performance evaluations their students will experience after they leave school. This would just be an added bonus, of course, but it does bring up the notion that because so few people in schools have ever been truly evaluated, or have ever given a true evaluation, it’s a safe bet that those of us who’ve spent the entirety of our working lives in school may not actually know very much about evaluation. Using tools vetted by others might be the most responsible course of action, at least at the beginning.

CONCLUSION: The notion that “education is a business” is one of the most over-worked and thoroughly inappropriate metaphors in all of reform. By errantly conflating a system with an entity, it encourages poor predictions (NCLB: a significant percentage of parents will move their kids out of failing schools to other schools with marginally better test scores) and justifies flawed theories of action (CHARTER SCHOOL POLICY: the existence of charter schools will improve nearby public schools as a result of competition and normal market forces). However, this does not mean we can’t use business tools if they are deemed appropriate and proven effective. In schools we should strive to emulate the processes and practices of high quality evaluation programs as already identified in real-world applications and best practice research in HR and organizational development.


-----------------------------------

4. 360-DEGREE MODELS WORK BEST

In school, evaluations only go half the distance. But 360-Degree approaches seem to make people feel more comfortable and to increase honest communication between the parties involved.

Imagine how different our approach to evaluation would be if teachers evaluated themselves first, were then evaluated by their evaluators, and finally met once more to square the two impressions. This brings in the vital component of self-evaluation. It also gives the evaluatee a voice in the evaluation process. Finally, it corrects errors and reduces misunderstandings.

Perhaps the best part of a 360-Degree approach is that it starts the evaluation process off as a team effort. This makes evaluation less adversarial and more cooperative. People who have been through well-run 360-Degree processes often remark how much more thorough these approaches are and how the process lead them to a clearer picture of their strengths and easily pinpointed areas for improvement. Again, this is a great approach if your goal is to help teachers improve. If your goal is to clean house with Operation Pink Slip, your best bet is to stick with traditional “180” evaluation, an overly complex evaluation instrument, and a legion of so-called “trained evaluators” who will turn in scores anonymously so that teachers have no recourse to contest or to clarify their results.

As an aside, our company has trained many teachers to use informal and formal 360-Degree approaches in their classrooms with their students. We’ve developed an original approach to grading based on a 50/50 split of student self-assessment and teacher assessment. Kids certainly need to be trained to do this well. But for many, especially at the high school level, this is some of the most valuable learning they have ever received. Teachers remark, too, that in using the 360-Degree approach, their assessment becomes more accurate and they get to know their students better than they ever have with the traditional 180-Degree point-percentage grading model.

CONCLUSION: Use 360-Degree assessment models performed by well-trained in-house personnel. And use the same style of evaluation for the evaluators as well. Everyone within an organization should have a visceral first-hand understanding of the power and practice of performance evaluation.

-----------------------------------

5. INTEGRITY IS EVERYTHING

If schools or districts are serious about instituting effective teacher evaluation systems that lead to improvements in student achievement, these systems will have to be taken seriously by teachers. They can’t be seen as temporary fads or – even worse – as tools for termination.

The integrity of an evaluation system can be measured subjectively by the degree to which the parties involved believe the evaluation aligns with organizational values and objectives. The integrity of evaluation systems can also be measured quantitatively by comparing evaluation results with degrees of individual improvement. If evaluations aren’t aligned with the true nature of the organizations that use them, or if evaluation data can’t be causally connected to improvement in teacher and student achievement, these instruments are likely to damage schools more than they are to help them.

CONCLUSION: Transparency in the evaluation process is essential to the creation of trust which is essential to the success of using evaluations to further improvement. Everything that can be done must be done to reduce the adversarial nature of the evaluation process. Teachers must be included in the selection, creation, and implementation of evaluation instruments. They must also be ready, willing, and able to state for themselves the types of evaluation systems they believe to be fair, accurate, and focused on improving teaching and increasing student achievement.

How should teacher effectiveness be assessed?

I don't know and I don't care, because I'm never asked a similar question when it comes to police, firefighters, architects, accountants, the military, engineers, nurses or even college professors. We judge those folks on whether they perform the tasks assigned and get the results we desire. If that doesn't happen, we either take our business elsewhere (private sector) or demand changes from elected officials (public sector).

I am unqualified to design a test for architects, but if her bridges collapse when they should have stood, I want a different architect.

I am unqualified to judge generalship, but if he loses battles he should have won, I want a different general.

I am unqualified to practice medicine, but if patients die when they should have survived, I want a different doctor, nurse and hospital.

I'm flummoxed by the notion that K-12 teaching is different from every other profession, and that the inability of a raft of children to read, write and compute has no relation to teacher quality. It might please education professionals if someone somewhere could design the perfect combination of credentials, certification, experience, personal qualities, subjective and objective evaluations, which when mixed according to the proper formula, would yield a figure by which we could assess teacher effectiveness, akin to batting averages for baseball players.

Alas, I'm afraid pig-headed parents and taxpayers will continue to assess schools and teachers based on whether their children are learning. Why bother with "great teachers" if we don't end up with great students?

Teachers should be evaluated mainly in two ways: One, by their supervisors who regularly visit their classes and observe their teaching. And two, by their peers, who interact with them and who know them well and know the students they have taught.

The current demand to evaluate teachers by their students' test scores is of limited value. It has no application to teachers in the earliest grades, because their students are not tested. Nor does it apply to high school teachers, for the same reason. Many of those who now teach in elementary and middle schools also do not have student test scores by which to be evaluated.

For those whose classes do produce scores, the scores have some value but are not nearly as important as the judgment of supervisors and peers. If we put too much emphasis on test scores as an evaluative tools, we will encourage bad teaching of a rote sort and help to squash creativity and demoralize teachers.

Diane Ravitch

Assessing teacher effectiveness is obviously not an easy and straightforward endeavor, but developing a robust system for evaluating effectiveness is probably the most important educational reform a district can undertake. As The Widget Effect helpfully noted, we can't continue to treat teachers as interchangeable parts if we want to attract and retain talented people in the profession, help them improve their practice, and provide additional rewards and responsibilities to those who we hope will remain in the profession and help their colleagues to improve.

While there are few exemplary systems, there are a number of important principles for developing a system for measuring teacher effectiveness. This list isn't comprehensive, but it's a good place to start.

1. To ensure that teachers embrace the system as legitimate and meaningful, they must be involved in developing and fine-tuning it.
2. The system should be based on a shared vision of what teachers should know and be able to do.
3. Because teaching is complex work, an evaluation system should include a variety of measures of teacher effectiveness. These measures must include some type of teaching observation framework or rubric and student growth on standardized assessments. Other measures might include graduation rates, other assessments, portfolios of student work, and parent feedback. None of these measures is sufficient to assess teaching practice on it's own.
4. Estimates of teacher effectiveness derived from student scores on achievement tests should play a role in the evaluation system, at least in grades and subjects where this is possible. Use of states' annual end-of-year assessments in this sense should be complemented by data from ongoing assessments tied more closely to fine-grained instructional objectives (See Vander Ark's post).
5. Evaluators should be knowledgeable about effective teaching and should be trained in using evaluation tools. It is also helpful if at least one of the evaluators is independent or based outside the school so they can be more objective.
6. The evaluation system should have both formative and summative roles. Teachers need feedback pinpointing ways in which they can improve their performance, and administrators need bottom-line indicators of effectiveness if they are to manage personnel strategically, that is with an eye towards maximizing student achievement gains.

There are few systems that contain all of these components, but one that does is the Teacher Advancement Program. TAP is a comprehensive school reform that includes a high quality evaluation system, additional roles and responsibilities for teachers, performance pay, and professional development. The evaluation system is based on a framework of standards and indicators that is operationalized against a five-point scale rubric. Teachers are evaluated four to six times a year by several trained evaluators. Teachers are also evaluated according to classroom level and school level student growth. To learn more about the TAP system, see http://www.tapsystem.org/.

The TAP system is one example of how districts could do a better job in measuring teacher effectiveness. There are likely others that could be developed.

Any new system must recognize the complexity of teaching, use a balanced approach to gauge teacher effectiveness, and promote professional growth by offering all educators meaningful feedback and opportunities to advance in their careers.

There must be an empirical component in both teacher (and administrator) evaluation. This would emerge from the results of standardized tests using robust value-added models to identify the most effective and least effective performers. Research makes clear that value added models are accurate in identifying the “tails” of distribution.

These student-learning results would be accompanied by a peer-review process that uses rigorous evaluation protocols to differentiate the quality of teaching behaviors, such as Charlotte Danielson’s “Framework for Teaching” that examines 22 components of teacher performance in four broad domains. Taken together, they would replace simplistic ratings of “satisfactory” and “unsatisfactory” and offer a much more comprehensive picture of teacher and administrator effectiveness.

Evaluation should feed into a system that has both positive and negative consequences. The multiple discussed above would then be used to determine the progress teachers and administrators make in climbing a career ladder. No teacher would earn less in the new system than he or she did in the old. Much higher salaries would be available for highly effective educators and those serving in leadership roles, and all teachers, regardless of subject taught or specialist function, would have an opportunity to earn additional compensation.

Teachers identified as “ineffective” would be provided with extensive supports through a program “peer assistance and review,” or PAR, but would lead to timely dismissal if a panel of teachers and administrators agreed on that recommendation. Unions would still provide legal representation to ensure due process, but experience suggests that courts would not be likely to overturn decisions made by a PAR panel.

Ted Hershberg and Claire Robertson-Kraft

We need to improve observation and value-added data to dramatically improve teacher evaluation. The best observation system I've seen is KC KS where teaching is a public act and where teachers receive frequent feedback on a well developed instructional framework--it's real time, broad based, and useful.

Value-added measures should incorporate periodic as well as summative assessment--it shouldn't be a surprise to anyone at the end of the year when a third grade classroom fails to make a year of progress in reading. I'm hoping that the $350 million the feds plan to spend on assessment around the Common Core results in a new generation of adaptive online assessment that with better data systems give us much better real time data about student progress. Frequent conversations about a body of evidence should replace 'gotcha' use of end of year standardized tests.

As Rozman points out, this gets a bit more complicated at the secondary level especially outside core subjects but the basic frame of frequent conversations about data and observation feedback should become a regular part of teacher evaluation.

We all know what a difference great teachers make. When it comes to raising student achievement, no school factor matters more than the person standing at the front of the class. And for disadvantaged children in particular, great teachers offer a path to high school graduation and a better life.

Knowing how much teachers matter in the function of an education, it seems obvious that we would do everything possible to ensure all students get the best and only the best. But as The Widget Effect demonstrated, this is not the case. Instead, schools nationwide operate as though all teachers are about the same and essentially interchangeable.

At the heart of this problem is the fundamental failure of our school systems to evaluate teachers accurately and credibly. Unable to distinguish differences between teachers in their impact on student learning, our schools cannot recognize or reward excellence, address poor performance, or give teachers the critical feedback and individualized support they need to grow as professionals. So we find ourselves in a situation where 99 percent of our teachers receive satisfactory ratings, while wide swaths of our students are failing.

If we are serious about changing this paradigm and ending the injustice of educational inequality, we have to focus steadfastly on one overarching goal: providing all students with highly effective teachers. And to do that, evaluations of teachers have to give us an accurate picture of their performance.

For decades, our teacher evaluation systems have relied on rote observations and checklists of teacher behaviors and other factors – such as classroom neatness – that have little or nothing to do with student outcomes. But great teachers are who they are not because of the orderliness of their bulletin boards, the impressiveness of their credentials, or even their years of experience; what makes them great is their consistent ability to advance student learning. Excellent teachers come in many shapes and sizes, but no teacher can be considered effective if his or her students do not show real evidence of academic growth.

How we measure a teacher’s impact on student academic growth will vary. For some teachers, value-added models based on standardized test scores provide useful information, particularly when multiple years of data show consistently outstanding or poor performance.  For most teachers, however, we have to create other measures of their impact on academic growth, such as periodic examinations of student work according to standard rubrics or district or school-designed assessment results.  This will require substantial work – most districts have no tools to measure the impact of art, physical education or even 10th grade social studies teachers on student achievement. 

In addition, it will be important to evaluate teachers’ instructional practice through classroom observations and examinations of lesson plans, though much more frequently than the current “drive-by” standard of once or twice a year for 20 or 30 minutes. 

But no matter which tools we use, we must focus evaluations squarely on a teacher’s impact on student learning. Just as importantly, we have to use that information to recognize the differences between teachers and act on those differences, retaining the most effective teachers at high rates, addressing poor performance fairly and swiftly, and helping all teachers reach their peak potential. Only then will we be able to give all students the chance to learn from effective teachers, year after year.