This week’s question, on teacher effectiveness, touches many of the issues raised in the February 22 query on effectiveness in tough times, so I would like to reiterate some of the points I made in my earlier response and highlight some key elements regarding educator evaluations and educator effectiveness.

Rhode Island has become a leading state in developing partnerships between management and labor, as seen in our Race to the Top application with strong labor endorsement and the development of our educator-evaluation model.

Over the past 18 months, we have been engaged with our educators and in the process of building the Rhode Island Model for educator evaluations. Throughout the process, we have been working side by side with the Rhode Island Federation of Teachers and Health Professionals, which is also developing an evaluation model. We hope to merge our two models into a single evaluation system, as they are very similar. Both of our developing models must comply with the Board of Regents standards and both will contain the components that AFT President Randi Weingarten proposed last year: professional teaching standards that spell out what teachers should know and be able to do; standards for assessing teachers’ practice based on multiple measures, including test scores; implementation benchmarks for putting the system into practice; and support for teachers throughout their careers.

Nobody knows more than educators about how we can make evaluations a truly useful tool, so we have brought stakeholders from across the state together to share ideas and to talk about what an equitable evaluation system will look like. More than 100 educators from 23 districts and organizations around the Rhode Island have participated in developing our evaluation model. In order to provide opportunity for teachers statewide to offer feedback and insight, we have held a series of public forums and hosted webinars on the educator-evaluation model. In addition, over the course of this school year I am visiting every district in the state, as I did last year as well. During each visit, I meet with teachers to hear their ideas and concerns, and I set aside a segment of time specifically for questions and discussion on the educator-evaluation system.

We are confident that we are creating a high-quality educator-evaluation system that will improve instruction by providing teachers with feedback that will enable teachers to grow as professionals. School leaders will use evaluations as a diagnostic tool to bring supports to teachers based on their needed growth areas. The evaluations will also help school leaders make decisions about hiring, assignments, promotion, and retention. This use of evaluations is completely appropriate, in teaching and in any other profession.

These are unprecedented times, as our communities face significant fiscal shortfalls and we all face the urgent responsibility of transforming public education and advancing student achievement. As our municipal leaders wrestle with difficult decisions involving revenues, expenditures, and budgets, the need for improving the quality of public education must remain a top priority. The challenges involved in balancing budgets while maintaining the quality of our schools may lead us to revisit some long-standing practices.

When it comes to hiring teachers, assigning teachers, or letting teachers go because of budget shortfalls, we cannot revert to old patterns and familiar behavior such as last in, first out. Therefore, seniority can no longer be the sole determinant by which school districts make decisions about teacher assignments and budget-driven layoffs in Rhode Island. I have notified all superintendents in our state to remind them that seniority-based policies are in conflict with our Basic Education Program, the regulations that set the standards for our system of public education.

I realize that, for districts facing the need to lay off teachers, there are varying levels of available information on teacher effectiveness as well as existing contracts that have not yet been changed. In these instances, school leaders have to make decisions thoughtfully, basing them on all the existing available evidence and guidance – not just on past practices. In the long term, as our schools develop more and better information about educator effectiveness, the layoff process will evolve and improve. In the short term, school leaders must think carefully about what is best for our students, which is always harder to do than it sounds.

I want to continue to work in partnership with our teachers and their unions throughout this process, and I believe we can. Our entire strategy for transforming schools is built on a foundation of respect for teachers, on the belief that having effective teachers in every classroom and effective leaders in every school is the most important single factor in student success.

We are field-testing our new evaluation system in several schools and districts this winter so that we can learn what works, what we can do better, and how we can best support our educators and our schools as they implement the system. We will rigorously test the evaluation system before its initial launch, and we will constantly improve it based on feedback from educators and other stakeholders. In the end, we will have an evaluation model that supports teachers, ensures educator effectiveness, and helps us meet our goal of preparing all students for success in colleges, careers, and life.

How did we get here? Most teacher evaluation systems have shared a critical flaw: they try to measure a narrow range of things that teachers do rather than what students learn. The evaluation checklist forms so prevalent in today’s schools strive to avoid controversy by focusing on whether teachers complete tasks such as turning in lesson plans and posting standards. While these actions are easily observable, even in a 10-minute classroom visit, there is no explicit linkage to how these tasks improve the instruction students receive each day. The result is an instrument that only takes into account information from a couple of point-in-time observations and offers little guidance for teachers or their principals about how to move teaching and learning forward. The state of teacher evaluations became woeful enough that all parties – educators, national union leaders, parents, and community members – began clamoring for change.

And the good news is as teachers, principals, district administrators, and union officials come together around the country, they are discovering that they aren’t so far apart in their views on good teacher evaluation. They are looking to design new systems that bridge the art and science of teaching and recognize teachers as professionals who have a tremendous impact on what and how students learn.

Designing a better teacher evaluation system begins with a culture shift from evaluation as a one-time event to a continuous performance management process. Talented teachers and leaders are the foundation of every successful school, and cultivating this talent is an ongoing process that requires continuous feedback and support for educators’ growth and development. Viewed through this lens, it becomes clear that a good teacher evaluation system requires a balance of self-reflection, teacher support, and accountability for results. A rigorous performance management system incorporates a variety of perspectives and data points. Student outcomes are one such data point that must be used in concert with other information, which can range from self-assessment to peer review to evidence-based observations of practice, to get a holistic view of teachers’ performance.

Most critical to the implementation of new evaluation system is that teachers understand the performance expectations and are given the support to help them get there. A tenet of good instruction is that students know what they’re being graded on; the same should be said for teachers, from the moment they are recruited to join a school community. At the same time, principals and other evaluators must be accountable for using the tools reliably and fairly and connecting teachers to the support and resources that they need.

While the news media often frame the teacher evaluation movement as a struggle between reformers and the status quo, communities across the country are rejecting the false choice between evaluating teachers and valuing them. The reality is much more complicated. A good teacher evaluation system reveals the critical role teachers play in shaping students’ lives, even as it highlights opportunities for improvement. We still have a way to go but are on the right path to creating performance management systems that nurture talent for the benefit of all—teachers, children and communities.

This is a guest response from Robert C. Pianta, Novartis US Foundation Professor and Dean of the Curry School of Education at the University of Virginia

The central piece of the Obama administration’s efforts to improve education is the selection, preparation, and proliferation of effective teachers in America’s classrooms. The past decade of educational evaluation research (largely fueled by standards-based reform and the availability of test data linking children to teachers) confirms conventional wisdom – that the feature of educational experience that appears to matter most in terms of the value to student learning and development is the teacher. And therefore identifying the features of effective teaching and effective teachers, and then preparing, selecting, and rewarding them are of paramount importance to any effort to improve educational outcomes for students.

American education has done a woefully poor job in rigorously studying, in a reliable and objective and replicable manner, any aspect of teacher preparation or improvement, and then testing, scaling-up, and continuously improving models that work. State governments, professional accrediting agencies, unions, schools of education, and even new alternative models, are all to blame as they work more to protect turf and brand than to push models that stand up to scrutiny. In fact I often think they collude in the creation of the problem rather than producing real solutions. The most common strategy for obfuscation and collusion is to focus on “standards” as the indicator of performance rather than some objective metric (i.e. value-added models of student test scores). Randi Weingarten is championing the American Federation of Teachers model for performance assessment, which has a long list of “standards” for performance. Just like the controversial National Center for Teacher Quality/US News and World report survey of teacher preparation focuses on NCTQ’s list of “standards” for teacher preparation. In any system of performance assessment, its critical to not confuse standards, which are broad statements of desirable features or inputs, with measurement – the standardized, objective indicators of those standards as displayed in actual practice. Our challenge in teacher performance assessment is not that we lack standards; we lack an acceptably broad and valid portfolio of measurements beyond student test scores.

Perhaps the most face valid indicator of teacher performance, student test scores, is not the answer. Why? Student test scores, and whether we post them, pay for them, fire teachers, or reward kids, are simply not, in themselves, a sustainable solution for the lack of effective teaching in too many U.S. classrooms because they do not systematically drive the production of effective teaching. Consider the Los Angeles example: How is posting a teacher’s student test scores supposed to lead to better teachers? Using only test scores as a way to improve teaching is like telling an orthopedic surgeon performing a hip replacement, “Good job, the patient lived” or “bad job, the patient died.” Mortality is important to the surgeon, just as student achievement is important to a teacher. But it carries virtually no information on how to do hip replacement surgery any better the next time (and if its your kid in a teachers’ classroom, you want the teacher getting better that year, not the next). It’s the same with test scores and teaching reading to 30 seven-year-olds. There is no doubt that our nation’s educators and children are desperate for better information on the techniques teachers can use to foster achievement, problem-solving, getting along with other kids, and a host of other things we value. We even more desperately need ways to systematically and predictably produce those behaviors in the 350,000 new teachers who enter the profession each year and the several millions more already in classrooms.

It is argued my many that the problem is selection – that if we only found a way to recruit and incent the best and the brightest, then our classrooms would be filled with effective teachers. There are at least three problems with relying on selection as the mechanism for delivering putatively effective teachers to classrooms. First, remember that in any selection system not all candidates predicted to be good teachers will be good teachers; not all Teach for America corps members are great teachers, just like not all graduates of any university-based preparation program are great teachers. Second, selection as a solution relies on the hope that there are is a supply chain capable of producing a sufficient number of effective teachers to staff America’s classrooms, and that those teachers supplied by that chain will stay in the classroom long enough to offset attrition. But the fact is that hundreds of thousands of new teachers are needed each year to staff schools. Third, even if we had a full supply chain of effective teachers-in-waiting, there are millions of teachers already in the nation’s classrooms whose performance could be augmented, and improving the performance of those in the classroom and ensuring our current infrastructure of teacher preparation does a much better job is perhaps as or more effective and efficient than relying in hiring and firing as the sole strategy.

Our and others’ efforts to develop, validate, and implement measures that objectively observe teachers’ classroom behavior as the core of a system for fostering more effective teaching are showing some promise. In fact, experience in early childhood education settings (e.g., Head Start and pre-kindergarten programs) has clearly demonstrated that teachers’ performance in classrooms can be observed –at-scale (e.g., state- and even nationwide) using standardized observational assessment tools, and that when those observed metrics are used in accountability or professional development frameworks, they drive improvements in teachers’ classroom practices that result in learning gains. The implementation of these systems in preschool settings, which typically lack the capacity of k-12 systems, is often messy and challenging, but nonetheless a clear demonstration of their potential and perhaps greater impact, in k-12.

Effective teaching is not an enigma. Rather, effective teaching consists of behaviors and interactions and relationship skills – teacher actions that can be objectively observed and perhaps more importantly, produced and improved in ways that lead to increases in student learning. More than a decade ago we set out to study scientifically the behaviors and interactions of teachers in classrooms; we attended to social and emotional features of teachers’ behavior (like warmth and contingent timing of responses), how teachers’ behaviors organize the classroom in relation to time and tasks, and how teachers’ interactions with students stretch them conceptually and provide the type of feedback that produces new levels of performance. We only watched teachers’ behavior and we codified our observations objectively, ensuring that others’ would make the same judgments, and we tested whether our observations related to student learning gains, and they did.

We are not the only ones to do this work. In fact there is a reasonable body of evidence, that teachers’ performance in classrooms, in terms of their actual behavioral interactions with students, can be assessed observationally using standardized protocols, analyzed systematically with regard to various sources of error and in turn shown to be valid for predicting student learning gains, changed (improved) as a function of specific and aligned supports provided to teachers, and that exposure to such supports is predictive of greater student learning gains. Although modest in terms of effect sizes in the general population, these effects are robust and consistent across investigator groups, samples of teachers, and samples of students that vary by grade and socioeconomic and geographic background. Moreover, effect sizes in relation to student outcomes generally double in groups characterized by risk, signaling their sensitivity to teacher-student interactions, and professional development aimed at improving performance on these observational metrics have shown effective in also improving student outcomes.

A major advantage of observational assessments of teachers for leveraging improvements in educational outcomes is that they can be the focus of professional development aimed at improving teaching – coaching, courses, videos that can be used to produce those very behaviors shown to predict student learning. Moreover, actual measures of teaching (rather than the “standards” promoted by AFT or NCTQ) could be embedded as performance metrics in teacher preparation programs, credentialing and licensing systems to anchor those systems to actual performance and by so doing drive them toward higher levels of impact on student performance (providing the observation system is validated). Furthermore, if these systems used observations of performance as an outcome or competence metric, it is highly likely that a market would emerge in the development of collateral professional development models oriented to producing performance at levels specified in licensing and credentialing systems. Finally, performance-based metrics anchored in observations in classrooms may also be compatible with market-based approaches to incentivizing performance such as merit pay structures.

As I mentioned earlier, the early childhood education sector is right now doing all this. Standardized observational assessment of teacher-child interaction is one basis for defining program quality in Head Start. Dozens of states are implementing Quality Rating and Improvement Systems, in which observations are part of a set of indicators of quality. In both Head Start and QRIS stakes are attached to observations – both incentives and sanctions. And as expected, these frameworks are now driving focused, aligned professional development designed to improve teachers’ classroom interactions. What we have learned in the early childhood work is the absolute importance of standardization, quality controls, and the need for training so that there is sufficient capacity to implement well and with fidelity.

Is anyone else uneasy about the way we're oversimplifying this discussion about teacher-effectiveness?

Listening, you'd think it were an absolute: that a teacher is either 'good' or 'bad'; effective or ineffective. But I remember an elementary principal explaining: It depends. Some teachers work well with bright students but are impatient . . . ineffective . . . with those who need more time. Others teachers are good with students who are struggling and feel those doing better will be OK on their own. One year he announced he was going to let parents choose their children's teacher. Teachers were terrified that that nobody would walk in their door on sign-up night. In the event: Not a problem. Teachers and parents matched up the children just fine.

So: Not simple.

Which leads to another puzzlement about this discussion. It clearly assumes traditional 'teaching', doesn't it? . . . the teacher facing the class. Everybody talks about "class-rooms". But now, as Tom Vander Ark and others have been telling us, the old technology of teacher-instruction is about to make way for new digital technology. 'Digital' personalizes school -- or certainly should. And personalization changes the pace of learning: Students who need more time get more time; those who can go faster do go faster. Consider what this means for the 'class'; for group-instruction; for the old model of "delivering education".

And in this new world, what's 'teaching'? What will 'effective' and 'ineffective' mean when we move away from the old technology of teacher-instruction?

Not simple.

Needs thinking.

Leaders in many school districts across the country earnestly want to improve their results and better serve their children. In Memphis, we are in the midst of an unprecedented and innovative push to ensure all of our teachers employ best practices in the classroom. The Teacher Effectiveness Initiative, made possible through funding from the Bill & Melinda Gates Foundation and the Memphis community, will create a comprehensive approach to evaluating teachers that is fair, accurate and objective.

Through this initiative we will create objective measures of effective teaching and provide our educators with appropriate feedback and guidance, thus guaranteeing that every student will benefit from teachers who are armed with the skills and knowledge they need to foster a love of learning and improve academic achievement.

Our entire community—children, parents, the elderly, businesses, and churches—will reap the benefits of this enormous effort. Watch us as we rally together to permanently transform our schools.

Secretary Duncan, Bill Gates and Randi Weingarten are all correct in their focus on the critical need to do a better job of linking teacher evaluation to teacher effectiveness. The question remains, how to fairly and accurately evaluate teachers relating to the complexity of student learning. Effective and fair teacher evaluation systems must at least include clearly defined standards of expected performance; meaningful differentiation of teacher effectiveness based on multiple evidence of student learning and classroom practice; rigorous training of evaluators; and be tied to individualized teacher feedback and professional development. The rigorous National Board assessment process not only identifies effective teaching but helps teachers to get better. The National Board Certification process is a scalable evidence- and performance-based measurement of effective classroom practice that reaches across all content areas and developmental levels. With more than 20 years experience and expertise in assessing effective classroom practice, the National Board process includes, but is not limited to, evidence of student learning; its standards and process are comparable across schools and classrooms; and only rigorously trained evaluators are used to identify and distinguish effective teaching. The challenge now is how to take this proven assessment model to scale and build a system in which all teachers are encouraged to put into use the standards of professional practice developed by the National Board. In states and districts throughout the country, hundreds of teachers are holding themselves to a higher standard by going through this rigorous process. Districts like Montgomery County, Maryland, are already using the National Board framework in the hiring, evaluation, and professional development of its teachers. On Thursday of this week, in a national webcast, the National Board will be releasing a new report, Student Learning, Student Achievement: How Do Teachers Measure Up? The report, written by an independent task force, includes such leaders in education evaluation, research and policy as Linda Darling-Hammond, Rick Hess, Lee Shulman and others. We anticipate that this report will break new ground in identifying the most effective ways to ground teacher evaluation in student learning—and also identify how the National Board Certification process can be used more effectively.

I doubt there is anyone who thinks current teacher evaluation is adequate. The weaknesses have been well known for some time. Good ideas for improving teacher evaluation, such as Charlotte Danielson’s work, have been around for decades.

What is happening now, however, is one more effort to claim a crisis, then stampede people into accepting whatever the proclaimer of “crisis” wants them to do. Through Race to the Top, Education Secretary Duncan pressured states to use student test scores as a significant factor in evaluating teachers.

This process is the equivalent of: “Fire” (teachers), “Aim” (prepare adequate evaluation systems), “Ready” (carefully evaluate existing evidence and good examples of best practices). The upside-down process caught on: several governors have used state fiscal crises to push for immediately changing how layoffs take place..

If we start with the evidence, we see that using student test scores, including mislabeled “value added measurement” (VAM) procedures, will massively disrupt the teaching force and educator morale in many states, without putting anything in place that stands up to reason or evidence. (I recently summed up some of the available data in a short piece for Mass. Citizens for Public Schools here. The basic conclusion: VAM is not ready for prime time.)

In addition to technical flaws, the larger problem is that evaluating educators primarily based on their students’ scores will lead to even more teaching to the narrow tests than we now have. Since inducing such academic damage is one of the major failures of NLCB, it makes no sense to make that problem worse.

The public is recognizing the absurdity of such politically-driven initiatives. In a recent Vanderbilt University survey, two thirds of Tennessee residents opposed paying teachers for their student test scores. The pollsters thought concerns over too much emphasis on testing produced that result. Sadly, the state had already adopted such a scheme.

Richard Rothstein points out clearly that the claims of crisis underlying the push to introduce payment for scores are far more manufactured than real. I would add that gains in NAEP have slowed or halted in recent years, quite possibly a consequence of the over-emphasis on testing. Slowing NAEP gains provide another reason not to extend the flawed logic and practice of NLCB.

Yes, education gaps remain large by many measures. But as “60 Minutes” pointed out on Sunday, March 6, soon one quarter of U.S. children will be living below the official (and inadequate) poverty line. Nations whose students outperform the U.S. have far lower rates of child poverty. Finland, often cited as global model, has a poverty rate of less than five percent. Closing the “poverty gap” would be a far more productive place to start in addressing the root causes of our nation’s mediocre educational performance. .

Yes, schools can and should improve, even in light of huge opportunity-to-learn inequities. Improvement, however, takes thoughtful action, not perpetuating the errors of NCLB and not pushing states to plunge into more test-based decision-making using bribes and manufactured crises.

Quality teaching starts at the beginning: nobody should enter a classroom unless they are truly qualified, and mentoring for new teachers should be the rule, not the exception. And we need better evaluations. Too often teachers themselves are the one group that never seems to be asked for an opinion on this critical issue. It’s a damaging omission, because teachers know better than anyone what constitutes effective teaching – and they also are aware of the type of feedback that will help them improve at their jobs. Teachers want real evaluations, but we must remember that the purpose is to improve practice, not punish. We need to get a more complete and nuanced picture of what teachers are doing, and no teacher assessment should rely primarily on a few bubbles on a standardized test.

It's time to open the door to a new conversation about how to hire only the truly qualified, support teachers throughout their careers with quality evaluations, and build a teaching profession that ensures that no unqualified person is allowed in a classroom, and no qualified person is removed unfairly for arbitrary or political reasons. NEA affiliates across the country are among the leaders in this area. The Massachusetts Teachers Association recently proposed comprehensive teacher evaluation reforms, and the Illinois Education Association has proposed reforms to evaluation and dismissal procedures. Attracting and keeping great teachers in America's classrooms by revising teacher evaluation is also the focus of all eleven states that received Race to the Top grants and is work in which many NEA state affiliates are deeply involved.

Getting the evaluation framework right is critical to the long term success of our schools and students. Rigorous evaluations that provide teachers with meaningful feedback on their work and that are applied fairly and consistently are the precursor to a quality fair dismissal system. If we want quality teachers, we need to create a system that develops them.

Can we develop systems to rate teaching quality? Can we use these systems to make hiring and firing decisions? Of course we can. The quality of virtually anything can be measured in some way. And data on quality can be used to make decisions any way the deciders want to make them.

But the Devil is in the details. And Arne Duncan, Bill Gates, and a lot of other smart people in this debate would come off even smarter if they acknowledged the inherent complexities of this process—and offered solutions instead of just commandments.

The technical aspects of this process are relatively simple. But three “human” factors deserve much more attention than they are receiving right now.

The first problem is the development, trial, and validation of the evaluation instruments. Even existing instruments probably haven’t been studied long enough yet for us to know if they work very well. Money could speed this up. Much of it could come from philanthropists. But even with all the money in the world, we still measure these kinds of studies in years, or even in bunches of years. This is longitudinal work and if we want it to work well, we can’t rush it.

If the first problem is the creation of the instruments, the second problem is surely our lack of experienced and talented instrumentalists. Our education system has almost no capacity at this time for providing high-quality teacher evaluation. How could it? We’ve never done high-quality teacher evaluation, so there isn’t a generation of school leaders who have mastered this task. Get them good tools. Give them training and time. And then we’ll see what happens.

Finally, we have a human capital supply chain issueto deal with. Bottom line: we just don’t attract and retain enough talented, committed, career-oriented classroom teachers each year. TFA doesn’t count, nor do most other alternative certification programs, because these programs tend not to retain high percentages of teachers in the classroom for career-length intervals. Neither do traditional certification programs, so this is a problem we have to address in virtually all forms of teacher training. It doesn’t matter how great a teacher is if she decides not to teach after just a few years.

In a decade, we could have evaluation instruments that we could have some confidence in. Ten years after that, we might have a nation of decent evaluators. But if we don’t fix the human capital supply chain problem, all the evaluations will show twenty years from now is what we already know today: most teachers aren’t well-trained; most don’t get that much better over time; just measuring and firing people isn’t workable because replacing poor teachers with even average teachers is very hard when you have a human capital supply chain problem.

We’ve all finally gotten around to agreeing that good teaching matters. But let’s not forget that it took us more than a generation to construct this very obvious idea. Having taken all that time to agree on something so simple, does it seem logical that we could create something infinitely more complicated in significantly less time?

We’re so angry right now with teachers that we’re obsessed with short-term, punishment-oriented, tactical approaches. We’re not thinking strategically. We’re not looking at the entire teacher quality system. We’re not looking seriously at teacher quality on a national level. If good teachers are important in Massachusetts are they any less so in Mississippi? This is a straight up federal issue. But the states’ rights crowd will never go for national management of teacher quality. And neither will the unions.

So sure, we can devise all manner and sort of teacher quality systems. But let’s not forget that this is merely a means to an end. The systems have to show that they improve teacher quality. And improving teacher quality means making dramatic changes in the way millions of teachers teach. As the old saying goes, “The impossible takes a little longer.”

I’m all in favor of tackling this problem head on. But I want to do it smart. I want to do it right. And I want to do it for everyone. If teacher quality is the #1 in-school factor in student academic success, why is not even close to the #1 national issue in federal education reform? If Secretary Duncan is going to give his teacher quality stump speech until the end of his term (and he probably is), how about tossing in some realistic solutions and executable plans every once in a while? Then he’ll have something worth talking about and we’ll have something worth listening to.

Until solutions come to dominate the discourse, the power players are just blowin’ smoke. And it’s hard enough to breathe in this debate already.

The common theme running through many (too many) teacher evaluation proposals is time. We need time to create new evaluations. We need time to observe a teacher (after taking the time to build them up). We need time to create a plan based on our observations. We need to give them time to prove they can get better (or not). We need time to figure out if they should be doing something other than teaching.

The problem with ’borrowing time’ is that no one wants to quantify what that means – how much we need, how soon, and whether we really even need more to begin with.

Before ‘Race to the Top’, states grappled with the notion of paying teachers based on performance, and some attempted modest measures, but most fell short. ‘Race to the Top’ further encouraged evaluation systems, but guidelines conveyed no urgency and states needed simply to promise changes. Evaluation systems adopted have proved fuzzier than many originally thought. Now with budget struggles in states and more understanding that first-hired/last-fired policies actually harm kids (what a discovery!), state lawmakers on both sides of the aisle are pushing hard to put hard, firm measurements with consequences in place.

But all we hear from the education groups and their defenders is give us time. The AFT’s Weingarten says she wants to help get it right. That takes, in her words, “a valid and comprehensive system of teacher development and evaluation in place” before a district can move on tenure and contract reform. Many nod their heads and smile. “Randi gets it now,” they say. Really?

Here’s what time actually looks like in Chicago. The Chicago Tribune has produced a graphic that shows how it could take as many as 27 unique steps – and from two to five years – for a teacher to be fired there. Is that acceptable? To parents? To principals? To fellow teachers? To you?

http://www.chicagotribune.com/news/opinion/editorials/ct-edit-0230-cps-dismissal-gfx.eps-20110226,0,3378793.graphic

Put aside the warm and fuzzy you feel when someone says we can’t risk evaluations being unfair. Consider that right now in the US, assessments exist that can be adopted and start tracking the performance of a student when she first walks into the classroom, and when she leaves in the spring. We can look at multiple measures for performance, right now. We can measure growth in a classroom under a given teacher’s watch right now, and file it away in a database that can be used in myriad ways to, at least, begin to assess whether this teacher shows trends of success, or failure. We can aggregate success across the school and help superintendents measure how the principal affects the rest of the school’s performance. And we can aggregate the data, school by school, and help manage expectations of district level leadership. It’s already being done in some programs around the country. Look at the Broad Prize, for one example, or the Teacher Advancement Program for another. The tools, the will, the demand exists.

What are we waiting for?

My blogging colleagues have it right. As Richard points out, “Education is complex, and the relationship between education and the economy even more so.” Renee reminds us of the need for due process in the teaching field—something that is separate from but quickly becomes entangled in the talk about teacher evaluation. (Anyone who has been around schools long enough has witnessed a version of the math teacher/basketball coach who comes under fire for not playing the school board member’s son in the big game.) Peggy rightly calls into question the purpose of evaluation systems. Are they for firing teachers or professional growth? Cross purpose leads to confusion.

The reality is that we are all victims of a system that is obsolete. We can continue to tinker with the parts but we won’t arrive at any meaningful solutions until we address the learning crisis in the world. Yes, the world. As far as I can see there are pockets of people getting this learning thing right, but not nations.

Forgive me for speaking plainly but test scores do not measure the kind of learning that translates to creative engagement and long term productivity—not here or in China, India or Sweden. Test scores do not measure or begin to equate with teacher effectiveness. High-test scores are not the formula to a strong economy. So let's be sure think more deeply about this challenge.

Each school is a unique learning community with individual learners that need customized approaches. In almost thirty years of working with teachers, I have yet to see a teacher improve due to an evaluation system. In fact, these systems are often catalysts for low performance. One of the reasons for this is because to an administrator, effective teaching is about both attitude and performance. Today’s leaders are flummoxed about how to address these separate issues using a system that bundles it all together and often sidesteps the real concerns leaders have about teachers in their schools.

Faculty members usually fall into one of four categories:

1. Strong Performer/Positive Attitude 2. Strong Performer/Negative Attitude 3. Weak Performer/Positive Attitude 4. Weak Performer/Negative Attitude

The administrator’s greatest challenge is in evaluating the strong performer with a negative attitude.

Weak performance usually falls into two categories: unprofessional behavior (such as arriving late for work, not completing lesson plans on time, missing meetings). These are accountability issues and principals can easily deal with these issues. The second area in the weak performance category is task ability (classroom management skill, ability to create meaningful lesson plans, content-area knowledge, ability to innovate, understanding of students and methodology). In order to improve in these areas, teachers usually need some kind of professional growth such as courses, training or coaching. Put another way, unprofessional behavior has to do with basic job expectations that are found across all fields, and task performance denotes the behaviors that are specific to the industry. If these two categories were all principals had to contend with then their evaluation programs would be easy. However, this is not the case. Strong performing teachers with positive attitudes need to be challenged to learn more and remain current in their fields. And negative attitudes must also be addressed. Teacher attitudes matter as much as much as performance skills-- any administrator will attest to this.

Teacher attitudes effect both the classroom as well as the overall school culture and are much more difficult to diagnose and correct. Positive attitudes and weak performance stand a good chance for improvement. Negative attitudes can bring a school to a stand still. Teachers who distrust students, refuse to implement new ideas, bad mouth the administration, are suspicious of parents—it doesn’t matter if these teachers can yield high test scores, they can drive otherwise engaged students and teachers from the school. An evaluation system that can be honest, even courageous in the face of this kind of teacher must work beside one that aims at task improvement or task advancement.

Although I have developed systems that address all these issues, the point here is that as with every issue that touches education: there is no silver bullet to “fix” it.

Each school has a unique culture with different protocols and expectations.

What we can do is begin to rethink the models we currently use in schools. Most schools use the “gotcha” method for teacher evaluation. That is, the administrator shows up unannounced for an annual evaluation and doesn’t always know what he looking for or how to coach a teacher, so they look for what is wrong rather than what is right about the teaching. This system breeds discomfort and suspicion. New systems may include several strands that address different kinds of teachers in schools. Sound familiar? It is the same thing many educators advocate with learners: one size does not fit all.

A hefty part of what is wrong with our schools today is that we have stopped seeing them as learning communities. When the systems approach is removed from our viewpoints of schools, we turn them into bastions of blame and fortresses of fear. Teacher effectiveness cannot be separated from leadership the same way that student achievement is inseparable from teacher effectiveness. Education is a system and as long as we address its parts in isolation we will continue to see weak performance and negative attitudes.

Not only are most teacher evaluation systems broken, they have never made much sense. For too long there was an assumption that all teachers are equally effective and that students either “got it” or they didn’t because they weren’t smart enough or were unmotivated. How else to explain the decades long use of the single salary scale based on years of service and graduate school credits? Not until the 1990s did policymakers and advocates begin to give serious attention to student outcomes. Before that the primary focus was on resource inputs, which were then—and sadly remain today--quite inequitable.

NCLB led to the major changes in the conversation about teachers. With every grade testing between grades three and 8, the reporting of school results by major subgroups, and in at least a few states the early data linking of teachers to their students, researchers had a wealth of data over several years to work with. They documented conclusively that the quality of teachers is the major in-school factor affecting student achievement and that there is a wide range in teacher effectiveness not only among schools, but within them.

So if teacher effectiveness varies so widely, why are all teachers with the same years of service and credentials paid the same? Policymakers, advocates, and educators including teacher representatives have come to realize that this makes little sense. Consequently, conversations began on how to differentiate teacher compensation with variation based not only on performance, but also additional responsibilities and incentives to work in hard-to staff schools and subjects. Then decision-makers ran up against a virtual brick wall—the lack of meaningful, useful evaluation systems.

As a result, districts and states have been scrambling to put in place better evaluation systems that include consideration of student achievement gains as one of a number of measures of teacher effectiveness. This is a work in progress and much good development work, i.e. experimentation and innovation, is underway across the country.

Of course, to factor in student gains, there has to be an examination of student testing results. State tests are of widely varying quality, and they can be used for maybe 30% of teachers at best. (Better state tests may appear in four or five years.) Other local tests and evidence of student learning can be added to the mix. In addition many districts and states are revamping the observation processes of teaching to include use of videos. And researchers are making great progress in using student surveys as a reliable measure of teacher performance.

We will never have perfection when it comes to measuring teacher effectiveness, and anything approaching great may take a decade. But that is no reason not to move ahead now. The imperfect systems coming on line today are better than the meaningless status quo of teacher evaluation to date.

Indeed, with budget cuts looming in most districts, educators desperately need a way to determine which teachers to let go other than through seniority. Since high poverty schools have large proportions of newer teachers than low poverty schools, releasing those with the least seniority will hurt the most those students who need the best teachers AND save less money. Researchers have documented that teacher effectiveness in general peaks at three to five years, not with years of service. Many newer teachers who are effective are likely to be fired.

Two recent papers by James Wyckoff and colleagues and Dan Goldhaber that simulated layoffs using data on teacher value-added to student learning show that (a)you keep more teachers, and (b)you get more learning gains under the simulated arrangement. While the evidence is not a reason to do value-added based layoffs, it does beg questions about moving aggressively to bring performance evaluation data into the picture as soon as possible and to consider other policies in the meantime (e.g. salary reductions, expansion of class size for effective teachers with salary enhancements).

On March 3, 2011 Secretary Arne Duncan on the National Call on Flexibility and Productivity stated, “We're challenging states and districts to use teacher effectiveness in the classroom as a factor in teacher layoffs. Districts should not let go of effective young teachers because it's the easiest path and they should not let go effective, higher-paid veterans to save money.”

This would be ideal if we did not have a teacher evaluation system that is broken. We cannot talk about teacher quality without considering principal quality and the quality of the evaluation instrument. Currently vast numbers of educators are evaluated by an instrument that is broken and by individuals who do not have time to effectively use the evaluation tool that currently exist. Countless of these instruments do not include critical Value Added components.

The intent of the evaluation tool needs to be a clear. Is it a tool for dismissal, or understanding how to strengthen and develop educators and the profession? If the tool is used for the latter, there is a need to expand it to look at factors that support and impede effectiveness.

This tool should give educators and the public a realistic look at students, testing, teaching assignments, interventions, and make clear the penalty for poor performance. The data such an instrument would provide could be used to determine the best use of the teachers’ talents and the most effective professional development for enhancement or improvement. Such an instrument does not presently exist.

Many states are currently in a race to implement new teacher quality legislation without realizing the time and expertise required for this task. A number of states think a fair, accurate, and valid instrument can be developed and implemented in a matter of months. Very few, if any, states have the resources or personnel to develop a scientifically valid, comprehensive evaluation tool. I think we can learn from the National Board of Professional Teaching Standards in this area. These standards and the accompanying assessments took years to create, field test and refine. My hope is that the National Board’s process of standards development is used to create an effective teacher and principal evaluation tool.

Last week, Microsoft Chairman Bill Gates published an op-ed in the Washington Post, “How Teacher Development could Revolutionize our Schools,” proposing that American public schools should do a better job of evaluating the effectiveness of teachers, a goal with which none can disagree. But his specific prescriptions, and the urgency he attaches to them, are based on the misrepresentation of one fact, the misinterpretation of another and the demagogic presentation of a third. It is remarkable that someone associated with technology and progress should have such a careless disregard for accuracy when it comes to the education policy in which he is now so deeply involved.

Gates’ most important factual claim is that “over the past four decades, the per-student cost of running our K-12 schools has more than doubled, while our student achievement has remained virtually flat.” And, he adds, “spending has climbed, but our percentage of college graduates has dropped compared with other countries.” Let’s examine these factual claims:

Bill Gates says: “Our student achievement has remained virtually flat”

The only longitudinal measure of student achievement that is available to Bill Gates or anyone else is the National Assessment of Educational Progress (NAEP). NAEP provides trends for 4th, 8th, and 12th graders, disaggregated by race, ethnicity, and poverty, since about 1980 in basic skills in math and reading (called the “Long Term Trend NAEP”) and since about 1990 for 4th and 8th graders in slightly more sophisticated math and reading skills (called the “Main NAEP”).[*]

On these exams, American students have improved substantially, in some cases phenomenally. In general, the improvements have been greatest for African-American students, and among these, for the most disadvantaged. The improvements have been greatest for both black and white 4th and 8th graders in math. Improvements have been less great but still substantial for black 4th and 8th graders in reading and for black 12th graders in both math and reading. Improvements have been modest for whites in 12th grade math and at all three grade levels in reading.

The following table summarizes these results, for the earliest and most recent years for which disaggregated data were collected.

The Main Assessment is not shown for 12th grade because it was not administered until 2005.

Source: NAEP Data Explorer: http://nces.ed.gov/nationsreportcard/naepdata/

We can see that in 4th grade math, black students now have higher average achievement than white students had when the assessments began. Average black students’ gains have been a full standard deviation, a rate of progress that would be considered extraordinary in any area of social policy. The black-white score gap has narrowed some, but not very much, because white students have also shown improvement.

Bill Gates may think that these improvements are insufficient, and perhaps he is correct. But, as Daniel Patrick Moynihan reportedly quipped, “everyone is entitled to their own opinions, but not to their own facts.” No rational reading of these NAEP data can support Bill Gates’ claim that “student achievement has remained virtually flat” over the last four decades.[†] And, to repeat, no other longitudinal data are available that describe student achievement over time.

These facts also don’t support the story that the typical teacher of disadvantaged children is ineffective. Certainly, some teachers are ineffective, and schools should do a better job of removing them. But that should not, if facts are to be believed, be the main story.

Yet it seems to be. Secretary of Education Arne Duncan recently asserted that “many, if not most, teacher-training programs are mediocre.” This may be true, but how does he know? What is his evidence? It wouldn’t seem that mediocre teacher training programs could consistently be turning out teachers who have posted the kinds of gains we’ve seen on NAEP in the last generation and more.

It is important to investigate why, in the most recent period, typical teachers have been more effective with elementary school children than with high-schoolers, but curiously, the reforms Bill Gates and like-thinking policymakers are pursuing concern elementary school teachers almost exclusively – because the student value-added scores on NCLB-required standardized tests by which they propose to evaluate these teachers are available only for elementary, not secondary school students. It is also important to investigate why teachers have apparently been more effective during most (though not all) of the last few decades in teaching math than reading, but it is difficult to motivate anyone to investigate this if our vision is clouded by the myth that all student achievement has been flat.

Bill Gates says: “The per-student cost of running our K-12 schools has more than doubled.”

Here, Bill Gates is nominally correct, but misleading. When properly adjusted for inflation, K-12 per pupil spending has about doubled over the last four decades, but less than half of this new money has gone to regular education (including compensatory education for disadvantaged children, programs for English-language learners, integration programs like magnet schools, and special schools for dropout recovery and prevention). The biggest single recipient of new money has been special education for children with disabilities. Four decades ago, special education consumed less than 4% of all K-12 spending. It now consumes 21%.[‡]

Detailed tables documenting these trends are available here: http://epi.3cdn.net/1726cc68ca1a71563a_o3m6bhrub.pdf

American public education can boast of remarkable accomplishments in special education over this period. Many young people can now function in society whereas, in the past, children with similar disabilities were institutionalized and discarded. But it is not reasonable to complain about the increase in spending on such children by insisting that it should have produced greater improvement in the achievement of regular children.

The increase in regular education spending has still been substantial, even if not nearly as great as Bill Gates implies. Should this spending increase have produced even greater improvement in achievement than has in fact occurred? This is a more difficult judgment to make. But in light of the actual achievement improvements documented by NAEP, it is not reasonable to jump to the facile conclusion of a productivity collapse in K-12 education. A more reasonable story is that spending has increased and achievement has increased as well. Perhaps we have gotten what we paid for.

Bill Gates says: “Spending has climbed, but our percentage of college graduates has dropped compared with other countries.”

This is the Bill Gates claim that can properly be called demagogic. It attempts to agitate readers by presenting a positive development in a negative light. A climb in spending should produce an increase in the percentage of college graduates. And it has. In the last four decades, the percentage of college graduates in the United States has nearly doubled. In 1970, 16% of young adults (ages 25 to 29) were college graduates. Today, it is 31%. The improvement has been across the board: the share of African-American young adults who are college graduates has gone from 10% to 19%; for whites it has gone from 17% to 37%. Somehow, Bill Gates saw fit to present this as an indictment.

Should our college graduation rate be rising faster? Of course, that would be a good thing. Should the spending increases we have experienced have generated a faster increase in college graduation than, in fact, they have? That would be worth exploring, but Bill Gates’ phrasing suggests to the less-than-careful reader that spending increases haven’t been productive at all, because our college graduation rate has “dropped…” Would a faster increase require even greater increases in spending? That is also likely, but it is not a conclusion that Bill Gates intends to suggest.

It is commonplace to imply, as Bill Gates does in his Washington Post op-ed, that our failure to increase our college graduation rate “compared with other countries” will prevent us from “build[ing] a dynamic 21st-century economy.” Certainly, we need a sufficient number of well-trained college graduates for such an economy, but there is no reason to believe that a graduate rate in excess of 30% is too small for this purpose, or that economic dynamism can, after reaching sufficiency, increase linearly with increases in the share of young people who graduate from college. The threats to a dynamic 21st century economy are likely to come from a failure of macroeconomic policy, regulation of speculation, and investment in education, not from inefficiency in the investment we already make.

We only need to examine the list of international college graduation rates to see the absurdity of efforts to make a direct link between college graduation rates and economic success. The Organization for Economic Cooperation and Development (OECD) publishes comparative data. One country that outranks the U.S. in college graduation rates is Ireland, whose economy has now collapsed because its regulation of the real estate bubble was even more careless and corrupt than ours. Another is Portugal, whose economic health is also worse than that of the U.S. Of course there are also nations on the list that are not on the verge of bankruptcy, but the chief lesson of the list is this: provided a nation has a sufficient number of college graduates for a dynamic economy, rankings above that point are irrelevant. Of course we should increase our college graduation rate, and there are many civic and cultural reasons to do so, even if we may already produce (as some analyses suggest) an apparent surplus, for economic purposes, of science, technology, engineering, and math graduates.

Education is complex, and the relationship between education and the economy even more so. Our ability to grapple with the challenges these present is not enhanced by factually inaccurate and hyperventilated appeals from those who should know better.

Endnotes

[*] In theory, the Long Term Trend (LTT) is distinguished from the Main Assessment because the LTT assesses the same skills, whereas the Main Assessment changes over time, as the curriculum changes. But in fact, the LTT also changes somewhat over time, and the Main Assessment is sufficiently stable to make longitudinal comparisons.

[†] If the data are further disaggregated by decade, there have been some interim periods of flatness within the overall growth. For example, gains were strongest for black elementary students in the LTT in the 1980s and 2000s, and flat in the 1990s, but on the Main Assessment they showed strong gains in the 1990s as well. Twelfth grade LTT reading scores have been mostly flat since 1990, after a dramatic leap of 24 scale points for blacks in the 1980s. Fourth grade LTT reading scores fell for blacks in the 1980s, but rebounded in the 1990s and jumped even more strongly in the 2000s. I will post tables showing these disaggregated data at the website of the Economic Policy Institute shortly. Readers who want copies sooner can request them from me at rrothstein@epi.org

[‡] Detailed tables documenting these trends are available here: http://epi.3cdn.net/1726cc68ca1a71563a_o3m6bhrub.pdf

Secretary Duncan, Bill Gates, and others’ calls for school districts to put more emphasis on teacher quality than seniority in the laying off or rehiring of teachers highlights ongoing contradictions in our national attitude towards teacher evaluation and teacher quality.

A major reason teachers turned to unionization, collective bargaining, and tenure was the notoriously capricious, subjective, and often discriminatory nature of teacher evaluations. Administrators could get rid of any teacher for any reason or no reason. Not surprisingly, this led to widespread abuses ranging from nepotism to retaliation. Today, most teachers have obtained the right to due process, so administrators need actual evidence of incompetence or misbehavior to dismiss a teacher.

Ideally, such evidence should come through a district’s established annual evaluation process. The process is supposed to go like this: A pre-conference between teacher and principal to discuss aspects of the teacher’s work that cannot be observed inside classroom (including review of student achievement data if available). An administrator observes the teacher in the classroom. Finally, and very important, a post-conference at which both parties summarize the observations, identify strengths and weaknesses, and set goals for the next year.

In reality, we too often have evaluations on paper only. Many principals are so overwhelmed or so under-trained, they never give each member of the faculty a full evaluation. There were many years as a high school teacher that no building or district administrator entered my classroom. My main reason for pursuing National Board Certification was to get a full, rigorous, peer evaluation of my teaching, so I could learn where I really needed to improve my teaching…and how.

In fact, among the most essential resources for developing teacher evaluations are the rigorous, comprehensive, scientifically developed, and field-tested standards produced by the National Board of Professional Teaching Standards. NBPTS now offers certification in 25 areas including: special education, music education, and guidance counseling, as well as the new certification for principals which examines how well they evaluate teachers. Want to know how effective teaching looks? Study any one of these standards.

For 20 years, NBPTS has identified highly accomplished teaching. President Obama, Secretary Duncan, along with Republican national and state leaders have extolled National Board Certification as the gold standard of teacher quality. In 2008, the National Research Council issued a comprehensive, Congressionally-mandated study that found National Board Certification has had a positive impact on student test scores and teacher retention.

How much more ironic then that rather than expand support for what tens of thousands of America’s best teachers insist is the best professional development opportunity they ever had, the House has stripped National Board of its federal funding in the most recent Continuing Resolution for fiscal year 2011. In December 2008, Congress passed and the President signed another Continuing Resolution that reduced the requirements for a person to be labeled as “highly qualified” to teach.

The Federal government is setting precisely the wrong example for states and districts on how to make budget choices based on teacher quality.

Improving teaching effectiveness at scale requires transforming the profession as a whole. Often the current discussion on teaching effectiveness implies that teachers either have “what it takes” or they don’t, and they will magically improve given the right carrot and stick. Yes, the carrot and stick are important, but alone they cannot hope to fix an outmoded system centered on teachers working in isolation. What has to change to secure an environment that attracts, supports and retains effective teachers?

Imagine a school system where each student experiences individual teachers as part of an integrated team that collectively brings a breadth of skills, knowledge and experience to bear for their students and each other. No longer are teachers alone in classrooms that grow increasingly larger with budget cuts. Instead, teachers are working together, providing all students with carefully monitored learning time, in groups of varying composition throughout the day depending on the project at hand.

In this system, an “evaluation” is not a score, narrowly defined, based on one or two high stakes tests and a handful of observations, but a detailed picture of each teacher’s contributions. This expanded evaluation would include not only the performance of their students, but also teachers’ contributions working with colleagues and students, and an understanding of their strengths and areas for further development. Evaluators would take into account key factors that support or impede teachers’ effectiveness including their course load, the stability of their student population, the mix of teachers’ skills on their team, and the type of professional development they have received. And while this evaluation would influence each teacher’s compensation and tenure, its more immediate purpose would be to provide critical data to the teacher, his or her principal and the district. This data would inform a wide range of other human capital decisions, including job and team assignment, support, professional development, coaching needs, and opportunities for additional responsibilities and promotion. (See The Teaching Job: Restructuring for Effectiveness for details and guidance.)

Such a system would provide the foundation for a teaching profession that attracts and retains the best candidates through emphasizing team collaboration, rewarding contribution within and beyond the classroom, and establishing interventions and consequences for poor performance. Equally important, such a system would give district and school leaders the flexibility and information they need to deploy and manage effective teaching teams.

This post was adapted from our commentary in Ed Week, January 18, 2011.