Download the PDF File Top of Page

Appendix A A Balanced School Accountability Model: An Alternative to High-Stakes Testing

• Premises
• A Balanced Model
  • Student learning
  • Opportunity to learn
  • Responsiveness to students, parents, and community
  • Organizational capacity
• The New Role of the State

Background

Table of Contents

Israel's paramount national resource is her people and their intellectual capacity. Israel's scientists are at the forefront in many fields and are the heart of her contributions to the world and her economic system. Israel's military learns and extends its capability based on continuous review and assessment of its activities. The quality of her thinkers, philosophers, scientists and researchers is continuously recognized in the disproportionate number of international awards garnered by Israelis and Jews.

Current educational practices are not designed to build on this capacity, and are failing Israel's students. Nearly 350,000 Israeli students are considered to be “at risk” as a result of abuse, neglect and poverty. As many as ten percent of students in the Jewish sector and thirty percent in the Arab sector will not complete high school. Israel's schools are not taking care of each and every student as a whole human being. Efforts at reform typically look to how to coerce students into attending and then how to assure that they acquire a certain set of facts before they leave.

Even when students show up, too often they are not inspired to think. They may be able to reduce 3/6 to ½, but fail to appreciate that these represent the same amount. They may know how to decode a word, but be unable to form a sentence or reduce an extended thought to writing.

The focus of this system must extend beyond imparting “traditional skills” to training and developing the ability to draw the connections that make these skills usable in student's lives.

Mission

Table of Contents

In order to accomplish a goal – educating Israel's children, for instance – it must be defined. Only then can a set of measures be established to determine if the system is accomplishing what it set out to. This Paper proposes the following:

Educational Mission Statement

In order to secure the future growth and prosperity of the people of Israel, the Ministry of Education guarantees a safe and healthy environment in which every child can realize his fullest potential as a student and as a human being. The Ministry's mission is to foster excellence and apply the best known educational practices to make of Israel's students:

1. Moral and Ethical Human Beings, people who love life and who care for their fellow human beings and the society to which they belong;
2. Active and Committed Citizens , people who understand Israel's history and are willing to take a stand for what they believe is right, for themselves, for their families and communities, and for society as a whole;
3. Risk Takers, people who continuously challenge themselves, who set for themselves the highest of standards and strive to realize them;
4. Scientists and Philosophers, people who are intellectually curious, who delve deeply into matters, who constantly reflect, evaluate, and incorporate new information in their life-long quest to understand their world and how it works; and
5. Individuals, people who are independent, proud, confident, and know what they stand for, while also accepting, respecting and relating with those who don't share their ideas and viewpoints.

For the purposes of this paper, it will be assumed that the mission set forth by the Ministry will accord in general principles with that proposed above.

The Best Practices Initiative

Table of Contents

Just as one would not call a plumber to fix a broken electrical panel, a desk sergeant for forensic analysis, or a dentist for neurological research, one should not trust other than recognized educational experts and researchers when it comes to the reform of education. Unfortunately, the conversation about reform of education has long been controlled by educational bureaucrats, politicians, and even well-meaning but woefully misinformed parents. This country is failing its children, and risking its economic well-being and political future as it continues to ignore what is known about education. This paper proposes the establishment of the following structure to ensure that the best practices are what drives Israel's educational system.

1. Research Review Committee

   Table of Contents

   This select committee of educational experts will consist of professional researchers in education and educational related fields. It will be their job to review all published educational research, analyze it critically and provide working summaries to the Coordinating Committee of the Best Practices Initiative (BPI), to the Best Practices Committee and, through the Public Information Office, to the press, educational policy makers, students and their parents. This committee will also comment on what areas need to be further researched and encourage the educational research community to look into these issues.

2. Best Practices Committee

   Table of Contents

   This committee will overlap significantly with the Research Review Committee. It's job will be to take the research and summaries provided by the Research Review Committee and establish a set of best practices. It will coordinate with the Feasibility and Budget Committee to set priorities as to implementation. In its ultimate form, it will serve as a training vehicle for teachers in the field. It will do this two ways:

   1. It will be publish The Professional Educator, a quarterly newsletter to provide teachers with access to best practices and the data and arguments to support them.
   2. It will run a teacher training unit. Under the committee's auspices will be a team of teacher trainers who will work with teachers and administrators in the field on an ongoing basis to bring best practices to the classroom. This will operate as an in house consulting organization.

   This committee will also have a special functionary, who is also part of the Public Information Office, to collect information on educational practices worldwide with a view to making available an executive summary of what works and what does not so that the Unions, Government and the Press can get their information straight.

3. Feasibility and Budget Committee

   Table of Contents

   This committee will work with the Research Review and Best Practices Committee to determine the feasibility of proposed changes and help to develop a set of standards for the determination of what programs will be implemented.

4. Best Practices Schools

   Table of Contents

   The Ministry will adopt one each of a failing primary, middle and secondary school. It will have a special team that is responsible for applying the best practices to these schools. These schools will also be training grounds for new teachers. In conjunction with the various teaching colleges, aspiring teachers will work in these schools and learn how to apply what works. These schools will also provide the space for working teachers and administrators to come and experience best practices that they can then take back to their own classrooms and schools. In the event that permanent model schools are established, there will also be on-campus facilities for veteran teachers to spend a part of their sabbatical year taking education courses and experiencing these schools.

5. Public Information Office

   Table of Contents

   This office's primary mission will be to recognize that teachers are professionals and that they know best what education is and should be. This office will establish for the public the value of the professional educator. Just as a person would not trust a reporter or editorial board to diagnose his medical condition, it should not look to the popular press to determine what is wrong with Israel's schools. It should look to the professionals, and when it does, the first place it should look is the PIO.

   This office will disseminate research summaries and information on best practices and highlight the Ministry's accomplishments. It will have information packets for the media which will provide the media ready access to research based knowledge about education and to the researchers, perhaps through designated “experts” of the Research Review Committee.

   It will hold regular conferences for the press and education policy makers.

6. Educational Resources Unit

   Table of Contents

   This office will:

   1. Maintain a website with public access materials suitable for teaching.
      1. It will work with the nations newspaper publishers to put archival materials on line so that students have access to primary sources regarding the history and events they are studying.
      2. It will provide links and access to math and science that are accessible to students at all levels and that makes these subjects come alive to them.
      3. It will provide access to diverse sources on the issues of the day so that students can learn to question their beliefs, incorporate new information, theorize about their world and test those theories.
   2. Provide educational materials in print form in support of the best practices established above.
   3. Provide access to supplementary materials of interest to students, again to help education come alive for all those concerned.
   4. Coordinate with bodies and companies outside of the ministry that are willing to make resources available or coordinate activities for educational purposes. This may include companies and businesses that are willing to provide internships, access to facilities, tours or special projects, or access to current and retired experts.

Accountability and Assessment

Table of Contents

Questions of accountability necessarily raise the issue of what is being accounted for. To establish a workable system of education, the Ministry must define what it is accountable for, to whom it will be held accountable, and how to systematically assure that it is producing results.

Accountability for What

Table of Contents

The focus should be on excellence in the process of learning rather than on the attainment of a set curriculum. This is not to suggest that curriculum is not important and should not be set or measured, but that the near exclusive reliance on such measures orients teaching in such a manner that it interferes with the Ministry's primary goals and paradoxically the attainment of the very curriculum sought to be taught.

This also extends well beyond the classroom and must include the provision of a safe, healthy and inspiring environment for the attainment of our objectives.

Who is Accountable and to Whom

Table of Contents

There are a number of stakeholders in all educational systems. Ultimately, each of these stakeholders is accountable to Israel's students to deliver the best education that the budget will allow. As resources are finite, it is also incumbent upon the Ministry to balance its priorities and tailor its initiatives for maximum impact and efficiency.

The Minister of Education

Table of Contents

The Minister of Education is accountable for providing the context in which students can learn. To the students, he is responsible for a safe and healthy learning environment. To the teachers, he is responsible for access to the support and resources necessary to conduct their classes. This support must include access to master educators and programming for professional development, in addition to mechanisms for dealing with student, parent and personal issues. These resources include educational research as well as all the materials necessary for the running of a classroom.

To the public, the Minister is accountable for efficient use and equitable allocation of resources, advocating for appropriate allocations from the Knesset, and ensuring that students' educational interests always trump political considerations.

Ultimately, the Minister is responsible for ensuring that what goes on in the classroom is in accord with best known practices. As a political appointee, it is understood that he may not have expertise in every matter over which he is accountable. It is therefore his duty to select respected professional educators as his functionaries and defer to their expertise in determining educational policy.

The Administrator

Table of Contents

The administrator is first and foremost responsible for a healthy, safe, and inspiring learning and teaching environment. It is his job to stand in support of the teacher and to help educate parents as to how the educational methods in use best serve his wards. As such, he must be a partner in the work the Best Practices Committee is bringing to his school and understand the research that underlies the Committee's work.

It is also his job to work with his teachers in the development of educational goals, methods and measurements on a class by class basis, provide the support necessary to achieve those goals and review progress periodically to ensure that it is being made.

The Teacher

Table of Contents

The teacher's job is to know her students, understand their interests and abilities, and to build on those to develop the subject matter that is her responsibility. Her job is to cause her students to think deeply about the subject at hand, to question it and theorize about it, and to put those theories to the test in order to gain the completest possible perspective and understanding of a matter. Her job is to question and challenge and respect all viewpoints.

Within this, she designs the program she wishes to accomplish and establish a mechanism to assess progress. She must be able to justify the program she is using to her students and their parents, and must inspire her students to take ownership over their own advancement and development.

She must also be an advocate for her students to all other stake-holders. She explains to parents what she is doing, what she to accomplish, and what is expected of them.

The Student

Table of Contents

The primary job of the students is to show up and be partners in their own advancement. They will be asked to set their own goals and determine how they will know when they have met them.

They will also be encouraged to challenge and inquire, and trained to take responsibility. Opportunity will be given for students to sit on committees and participate in decision making at all levels of their school.

As students advance, they will be asked to consider their own future goals and will be counseled on what is necessary for their achievement and how their studies must accordingly be organized.

The Parent

Table of Contents

Parents are responsible for supporting both student and teacher. They are expected to teach their children manners and respect. They are also expected to take an interest in their children's education and to provide input into the schools direction and priorities via appropriate structures.

Measurement

Table of Contents

Measurement of a thing often interferes with the thing measured. Think about boiling a pot of water. Every time we take off the lid to check if it is boiling, heat is let out and the process slows. If we keep taking the pot off the flame to stick a thermometer in, it might never come up to temperature. So we learn to be patient and get a whistling tea pot or a glass lid. We also remember that we don't need a thermometer to tell us if water is boiling. We can hear it. We can also touch the pot to see if it is getting hot or look under it to check for a flame.

As educational policymakers, we must always be wary of the risks of focusing too much on certain measures. We must recognize that learning is a process that must be given time. We must trust that as heat is applied, the water will eventually come to a boil. As administrators and bureaucrats, we must learn how to check the educational flame without lifting the lid.

The Cost of Israel's Current System

Table of Contents

Most of Israel's students have been indoctrinated to believe that their role is to come home with a grade and pass examinations. Those who feel they can not often believe they have failed as students, and perhaps even as human beings. With this limited viewpoint, Israel's students do what is logical to maximize their “success.” They “learn” the materials they are supposed to in order to regurgitate what they have been asked to by their teachers, and they “learn” what is going to be on the examinations. Unfortunately, this type of learning often fails to lead to deep understanding of a subject and much of the material learned is forgotten within a short time after examinations have completed. Given the sweeping nature of the many examinations students have to take, test preparation devolves into an exercise in memorization, with the consequence being that the knowledge gained is often extremely superficial and the ability to apply it in an authentic context is limited. In short, setting an external standard often reduces the standard to the minimum necessary, instead of the maximum of which a student is capable.

The Ministry of Education must be responsible for the fact that different manners of measurement are appropriate to different purposes. The Mission set out above is not one suited to evaluation exclusively by way of summative assessments. But progress must be made, and there must be a means of measuring it.

Manner of Assessment

Table of Contents

Accountability for this paper's purposes will deal with how schools, teachers, parents and students can be held answerable for meeting the objectives of Israel's educational system. Assessment comprises the means by which we measure the educational progress of Israel's students and the effectiveness of Israel's teachers, administrators and schools at delivering education.

The point of departure should be Israel's objectives for her students. While there is certain basic knowledge that all students should attain, the context in which they meet these objectives is decisive to the people they will become. This paper assumes as Israel's goals the molding of Moral and Ethical Human Beings, Scientists and Philosophers, Life-Long Learners, Risk Takers, Active and Committed Citizens, and Individuals.

When a system enrolls its students in these ideals and work out of this context, students innate drive for their own advancement will take over. Only when a student takes ownership over his own learning – instead of feeling that it is something he is compelled to do – will true learning be available to him.

The primary means of determining whether a student is learning will be to look at his works. He will be expected to keep a log or portfolio in his subjects, encouraged to maintain a journal, and will be asked to explain what he sought out to accomplish, and analyze how well he met his own goals.

How to Replace the Current System with One That Will Meets These Goals

Table of Contents

There are numerous ways assessment can be characterized. One of these is as summative verses formative assessments. Summative assessment is assessment of status, what has been learned or where a particular matter stands. The Bagrut is an example of this. After a course of study, a test is given to determine if a student has learned the material that he should know for the test. The Meitzav is also summative, being used, to determine where students, and also schools, stand in relation to a desired outcome.

Its focus is on whether a student has learned what he was supposed to. While there is no question that a certain fount of knowledge is desirable, the course of study followed to meet the demands of an exam often simply stresses what will be on the exam and fails to put the material gained in a context that a student will be able to draw on it meaningfully to further his own education. Unfortunately, even when students do learn what they need for a test, they often forget much of what they have “learned” in short order afterwards.

In contrast to this stands formative assessment. Formative assessment is an ongoing process using both formal and informal methods that allow “teachers and students to recognize and respond to student learning in order to enhance that learning, during the learning.” Students are trained to self-assess and reflect on their learning and its purposes in order to “grasp what they need to do to achieve."

The teacher's role in this context is to work with students to create assessments and review processes to determine how best to proceed with a student. Evaluation thus takes on a more holistic approach, one which has as its focus the development of the student instead of the achievement of a grade.

To the extent summaries are necessary, these can be in narrative form with exemplars attached and specifically addressing the competencies expected of a child at his particular grade level.

Similar assessment models will be established for teachers and administrators.

Shifting the Burden

Table of Contents

In order to have a meaningful assessment, we should consider why the assessment is being made. In most cases, assessments made under the auspices of the Ministry of Education will be used for military and/or university placement. Considering the many jobs and fields of study in which the current evaluations are used for placement, they can not adequately evaluate the skill sets necessary for any particular one. While they may lead to some reproducible ranking of students, they can not tell us who can formulate a scientific question well and design the experiment to test it, who will likely relate well to patients, who will effectively represent someone in court, or whose artistic expression merits the study of fine arts.

The burden must therefore be shifted to those who are selecting to determine appropriate criteria. These criteria will be communicated to students and they can then plan their studies accordingly.

A Solution

Table of Contents

Though the foundations of this program are a Best Practices Initiative and a revised structure for accountability and assessment, these will only be effective in the right context. Icing on a mud cake still leaves a mud cake. To create something palatable, the ministry must clear the mud before it can produce results. To do this, the Ministry should consider taking the following steps immediately:

1. In partnership with all of the stakeholders, establish a mission statement including our goals and a set of core competencies that our students should achieve.
2. Take a critical look at current practices, and eliminate those that don't work.
3. Teach students to regulate their own learning and use self-assessment strategies to achieve at their highest potential.
4. Give students choice in the matter of their own education. Give them a voice in how the school is run and what they will learn. Encourage them to set their own priorities in line with their future educational and career goals. Train staff, teachers and social workers to help students do this in a meaningful manner.
5. Give parents a voice in their children's education at the level of the management of the school and the setting of its priorities.
6. Give schools and teachers the opportunity to set out their own specific goals, and provide a structure for review of methods and progress at various times during the year.
7. Train teachers and administrators to most effectively assess and use assessments to guide their teaching and achieve educational goals.
8. Train teachers to analyze their own efforts, to measure progress toward their goals, and to adjust their teaching appropriately. At the same time, give them the freedom to experiment and determine what is best in their own classrooms. Training should be intensive, on-site and ongoing.
9. Provide our administrators the training to be able to measure defined goals and assist teachers in achieving them.
10. Ensure that all stakeholders have access to valid current educational research so that they can place their schools on a firm foundation.
11. Provide for an extensive development and review structure within the schools. This should include time for observations of other classes and debriefings to analyze what works and institutionalize it.
12. Create a consultancy and evaluation structure, preferably drawn from the best professors at the various teachers colleges in ongoing training and restructuring efforts at schools, and make them available over extended periods of time.
13. Use summative assessments sparingly. To the extent status reports are necessary, randomly select a statistically significant sampling of students and administer tests to them. Do not require students to do comprehensive exams, but break the tests into parts that will be spread over a larger sampling of students. Do not divulge individual results to students, but instead use the results for formative purposes only.
14. Where particular bodies of knowledge are required (i.e. army placements and university program admissions), put the burden on those making the selections to provide sufficiently targeted evaluations to meet their needs. The military can rely on armed services vocational aptitude batteries and additional tests for special services. Universities can use portfolio reviews, personal interviews, statements of interest and proposals for research – in addition to or in place of targeted tests – to provide a more complete and accurate assessment of a student's potential than a Bagrut certificate provides.
15. Establish a Best Practices Initiative to provide teachers with the research and tools to be the most effective teachers they can be. Continue to train and work with teachers, both in and out of their classrooms, in order to bring these best practices to fruition.

Conclusion

Table of Contents

We are about creating Inspired and Engaged Human Benings. Our commitment in the Israeli system must therefore be to the excellence, happiness and success of all of our children. To assure that we achieve this, we must specifically define our mission and create a structure in service of that mission. Every initiative and allocation of resources should be viewed in the light of that mission. The lens which must be applied is that of the scientific process. Our thinking and our planning should proceed from that which we know, not from the biases and beliefs we have developed based on our specific experiences. When we apply these best practices, we will achieve our mission.

Appendix A

The following, from Ken Jones' article A Balanced School Accountability Model: An Alternative to High-Stakes Testing in Phi Beta Kappan, http://www.pdkintl.org/kappan/k0404jon.htm, should form the foundation for a discussion on how to reorganize the Israeli system to truly serve Israel's students. Ken Jones is the director of teacher education, University of Southern Maine, Gotham. I have modified the following only to the extent necessary to reference Israeli institutions instead of American ones and to remove comments specific only to the US system.

A Balanced School Accountability Model: An Alternative to High-Stakes Testing

FOR SOME time now, it has been apparent to many in the education community that . . . policies intended to develop greater school accountability for the learning of all students have been terribly counterproductive. The use of high-stakes testing of students has been fraught with flawed assumptions, oversimplified understandings of school realities, undemocratic concentration of power, undermining of the teaching profession, and predictably disastrous consequences for our most vulnerable students. Far from the noble ideal of leaving no child behind, current policies, if continued, are bound to increase existing inequities, trivialize schooling, and mislead the public about the quality and promise of public education.

What is needed is a better means for evaluating schools, an alternative to the present system of using high-stakes testing for school accountability. A new model, based on a different set of assumptions and understandings about school realities and approaches to power, is required. It must be focused on the needs of learners and on the goals of having high expectations for all rather than on the prerequisites of a bureaucratic measurement system.

Premises

Table of Contents

In the realm of student learning, the question of outcomes has often been considered primary: what do we want students to know and be able to do as a result of schooling? Once the desired outcomes have been specified, school reform efforts have proceeded to address the thorny questions of how to attain them. Starting from desired outcomes is an important shift in how to think about what does or does not make sense in classroom instruction.

In the realm of school accountability, however, little attention has been paid to corresponding outcome-related questions. It has simply been assumed that schools should be accountable for improved student learning, as measured by external test scores. It has been largely assumed by policy makers that external tests do, in fact, adequately measure student learning. These and other assumptions about school accountability must be questioned if we are to develop a more successful accountability model. It would be well to start from basic questions about the purposes and audiences of schools. For what, to whom, and by what means should schools be held accountable? The following answers to these questions provide a set of premises on which a new school accountability system can be based.

For what should schools be accountable? Schools should be held accountable for at least the following:

• The physical and emotional well-being of students. The caring aspect of school is essential to high-quality education. Parents expect that their children will be safe in schools and that adults in schools will tend to their affective as well as cognitive needs. In addition, we know that learning depends on a caring school climate that nurtures positive relationships.
• Student learning. Student learning is complex and multifaceted. It includes acquiring not only knowledge of disciplinary subject matter but also the thinking skills and dispositions needed in a modern democratic society.
• Teacher learning. Having a knowledgeable and skilled teacher is the most significant factor in student learning and should be fostered in multiple ways, compatible with the principles of adult learning. Schools must have sufficient time and funding to enable teachers to improve their own performance, according to professional teaching standards.
• Equity and access. Given the history of inequity with respect to minority and underserved student populations, schools must be accountable for placing a special emphasis on improving equity and access, providing fair opportunities for all to learn to high standards. Our press for excellence must include a press for fairness.
• Improvement. Schools should be expected to function as learning organizations, continuously engaged in self-assessment and adjustment in an effort to meet the needs of their students. The capacity to do so must be ensured and nurtured.

To whom should schools be accountable? Schools should be held accountable to their primary clients: students, parents, and the local community. Current accountability systems make the state and federal governments the locus of power and decision making. But the primary clients of schools should be empowered to make decisions about the ends of education, not just the means, provided there are checks to ensure equity and access and adherence to professional standards for teaching.

By what means should schools be held accountable? To determine how well schools are fulfilling their responsibilities, multiple measures should be used. Measures of school accountability should include both qualitative and quantitative approaches, taking into account local contexts, responsiveness to student and community needs, and professional practices and standards. Because schools are complex and unique institutions that address multiple societal needs, there should also be allowances for local measures, customized to meet local needs and concerns. A standardized approach toward school accountability cannot work in a nation as diverse as [Israel]

Given these premises, what are the proper roles of a government-developed and publicly funded school accountability system?

• It should serve to improve student learning and school practices and to ensure equity and access, not to reward or punish schools.
• It should provide guidance and information for local decision making, not classify schools as successes or failures.
• It should reflect a democratic approach, including a balance of responsibility and power among different levels of government.

A Balanced Model

Table of Contents

An accountability framework called the “balanced scorecard” is currently employed in the business world and provides a useful perspective for schools.1 This framework consists of four areas that must be evaluated to give a comprehensive view of the health of an organization. The premise is that both outcomes and operations must be measured if the feedback system is to be used to improve the organization, not just monitor it. In the business context, the four components of the framework are: 1) financial, 2) internal business, 3) customer, and 4) innovation and learning.

Applying this four-part approach to education, we can use the following aspects of school performance as the components of a balanced school accountability model: 1) student learning; 2) opportunity to learn; 3) responsiveness to students, parents, and community; and 4) organizational capacity for improvement. Each of these aspects must be attended to and fostered by an evaluation system that has a sufficiently high resolution to take into account the full complexity and scope of modern-day schools.

1. Student learning.

Principles of high-quality assessment have been well articulated by various organizations and should be followed.2 What is needed is a system that

• is primarily intended to improve student learning;
• aligns with local curricula;
• emphasizes applied learning and thinking skills, not just declarative knowledge and basic skills;
• embodies the principle of multiple measures, including a variety of formats such as writing, open-response questions, and performance-based tasks; and
• is accessible to students with diverse learning styles, intelligence profiles, exceptionalities, and cultural backgrounds.

Currently, there is a mismatch between what cognitive science and brain research have shown about human learning and how schools and educational bureaucracies continue to measure learning.3 We now know that human intellectual abilities are malleable and that people learn through a social and cultural process of constructing knowledge and understandings in given contexts. And yet we continue to conduct schooling and assessment guided by the outdated beliefs that intelligence is fixed, that knowledge exists apart from culture and context, and that learning is best induced through the behaviorist model of stimulus/response.

Scientific measurement cannot truly “objectify” learning and rate it hierarchically. Accurate decisions about the quality and depth of an individual's learning must be based on human judgment. While test scores and other assessment data are useful and necessary sources of information, a fair assessment of a person's learning can be made only by other people, preferably by those who know the person best in his or her own context. A reasonable process for determining the measure of student learning could involve local panels of teachers, parents, and community members, who review data about student performance and make decisions about promotion, placement, graduation, and so on.

What is missing in most current accountability systems is not just a human adjudication system, but also a local assessment component that addresses local curricula, contexts, and cultures. A large-scale external test is not sufficient to determine a student's achievement. District, school, and classroom assessments must also be developed as part of a comprehensive means of collecting data on student learning. The states of Maine and Nebraska are currently developing just such systems.4

Most important, locally developed assessments depend on the knowledge and “assessment literacy” of teachers.5 Most teachers have not been adequately trained in assessment and need substantial and ongoing professional development to create valid and reliable tasks and build effective classroom assessment repertoires. This means that an investment must be made in teacher learning about assessment. The value of such an investment is not only in the promise of improved classroom instruction and measurement. Research also shows that improved classroom assessment results in improved student achievement on external tests.6

Last, the need to determine the effectiveness of the larger state school system can either support or undermine such local efforts. If state or federal agencies require data to be aggregated from local to state levels, local decision making is necessarily weakened, and an undue emphasis is placed on standardized methods. If, however, the state and federal agencies do not rely on local assessment systems to gauge the health of the larger system, much may be gained. In New Zealand, for example, a system of educational monitoring is in place that uses matrix sampling on tasks that include one-to-one videotaped interviews, team tasks, and independent tasks.7 No stakes are entailed for schools or students. The data are profiled and shared with schools for the purpose of teacher professional development and as a means of developing model tasks for local assessments. Such a system supports rather than undermines local assessment efforts.

2. Opportunity to Learn.

How can students be expected to meet high standards if they are not given a fair opportunity to learn? This question has yet to be answered with respect to school accountability. Schools should be accountable for providing equitable opportunities for all students to learn, and we must develop ways to determine how well they do so.

. . .

How should we define and put into practice our understanding of opportunity to learn? How will we measure it? How can an accountability system foster it?

At a minimum, one might expect that schools and school systems will provide qualified teachers, adequate instructional materials, and sound facilities. . . . [T]hat is, while schools are held accountable for performance, the state is held accountable for ensuring adequate resources.8

But there is more to this issue than just funding. Jeannie Oakes describes a framework that includes opportunity-to-learn indicators for access to knowledge, professional teaching conditions, and “press for achievement."9 Linda Darling-Hammond stresses the “fair and humane treatment” of students in a set of standards for professional practice.10

As such standards for opportunity to learn are articulated, the question arises as to how to monitor and report on them. Clearly, the degree of adherence to these standards cannot be determined through the proxy of testing. It is necessary to conduct observations in schools and classrooms and to evaluate the quality both of individual teachers and of the school as a whole.

Teacher evaluation has received a great deal of criticism for being ineffective. The hit-and-run observations that principals typically conduct do little to determine whether teachers are meeting established professional teaching standards. Unions have been described as more interested in protecting their membership than in ensuring high-quality teaching. A promising development that has potential for breaking through this impasse is the recent initiation of peer-review processes by a number of teacher unions. Adam Urbanski, president of the Rochester Teachers Association and director of the Teacher Union Reform Network (TURN), has been a leader in advocating for and implementing such teacher evaluation processes. In a recent unpublished manuscript, he describes how the process should work:

• Some classroom observation by peers and supervisors, structured by a narrative instrument (not a checklist) based on professional standards such as those of the National Board for Professional Teaching Standards (NBPTS) and framed by the teacher's goals for the lesson/unit;
• Information from previous evaluations and feedback, such as structured references from colleagues and other supervisors;
• Portfolios that might include examples of teaching syllabi, assignments made, feedback given to students and samples of student work, feedback received from parents and students as well as colleagues, data on student progress, teaching exhibitions such as videotaped teaching samples, professional development initiatives taken, and structured self-evaluation. All summative evaluation decisions about promotions or continued employment should be made by a specially established committee of teachers and administrators.

Urbanski goes on to describe safeguards for due process and for preventing malpractice. He also describes how such a process could be used in conjunction with professional development for improving teaching and school practice.11

In order to evaluate the performance of a school as a whole, a school review process will be necessary. Variations of inspectorates and school-quality reviews have been developed in New York, Rhode Island, Maine, and other states, as well as in Britain, New Zealand, Australia, and other countries.12 In order for such reviews to serve the purpose of school improvement, the data should be collected in a “critical friend” manner, through a combination of school self-assessment and collegial visitations. Findings from such a process should not be employed in a bureaucratic and judgmental way but rather should be given as descriptions to local councils charged with evaluating school accountability. As with all aspects of any school renewal initiative, the quality and effectiveness of a review system will depend on the time, resources, and institutional support given to it.

Who will ensure that adequate opportunities to learn are present in schools? As described below, a system of reciprocal accountability must be set up so that both local accountability councils and the state itself serve to “mind the store” for all students. The issue of equitable funding will undoubtedly be resolved through the courts.

3. Responsiveness to students, parents, and community.

Table of Contents

Current accountability systems move power and decision making away from the primary clients of the education system and more and more toward state and federal agencies. As high-stakes testing dictates the curriculum, less and less choice is available for students. Parent or community concerns about what is happening in the classroom and to the students have become less important to schools than meeting state mandates.

As the primary stakeholders in the schools, parents and communities must be made part of the effort to hold schools accountable. There are many examples of local community organizations, especially in urban areas, that have taken on the task of insisting that schools are responsive to the needs of children.13

To demonstrate responsiveness to students, parents, and the community, schools must go beyond sponsoring parent/teacher organizations or encouraging parent involvement as a means to gain support for existing school practices. They must also do more than gather survey information about stakeholders' satisfaction. True accountability to the primary clients for schools entails shifting power relationships.

Local school-based councils must be created that have real power to effect school change. These councils would review accountability information from state and local assessments as well as from school-quality review processes and make recommendations to school boards about school policies and priorities. They would hold school boards accountable for the development and implementation of school improvement plans. Phillip Schlechty discusses how such councils might work:

Community leaders who are concerned about the futures of their communities and their schools should join together to create a nonprofit corporation intended to support efforts of school leaders to focus on the future and to ensure that lasting values as well as immediate interests are included in the education decision-making process. It would also be the function of this group to establish a small subgroup of the community's most trusted leaders who would annually evaluate the performance of the school board as stewards of the common good and would make these evaluations known to the community. . . .
In a sense, the relationship between the school district and the monitoring function of the new corporation should be something akin to the relationship between the quality assurance division of a corporation and the operating units in the corporation. . . .

When the data indicate that goals are not being met, the president of the corporation, working with the superintendent and the board of education, would seek to discover why this was the case, and would seek as well to create new approaches that might enhance the prospect of achieving the stated goals and the intended ends. It is not intended that the new corporation simply identify problems and weaknesses, it is intended that the leaders of this organization also participate in the creation of solutions and participate in creating support for solutions once they have been identified or created.14

Communities must determine how to sustain such councils and ensure that they do not pursue narrow agendas. The composition of councils in urban settings will probably be different from those in rural or suburban settings. Standards and acceptable variations for councils will be important topics for public discussion.

4. Organizational capacity.

Table of Contents

If schools are going to be held accountable to high levels of performance, the question arises: Do schools have the internal capacity to rise to those levels? To what degree are the resources of schools “organized into a collective enterprise, with shared commitment and collaboration among staff to achieve a clear purpose for student learning"?15

The issue of meaningful and ongoing teacher professional development is especially pertinent to whether or not schools are capable of enabling all students to meet higher standards of performance. A great deal of research has shed light on what kind of professional development is most effective in promoting school improvement.16

Schools must also attend to the issue of teacher empowerment. Teachers are increasingly controlled and disempowered in various ways. This leads to a declining sense of efficacy and professionalism and a heightened sense of job dissatisfaction and has become a factor in the attrition that is contributing to the growing teaching shortage.17 Principals must share leadership with teachers and others as a means of sustaining capacity.

To be an effective collective enterprise, a school must develop an internal accountability system. That is, it must take responsibility for developing goals and priorities based on the ongoing collection and analysis of data, it must monitor its performance, and it must report its findings and actions to its public. Many schools have not moved past the stage of accepting individual teacher responsibility rather than collective responsibility as the norm.18 States and districts must cooperate with schools to nurture and insist upon the development of such collective internal norms.

The New Role of the State

For a balanced model of school accountability to succeed, there must be a system in which [the Ministry of Education is] jointly responsible with schools and communities for student learning. Reciprocal accountability is needed: one level of the system is responsible to the others, and all are responsible to the public.

The role of [the Ministry] with respect to school accountability is much in need of redefinition. . . . [Its role] should be to establish standards for local accountability systems, to provide resources and guidance, and to set in place processes for quality review of such systems. Certainly there should be no high-stakes testing . . . , no mandatory curricula, and no manipulation through funding. Where there are clear cases of faulty local accountability systems - those lacking any of the four elements discussed above (appropriate assessment systems; adequate opportunities to learn; responsiveness to students, parents, and community; or organizational capacity) - supportive efforts from the state and federal levels should be undertaken.

Are there any circumstances in which a state should intervene forcibly in a school or district? If an accountability system is to work toward school improvement for all schools, does that system not need such “teeth"? This question must be addressed in a way that acknowledges the multi-level nature of this school accountability model. One might envision at least three cases in which the state would take on a more assertive role: 1) to investigate claims or appeals from students, parents, or the community that the local accountability system is not meeting the standards set for such systems; 2) to require local schools and districts to respond to findings in the data that show significant student learning deficiencies, inequity in the opportunities to learn for all students, or lack of responsiveness to students, parents, or communities; and 3) to provide additional resources and guidance to improve the organizational capacity of the local school or district. Is it conceivable that a state might take over a local school or district in this model? Yes, but only after the most comprehensive evaluation of the local accountability system has shown that there is no alternative - and then only on a temporary basis.

It is of great importance to the health of our public schools that we begin as soon as possible to define a new model for school accountability, one that is balanced and comprehensive. Schools can and should be held accountable to their primary clients for much more than test scores, in a way that supports improvement rather than punishes deficiencies. The current model of using high-stakes testing is a recipe for public school failure, putting our democratic nation at risk.

Notes to Ken Jones' Article

1Robert S. Kaplan and David P. Norton, “The Balanced Scorecard - Measures That Drive Performance,” Harvard Business Review, January/February 1992, pp. 71-79.

2National Forum on Assessment, Principles and Indicators for Student Assessment Systems (Boston: FairTest, 1993), available at www.fairtest.org/k-12.htm.ard Business Review, January/February 1992, pp. 71-79.

3Lorrie A. Shepard, “The Role of Assessment in a Learning Culture,” Educational Researcher, October 2000, pp. 4-14.ard Business Review, January/February 1992, pp. 71-79.

4Debra Smith and Lynne Miller, Comprehensive Local Assessment Systems (CLASs) Primer: A Guide to Assessment System Design and Use (Gorham: Southern Maine Partnership, University of Southern Maine, 2003), available at www.usm.maine.edu/smp/tools/primer.htm; and “Nebraska School-Based, Teacher-Led Assessment Reporting System (STARS),” www.nde.state.ne.us/stars/index.html.ard Business Review, January/February 1992, pp. 71-79.

5Richard J. Stiggins, Student-Centered Classroom Assessment (Columbus, Ohio: Merrill, 1997).ard Business Review, January/February 1992, pp. 71-79.

6Paul Black and Dylan Wiliam, “Inside the Black Box: Raising Standards Through Classroom Assessment,” Phi Delta Kappan, October 1998, pp. 139-48; and Paul Black et al., Working Inside the Black Box: Assessment for Learning in the Classroom (London, U.K.: Department of Educational and Professional Studies, King's College, 2002).

7Terry Crooks, “Design and Implementation of a National Assessment Programme: New Zealand's National Education Monitoring Project (NEMP),” paper presented at the annual meeting of the Canadian Society for the Study of Education, Toronto, May 2002.

8Jeannie Oakes, “Education Inadequacy, Inequality, and Failed State Policy: A Synthesis of Expert Reports Prepared for Williams v. State of California,” 2003, available at www.decentschools.org/experts.php.

9Jeannie Oakes, “What Educational Indicators? The Case for Assessing the School Context,” Educational Evaluation and Policy Analysis, Summer 1989, pp. 181-99.

10Linda Darling-Hammond, Standards of Practice for Learning Centered Schools (New York: National Center for Restructuring Education, Schools, and Teaching, Teachers College, 1992).

11Adam Urbanski, “Teacher Professionalism and Teacher Accountability: Toward a More Genuine Teaching Profession,” unpublished manuscript, 1998.

12Jacqueline Ancess, Outside/Inside, Inside/Outside: Developing and Implementing the School Quality Review (New York: National Center for Restructuring Education, Schools, and Teaching, Teachers College, 1996); New Zealand Education Review Office, Frameworks for Reviews in Schools, available at www.ero.govt.nz/EdRevInfo/Schedrevs/SchoolFramework.htm; Debra R. Smith and David J. Ruff, “Building a Culture of Inquiry: The School Quality Review Initiative,” in David Allen, ed., Assessing Student Learning: From Grading to Understanding (New York: Teachers College Press,1998), pp. 164-82.

13Kavitha Mediratte, Norm Fruchter, and Anne C. Lewis, Organizing for School Reform: How Communities Are Finding Their Voice and Reclaiming Their Public Schools (New York: Institute for Education and Social Policy, Steinhardt School of Education, New York University, October 2002).

14Phillip Schlechty, Systemic Change and the Revitalization of Public Education (San Francisco: Jossey-Bass, forthcoming).

15Fred M. Newmann, M. Bruce King, and Mark Rigdon, “Accountability and School Performance: Implications from Restructuring Schools,” Harvard Educational Review, Spring 1997, p. 47.

16Judith Warren Little, “Teachers' Professional Development in a Climate of Educational Reform,” Educational Evaluation and Policy Analysis, vol. 15, 1993, pp. 129-51; and Milbrey W. McLaughlin and Joan Talbert, Professional Communities and the Work of High School Teaching (Chicago: University of Chicago Press, 2001).

17Richard M. Ingersoll, Who Controls Teachers' Work? Power and Accountability in America's Schools (Cambridge, Mass.: Harvard University Press, 2003). 18Charles Abelman et al., When Accountability Knocks, Will Anyone Answer? (Philadelphia: Consortium for Policy Research in Education, University of Pennsylvania, CPRE Research Report Series RR-42, 1999).

A Balanced School Accountability Model: An Alternative to High-Stakes Testing
Last modified 25 March 2004
URL: http://www.pdkintl.org/kappan/k0404jon.htm
© 2004 Phi Delta Kappa International