Reassessing assessment in Maine

For schools and school districts the major preoccupation is money. However, the assessment of educational progress, now oppressive in federal and state legislation, is, or should be, a close second. In addition to daily classroom and annual district tests, legislated “comprehensive” assessment has triangulated on schools in ways that should have coordination but clearly do not.

For public school educators, the assessment imperatives are confusing and even nonsensical. For public school critics, downside prospects are delightfully chaotic. Three areas have serious implications for schools.

The first is an old friend, the Maine Educational Assessment given annually to students in grades 4, 8 and 11.

A model Maine initiative in 1985, the passage of the Maine Learning Results changed its purpose, content, evaluation and public reporting. Originally a test of school and subject achievement, the 1998 test carried the additional responsibility of assessing individual student achievement. The ink on the Maine Learning Results document was not even dry when testing commenced, and it is not surprising that Maine students went from outstanding state and national achievement to wholesale failure on the MEA. Testing students on untaught material has that effect. Jumbled implementation of the Maine Learning Results over the past four years has not improved the situation.

The second is the heavy-handed national assessment mandate within the No Child Left Behind Act (NCLB). Born out of the Texas educational miracle and a testing blitz based on eighth-grade standards, the federal solution to public school failure is another layer of testing in reading and mathematics in grades 3-8. States, including Maine, have scurried to turn their newly conceived state assessment initiatives into federal placation. On a more practical level for schools, the failure to meet poorly defined “annual yearly progress” standards ultimately results in their closing. While there are probably schools in this country that should close, the federal government is not the appropriate body to legislate this beheading.

Last is the Local Assessment System (LAS), a potentially valuable concept, that is required by state law, unrealistically, to be in all Maine school districts by 2004. Unfortunately, it too is poorly defined with requirements for an integrated multiple measure assessment of all content standards of the Maine Learning Results for all children. Nobody has ever seen this mythical animal, let alone knows where it might live, so it is not surprising that the state charge to build a Local Assessment System has school districts clamoring to figure out what is expected of them. Ultimately, the Local Assessment System should provide reliable evidence that students are “meeting the standard” of the Maine Learning Results at all grade levels.

To its credit, the state has attempted to create assessment tasks, but the design and content of a Local Assessment System cannot be a hodgepodge of off-the-shelf state assessment tasks. It is, after all, a local assessment, and local ingenuity will make the LAS valuable in improving student achievement. The problem is that the state is faced with local interpretation, especially in technical standards, that will make the aggregation, transfer and tracking of student achievement data nearly impossible. Most significantly, at the end of the K-12 Local Assessment pipeline is the 2007 requirement that achievement of the Maine Learning Results by each student be certified by the superintendent before a high school diploma can be conferred. Since the Maine Learning Results are not simple do-you-know standards and require higher-order thinking, assessment leading to confident certification is daunting.

The temptation will be to translate grades into performance standards, not, in my opinion, what was envisioned in the legislation, but the best answer when the architects of the “system of Maine Learning Results” are not providing practical solutions.

It should not be surprising that the convergence of these assessment mandates has resulted in local bewilderment. Recently passed state rules have forced implementation with little regard for local capacity to respond, though educators are keenly interested. In the past six months a variety of confusing and conflicting directives, born out of a state-level frenzy to accelerate development, have left schools in a state of paralysis. In addition, the specter of an unfunded Essential Programs and Services school-funding model is creating justifiable paranoia. State committees meet. Models are promised. Deadlines are set. The tide keeps rising.

Rather than a sudden bureaucratic about face in the next few months, certain immediate actions can remove the assessment confusion, and coincidentally improve student achievement. Here are some ideas.

. Address the 500-pound gorilla, NCLB assessment. Unfortunately, Sen. Susan Collins was led to write a letter to Education Secretary Rod Paige suggesting that Local Assessment System data could meet the NCLB accountability mandate. He responded with guarded encouragement, and the state took off with the idea that the LAS was the answer. For any number of reasons, the idea will not work. The Maine Department of Education should create a uniform technically defensible test for reading and mathematics in grades 3-8 using a selected response format. While aligned with the Maine Learning Results, it should avoid the obsessive tendency of some designers to test everything in sight.

The test should be predictable, straightforward, sufficiently lengthy to obtain a good profile of individual student achievement and diagnostically useful. Designers should avoid a “gotcha test” and permit students to show what they know. This does not happen when a test starts with a question on the ionization levels of certain elements. Finally, tell the teachers and students, in clear unambiguous terms, what is expected.

. Renew the instructional value of the MEA. In the three domains of assessment, the MEA serves primarily an accountability function, and observers are caring less and less about scaled scores with no meaning and discussions where “partially meets” is acceptable performance. The MEA can and should be used for much more, especially within the LAS where its technical qualities can be appreciated. Of course, schools seem to be headed toward an embarrassing inconsistency in achievement statements between an established MEA and a developing LAS.

In spite of this circumstance, the MEA offers great potential to augment the LAS and improve achievement. For example, use the first administration as a pretest similar to the first spelling test of the week. Those who meet the standard in the content areas move on.

Using the “common item” questions released from the test, instruct in weak areas in preparation for a second assessment of the released items given at the time of the next MEA administration in late fall. Obviously, teachers cannot and should not teach to a specific test item, but they can teach to the domain. For instance, in the grade 9-12 content standard Data Analysis and Statistics, the test item may be to calculate an arithmetic mean. The school district could set up various kinds of relearning experiences but with the objective of teaching central tendency which includes mean, median, mode and statistical relationships.

If results of the “second” MEA, were tied to the LAS certification for graduation, the incentive to “meet the standard” would be significant. Allowing students a second chance on a high-stakes test such as the MEA, after instructional intervention, makes good educational sense. Failing to use the MEA for specific instruction that improves the individual student achievement is a waste of time and money.

. Offer practical and technical solutions to school districts struggling to embrace the assessment of eight Content Areas and 67 Content Standards on the Maine Learning Results. Standing committees at the state level can help immeasurably by redefining the areas of assessment and demonstrating useful technical standards. The promised LAS model from the Center for Assessment would be helpful but time goes by, deadlines await, and no practical solutions are forthcoming. Educators are well aware of the Chapter 127 requirements of validity and reliability, but the simple pronouncement of terms does not help. Without a clear understanding of what these terms mean and require at the individual student, classroom and system level, school districts will not move for fear of violating some statistical tenet that invalidates all their work.

. Convene a group of individuals who have the capacity to understand both a practical and technical discussion of what constitutes a Local Assessment System. By the spring of 2003 the Maine Department of Education must be aggressively involved in leading statewide or regional efforts to demonstrate how various kinds of assessments can be used to generate data that can be joined to improve instruction, judge program effectiveness, and, ultimately, certify achievement.

By summer, a dynamic Local Assessment System model with a menu of complementary assessment strategies should be available to school districts with different instructional and administrative circumstances.

. Create a clearinghouse to assemble strategies for the assessment and certification of student achievement leading to the diploma. High schools are working in isolation without any idea of what may be acceptable in 2007. No doubt some have found useful solutions, but will those ideas ever make it across the town line? Probably not. The University of Maine is uniquely suited for this task, and, with its involvement, certification procedures can evolve, receive critical attention and have broad dissemination.

The publication of “A Nation at Risk” in 1983 and the subsequent publication of content standards in every discipline fueled the idea of high achievement for all students. By the late 1990s Maine law had made a similar statutory declaration for all students and proceeded to set very high performance standards.

Maine’s position was, in turn, trumped by the federal No Child Left Behind Act that mandated competing performance standards for all students by 2014. Aside from the fact that for all children is political rhetoric and fails to recognize the diversity of our society, Maine schools cannot retreat from the standards set five years ago.

The educational community must create an assessment system that assures that students will meet the standards necessary to live productive lives in today’s and tomorrow’s global society.

Robert Ervin, Ed.D. is superintendent of schools for the Bangor School Department.