������Ƶ

Civil Beat reported recently on a bill supported by the Hawaii State Teachers Association that seeks to limit testing. An important aspect of this question that I would like to address is the educational impact of the Smarter Balanced Assessment, which Hawaii students take in grades three to eighth, then once more in grade 11.

Schools Superintendent Schools Superintendent Christina Kishimoto was quoted on KHON recently as saying that standardized testing is valid and reliable. She wasn’t speaking specifically about SBA, the test that is at the heart of the matter.

However, due to the great weight placed on SBA as the test chosen to fulfill our obligations under the Every Student Succeeds Act, and given the fact that the assessment community has ruled overwhelmingly against the assertion that SBA is valid and reliable, I believe it is necessary to question the superintendent’s remark.

I’m speaking as a full-time Advancement Placement teacher and an official reader for the College Board who participates each summer in the scoring process for the AP Language and Composition test that takes place on the mainland. Because I have an intimate knowledge of the high quality assessments that are a major component of the Advanced Placement program, I believe I am in a good position to comment on the shortcomings of SBA.

I also have a certain insider’s insight to SBA, having served on the item-writing committee for the Language Arts portion during its startup year 2013-2014. This was an eye-opening experience.

Prior to writing test questions, committee members received training in various assessment methodologies and procedures, and I was initially impressed with the complexity and seeming rigor of the process which led to our choice of texts featured on the assessment and to the construction of test items based on them. Subsequently, however, my enthusiasm was heavily qualified by what I felt was the mediocre quality of the items we actually produced.

I won’t go into details, but if I had to characterize the mediocrity of SBA, I would say on the one hand that it’s a proverbial case of “too many cooks in the kitchen,” and on the other hand that it is too slavishly bound to item-by-item alignment with the Common Core State Standards to produce test questions that are not confusing even to expert adult readers, or to accurately measure what the test claims to measure.

Teachers Feel Pressured

I want to emphasize that, as an AP teacher, I love good tests and I use them all the time. While I have never served on the test-writing committee for the College Board, I am certain that the AP item-writing process is less rigid and “cookie-cutter” than with SBA because the AP assessment criteria do not contain a list of rigidly prescribed learning benchmarks that item-writers must adhere to, whether the texts they have selected and that have been approved for item-writing are suitable for assessing these benchmarks or not.

I also know that, in contrast to SBA, AP test items are heavily field tested and that there may be a gap of several years from the construction of a particular test item to the time it appears on an official test.

I’d also like to comment on my experience with SBA at the high school level.

At my high school I feel we’ve done a good job of limiting the test’s overall intrusiveness on the school year. As a team we seem to have come to an informal consensus that the test is poor in quality and limited in information it gives us on our students, but also that we need to fulfill our obligations by encouraging our students to make their best effort and by providing them with at least minimal preparation for an online test environment that is somewhat idiosyncratic and that they are not used to.

However, I know from conversations with teachers at other high schools that there are administrative and curriculum teams that place more emphasis on the test than we do, and I also know that elementary and middle schools are in general more heavily impacted by the amount of testing and practice testing that is expected and the amount of teaching to tests of low quality that many teachers feel pressured into.

In general, there are an insufficient number of test items to assess what SBA claims to assess.

Although we have minimized the amount of time we devote to SBA and its potentially negative impact on students at my high school, from a personal standpoint, any amount of time that SBA takes away from my duties as an AP teacher is too much time. In addition to losing valuable class time by devoting at least two days of the school year to preparing 11th-grade AP students for SBA and at least one more day to its administration, we are also obliged to use valuable collaboration time after school analyzing the dubious data that we receive from the test.

The major problem with this, contrary to the claims of SBA advocates, is that the data is fairly meaningless if we do not also have access to the tests that students took (and Smarter Balanced, contrary to normative assessment practice, does not release exam material for teacher perusal) and if we also lack the standard written explanation of particular test items in terms of the benchmarks they are supposedly aligned to.

One example will suffice. We receive information on needs and strengths based on quantitative analysis of student performance on the variety of learning benchmarks that SBA items supposedly assess. But if I am told, for instance, that “identifying main themes and key ideas” is a “need” area but I am unable to examine the test subsequently to determine whether there were enough test items reflecting the benchmark in question to legitimately assess it, or whether the test items were well-written, then the data analysis activity is largely meaningless.

Having served on the item-writing committee, I can answer both questions in the negative: In general, there are an insufficient number of test items to assess what SBA claims to assess; more significantly, the test items themselves are too inconsistent in quality to assess much more than student ability to outwit a poorly designed assessment.

I would like to refer interested readers to a research document from 2016 that was written and signed by a group of 100 California education professors and that is available at the California Alliance of Researchers for Equity in Education . It is a 10-page summary of the numerous criticisms that SBA and the other Common Core-aligned tests including PARCC have received.

I believe that it is incumbent on everyone responsible for making crucial decisions involving standardized testing to familiarize themselves with the expert opinion of the educational research and assessment communities. The superintendent’s remarks suggest that the Hawaii Department of Education has not done so. The report is a good indication that Hawaii teachers have been forced to waste their and their students’ valuable time on a test of very low quality.

Thoughts on this or any other story? Write a Letter to the Editor. Send to news@civilbeat.org and put Letter in the subject line. 200 words max. You need to use your name and city and include a contact phone for verification purposes. And you can still comment on stories on.