Achievement test data sparks heated discussion
Original post made
on Nov 9, 2011
Responding to criticism that he's excessively focused on test scores, Palo Alto School District Superintendent Kevin Skelly said Tuesday that supporting struggling students as well as top achievers is a central challenge for the school district.
Read the full story here Web Link
posted Wednesday, November 9, 2011, 9:48 AM
Posted by Michele Dauber
a resident of Barron Park
on Nov 9, 2011 at 2:17 pm
Our concern about APs is not that we don't think that they should be offered or discussed. These are both red herrings. What we do think, however, is that the report produced by the district was unreasonably celebratory, repeatedly using terms like "amazing" and "super impressive" (as did the district staff member who presented it and essentially ran out of adjectives to describe how "fantastic," "amazing" and "impressive" our AP test performance is.
1. There are a number of undefended assumptions embedded within a report about how "amazing" it all is. First, that taking a lot of AP classes and tests is a good thing. Second, that successful AP performance predicts college success. There is little, or no, evidence to support either of these assumptions. As Ken explained to the Board in his comments (not quoted or described in this story) the College Board thinks that its test predicts college success. However, scholars who have examined this issue have concluded that once you control for the variables that also predict taking an AP test (race, income, school district resources, etc.) the independent effect of the AP test on college performance drops out of the equation. AP tests themselves have no predictive power on college success. The College Board would like you to believe otherwise, but this is not based on rigorous research.
Furthermore, the District's own slides demonstrated this, in that Palo Alto, for all its ramped up AP arms racing, performs no better in college graduation rates from the UC system than students from other districts who lack our AP test-taking record.
This means it is reasonable to question the benefits of the AP program. District staff at last night's meeting displayed unalloyed cheerleading for the AP program and also a belief in the inherent goodness of the AP program that appeared to be impervious to facts. That is unfortunate because it makes it hard to ask questions about the opportunity costs associated with AP testing, which is my next point.
2. What are the costs associated with an expansive AP program. The district reported, as a clear success and all-around great news- that we have had a 25% increase in the number of AP test takers and the number of tests taken per student over the past 7 years. What is the evidence that this is "good"? In order to answer that question we would need to know what was displaced from the curriculum in order to make room for the standardized curriculum from New Jersey.
We know that the AP curriculum has often been criticized for sacrificing depth in favor of breadth. Teachers lack the freedom to develop their own advanced coursework when they are forced to teach this College Board developed curriculum. We don't know what else we might do with those resources because we apparently can't even ask that question, so deep is the article of faith that AP == good. How then are we to interpret the fact that peer schools such as Castilleja and Scarsdale High and a raft of high-status schools are dropping the AP program and developing their own classes?
Interestingly, Melissa Caswell expressed skepticism that PAUSD teachers would be up to that task, stating that the school her daughter attends, Castilleja, had decided to develop its own rigorous courses, but "of course" PAUSD teachers could "never" do that. I have more faith in our teachers and think that not only could they do that, many of them would love to do so if they were given that opportunity.
3. There is no problem with issuing a report disclosing scores. The problem is that in a report entitled "High School Achievement Measures" these were the only measures chosen for the annual report to the Board and the public. The report is an unalloyed celebration of how great it is that our kids are taking more and more AP tests and classes. There are essentially no other reports on achievement or measures in the report. When Ken suggested that we should have a broader definition of achievement and more measures, such as grades and course completion rates, the district's new "data" director, Diana Wilmot characterized such measures as "noncognitive" (!) suggesting that the only measures she considers connected to cognition are those involving standardized tests produced by the College Board and ETS.
At the end of this report, we actually know little about the picture of achievement (or lack of it) in our high schools. One part of the problem is that "achievement" in this context means "progress," that is, an update on the state of our high schools but it is more fun for the staff and board to report on achievement as if the term has a positive valence -- who is "achieving" and in PAUSD, "achieving" is a fortiori defined as "high scores on AP and SAT tests." This is a sad situation and one that is sure to lead to increased stress and decreased self-esteem on the part of students who do not fit this narrow definition of success. This leads to my fourth point.
4. Using this narrow definition of success is not just wrong it is harmful. Kevin Skelly is constantly issuing encyclicals about how we need a broader definition of success. He frequently states that it is parents, not the schools, that are responsible for the narrow definition of success that focuses on scores, tests, and college admissions. Here was a chance for him to ensure that his actions were consistent with his words on this score. This document, and the discussion that accompanied it at the board meeting, reflected precisely the narrow definition of success that he claims to be battling against.
We need leadership on this issue from the district. What are children who are not AP all-stars to gather from the message in this report about how "super amazing" all those AP test takers are? What about the fact that the majority of our minority students are not even A-G graduates, let alone taking a million APs? It is not wrong to report the scores. It is wrong to put up on screen in big letters "SUPER AMAZING" and then go on and on and on and on for over an hour, waxing rhapsodic about how great it all is.
This lapse into happy talk about SATs and APs is fun for staff, which surely must be tired of talking about mental health, suicide, and stress. But it is very important the PAlo Alto not just go back to business as usual but make some real changes.
One real change that is easy to make is not to issue reports like this one. We should be much more matter of fact when reporting on things like this rather than gloating and jumping about like children bragging about how great they did on a test.
I have no issue with AP tests. I have an issue with the way this was handled. There is no evidence to support the idea that all this AP test taking is a good thing. But even if there was such evidence, perhaps we could handle it more gracefully next time.