News

Committee: Palo Alto school district needs better data system

Achievement-gap committee finds data collection, analysis lacking

The Palo Alto school district's "antiquated and disjointed" data system is hindering the work of the superintendent's minority achievement and talent development committee, members said this week.

The district's data system presents a particular challenge for a data subcommittee, which is tasked with examining how subsets of students are performing, analyzing their trajectories over time and looking for patterns that might point to root causes of the achievement gap and corresponding solutions the district can act on.

The data subcommittee's work is also critical as the entire group continues to analyze and unpack five overarching problems it has identified as the main drivers of the disparity in achievement between certain minority groups and other students.

These five problems are bias against minority students that informs decision-making at all levels of the district; quality and nature of parent-student-school-community connections; instructional standards and accountability; identification and intervention structures, procedures and policies; and inequitable access and mismatched needs of students from low socio-economic backgrounds and under-represented minorities.

"The challenge has been with the data system that we currently have at the district," said district parent and Palo Alto University psychology professor Teceta Tormala, one of four committee members who make up the data subgroup. "It's not up to the task of doing what we're trying to do."

The district currently uses Cruncher Solutions, a student-data software system. Tormala said the system displays a student's current coursework, but it's harder to get at their history in the district and more nuanced details about achievement.

"You can look at a student, click on a button and see a student's current status, but to get the full picture of them over time, that's clicking through multiple screens," Tormala explained. "There isn't a way to do what I think we need to do as a committee to really understand what's happening with the system we have in place."

Subcommittee member Ze'ev Wurman said during a presentation at the committee's latest meeting Tuesday that within the current system, student data from the California Standards Test (CST) results to demographic data are spread out in different places and not all formatted the same way.

Superintendent Max McGee told the Weekly he encountered similar problems this year when trying to prepare data reports on first-grade reading benchmarks and Advanced Placement (AP) and SAT scores.

"We can get everything; we can get what we need; it's just in different areas," McGee said.

"I would characterize it as inadequate for our needs," rather than ineffective, he added.

The subgroup members Tuesday presented some preliminary findings they have drawn from data on a random sample of 33 Palo Alto students of color in the class of 2015. The group looked at whether or not the students had met the University of California and California State University's A-G subject requirements; their CST results for English Language Arts (ELA) and math; results from the ACT's EXPLORE test, administered in ninth grade; and each student's cumulative unweighted GPA senior year.

About 30 percent of the sample student group had a GPA above a 3.0 and about 30 percent have met the A-G requirements, but only about half (15 percent) of both these groups overlap.

Most, but not all, of the students who failed to meet the A-G requirements were "not proficient" based on their English and math scores on the CST and had a low-performing record early on. More than 60 percent showed a lack of proficiency in second and third grade, and many did not improve as they went through the system, Wurman said.

This underscores one of the overarching problems the committee has prioritized as a persistent driver of the achievement gap: a lack of effective early identification and intervention for struggling students.

"We cannot wait until high school," Wurman said. "The problem is manifested in high school. You have transcripts, colleges are looming ... but the problems, in most cases, started much earlier. Teachers and principals should be able to look at this data and say, 'Whoops, we should do something.'"

However, about half of students in the sample set who were chronically low-achieving in elementary and middle school still did well in high school and met the A-G requirements.

Also, the committee found that the major stumbling block to meeting A-G requirements is not math, but rather English. One hundred percent of those who did not pass the A-G coursework had low English test scores compared to only 60 percent with low math scores.

Wurman stressed that any findings at this point are tentative, as they could be unique to this group of students and have yet to be compared to Asian and white students or any previous similar cohorts (that data is not yet available). The report was also "manually generated, tedious and prone to error," he said in the presentation.

Data subcommittee member and parent April House warned of the dangers of misinterpreting piecemeal data.

"One of the outcomes may be literally coming to the wrong conclusion," she said, pointing to the fact that they had access to students' test scores and GPAs but not individual grades.

"Without data, it's going to be very hard to say what we think is anchored in reality," said Wurman, who since the committee's first meetings in December has stressed the urgent need for more concrete, complete data.

Another subcommittee is organizing focus groups with students, parents, teachers and staff to gather qualitative data. The group will also be sending out a survey in March to collect perception data on expectations for minority students, the district's intervention programs, parent engagement and access, among other topics.

With only four meetings left before the committee is expected to issue a set of recommendations to the board this spring, several members and McGee have expressed a commitment to continuing their work over the summer and into next fall. The data subcommittee is also recommending that the district "quickly and thoughtfully upgrade its data system."

"It is our fervent hope and recommendation that we actually do switch to a new system so that we can keep tracking these things over time," Tormala said, though the committee has not yet discussed specific systems that are being used by other districts or organizations that Palo Alto could implement.

McGee said newly hired Director of Research and Assessment Chris Kolar, who began work this week, will be tasked with looking at short- and long-term relief for challenges presented by the current data system.

"This has to be at our fingertips," McGee said.

The minority achievement and talent development committee's next meeting is Tuesday, March 3, at 6:30 p.m. at district headquarters, 25 Churchill Ave.

Related content:

Minority students speak to challenges they face in school district

New committee addresses Palo Alto Unified's achievement gap

Superintendent names members of achievement gap task force

'Broad action' encouraged to fix district's achievement gap

Palo Alto superintendent to convene achievement gap committee

Comments

1 person likes this
Posted by Misguided Teacher
a resident of Barron Park
on Feb 20, 2015 at 4:03 pm

But according to one teacher, students are more than data:

Web Link

So it's clear we shouldn't do this.


7 people like this
Posted by Parent
a resident of JLS Middle School
on Feb 20, 2015 at 7:16 pm

As a parent who has found getting even the most innocuous public data out of the district to be a nightmare in which district people inevitably have some kind of guilty-flee-where-no-man-pursueth kneejerk defensiveness (and usually failure to supply the requested information while making a big stink about how hard fulfilling any information request is), I am reading this and wondering if this means the district will fix its own internal processes while continuing to obstruct transparency, or they are planning on making it possible to open-source as much data as possible so the community can help, too.


1 person likes this
Posted by Wayne Martin
a resident of Fairmeadow
on Feb 22, 2015 at 1:56 pm

---
I found this article more than a little bit odd. For starters—what is a “data system”? This seems like a term used by someone who knows that there is a computer somewhere in the building, but doesn’t want to know any more about it than it “crunches numbers”.

As someone who has requested data from the PAUSD, from time-to-time, I’d have to say that most of my requests have been fulfilled. A few haven’t, but I will keep requesting until they do.

I have written a number of letters to the PAUSD BoEd about the lack of institutional IT services at the PAUSD, but never received an answer to any of these observations. One of the problems has always been the lack of a coherent technology plan that had at least a five year timeline. Such a plan would need yearly updates, as projects are completed and new projects added for future implementation. So far, it’s hard to find the BoEd calling for this sort of intelligent IT management.

The District has hired IT managers, and it also hired a PhD to do “data stuff” for the past few years. This leaves us with the obvious question—what did these people do to justify their salaries? Did they have work plans that were used to perform their yearly evaluations? Or did they just “fight fires” and not provide any long term vision of where the PAUSD should be in five years, technology wise?

Anyone working in “big data” will tell you that cleaning up data incompatibilities makes up as much as 75% of the job of actually doing “big data analysis”. So, the idea that all data should be available to anyone who wants it in any format is probably not something that is going to happen any time soon. Got to wonder if the people quoted in this article understand that?

By and large, there isn’t enough information in this article to know exactly what problems this committee is having, other than it doesn’t want to reformat data—like everyone else in the world has to do.

Perhaps someone on the committee might actually write up their problems and post them on the PAUSD web-site?

BTW—the State has been lumbering away with what they call a longtitudinal student performance tracking system for what seems like almost a decade. It would pay the PAUSD to periodically spend some time trying to understand the state of this system, with the goal of determining if this system will provide the kinds of data that is needed to track, and manage, students here in the PAUSD.


Sorry, but further commenting on this topic has been closed.

Salt & Straw Palo Alto to open Nov. 23
By Elena Kadvany | 0 comments | 4,346 views

Lakes and Larders (part 2)
By Laura Stec | 0 comments | 1,479 views

Can we ever improve our schools?
By Diana Diamond | 8 comments | 1,374 views