The Palo Alto school district's "antiquated and disjointed" data system is hindering the work of the superintendent's minority achievement and talent development committee, members said this week.
The district's data system presents a particular challenge for a data subcommittee, which is tasked with examining how subsets of students are performing, analyzing their trajectories over time and looking for patterns that might point to root causes of the achievement gap and corresponding solutions the district can act on.
The data subcommittee's work is also critical as the entire group continues to analyze and unpack five overarching problems it has identified as the main drivers of the disparity in achievement between certain minority groups and other students.
These five problems are bias against minority students that informs decision-making at all levels of the district; quality and nature of parent-student-school-community connections; instructional standards and accountability; identification and intervention structures, procedures and policies; and inequitable access and mismatched needs of students from low socio-economic backgrounds and under-represented minorities.
"The challenge has been with the data system that we currently have at the district," said district parent and Palo Alto University psychology professor Teceta Tormala, one of four committee members who make up the data subgroup. "It's not up to the task of doing what we're trying to do."
The district currently uses Cruncher Solutions, a student-data software system. Tormala said the system displays a student's current coursework, but it's harder to get at their history in the district and more nuanced details about achievement.
"You can look at a student, click on a button and see a student's current status, but to get the full picture of them over time, that's clicking through multiple screens," Tormala explained. "There isn't a way to do what I think we need to do as a committee to really understand what's happening with the system we have in place."
Subcommittee member Ze'ev Wurman said during a presentation at the committee's latest meeting Tuesday that within the current system, student data from the California Standards Test (CST) results to demographic data are spread out in different places and not all formatted the same way.
Superintendent Max McGee told the Weekly he encountered similar problems this year when trying to prepare data reports on first-grade reading benchmarks and Advanced Placement (AP) and SAT scores.
"We can get everything; we can get what we need; it's just in different areas," McGee said.
"I would characterize it as inadequate for our needs," rather than ineffective, he added.
The subgroup members Tuesday presented some preliminary findings they have drawn from data on a random sample of 33 Palo Alto students of color in the class of 2015. The group looked at whether or not the students had met the University of California and California State University's A-G subject requirements; their CST results for English Language Arts (ELA) and math; results from the ACT's EXPLORE test, administered in ninth grade; and each student's cumulative unweighted GPA senior year.
About 30 percent of the sample student group had a GPA above a 3.0 and about 30 percent have met the A-G requirements, but only about half (15 percent) of both these groups overlap.
Most, but not all, of the students who failed to meet the A-G requirements were "not proficient" based on their English and math scores on the CST and had a low-performing record early on. More than 60 percent showed a lack of proficiency in second and third grade, and many did not improve as they went through the system, Wurman said.
This underscores one of the overarching problems the committee has prioritized as a persistent driver of the achievement gap: a lack of effective early identification and intervention for struggling students.
"We cannot wait until high school," Wurman said. "The problem is manifested in high school. You have transcripts, colleges are looming ... but the problems, in most cases, started much earlier. Teachers and principals should be able to look at this data and say, 'Whoops, we should do something.'"
However, about half of students in the sample set who were chronically low-achieving in elementary and middle school still did well in high school and met the A-G requirements.
Also, the committee found that the major stumbling block to meeting A-G requirements is not math, but rather English. One hundred percent of those who did not pass the A-G coursework had low English test scores compared to only 60 percent with low math scores.
Wurman stressed that any findings at this point are tentative, as they could be unique to this group of students and have yet to be compared to Asian and white students or any previous similar cohorts (that data is not yet available). The report was also "manually generated, tedious and prone to error," he said in the presentation.
Data subcommittee member and parent April House warned of the dangers of misinterpreting piecemeal data.
"One of the outcomes may be literally coming to the wrong conclusion," she said, pointing to the fact that they had access to students' test scores and GPAs but not individual grades.
"Without data, it's going to be very hard to say what we think is anchored in reality," said Wurman, who since the committee's first meetings in December has stressed the urgent need for more concrete, complete data.
Another subcommittee is organizing focus groups with students, parents, teachers and staff to gather qualitative data. The group will also be sending out a survey in March to collect perception data on expectations for minority students, the district's intervention programs, parent engagement and access, among other topics.
With only four meetings left before the committee is expected to issue a set of recommendations to the board this spring, several members and McGee have expressed a commitment to continuing their work over the summer and into next fall. The data subcommittee is also recommending that the district "quickly and thoughtfully upgrade its data system."
"It is our fervent hope and recommendation that we actually do switch to a new system so that we can keep tracking these things over time," Tormala said, though the committee has not yet discussed specific systems that are being used by other districts or organizations that Palo Alto could implement.
McGee said newly hired Director of Research and Assessment Chris Kolar, who began work this week, will be tasked with looking at short- and long-term relief for challenges presented by the current data system.
"This has to be at our fingertips," McGee said.
The minority achievement and talent development committee's next meeting is Tuesday, March 3, at 6:30 p.m. at district headquarters, 25 Churchill Ave.