News

After spate of outages, Palo Alto Utilities vows to improve reliability

City commissions third-party review of electric systems and procedures, upgrades to phone systems

A power outage that affected about 7,000 Palo Alto Utilities customers occurred when employees were replacing circuit breakers at the Park Boulevard substation on March 27, 2021. Photo by Magali Gauthier.

For the Palo Alto Utilities, an organization that prides itself on reliability, March 27 marked the beginning of a brutal and humbling half-month stretch.

That was when a maintenance project went awry at the Park Boulevard substation, cutting off power just before 5 p.m. for about 7,000 customers, who were without service for about two hours. The power failure was accompanied by a communication breakdown, as the city struggled to get the word out, the Utilities Department's outage map malfunctioned and hundreds of customers who called to report the outage were greeted with automatic messages, placed into queues or hung up on.

Even days later, customers and members of the media struggled to understand what caused the outage. City Manager Ed Shikada and utilities staff repeatedly referred to a "preventive maintenance project" that went wrong at the substation without defining what that means.

Ultimately, it was determined that the March 27 outage occurred when the circuit breakers were replaced at an aging substation. The project involved changing settings on substation relays — devices in power systems that disconnect an element from the system from service when it suffers a short circuit or starts to operate abnormally, according to Catherine Elvert, communications manager at the Utilities Department. Relays, she said, are designed to prevent shorts or abnormal operations from interfering with the rest of the system.

After the relays were put back into service at the substation, they initiated a circuit breaker to trip, causing the power outage.

What's local journalism worth to you?

Support Palo Alto Online for as little as $5/month.

Join

"These are abnormal operating conditions at a substation and not an occurrence that customers should anticipate will happen on a regular or frequent basis," Elvert said in an email. "It was an unusual situation which hopefully should not occur again."

A sign on the entrance of the substation on Park Boulevard in Palo Alto on April 21, 2021. Photo by Magali Gauthier.

Even despite the wide scope of the outage and the communication struggles, the March 27 outage probably wouldn't have stood out if it hadn't been followed by another power outage on April 4, when a failed underground transformer knocked out power to about 570 customers around 9 p.m. According to Utilities Director Dean Batchelor, the city was able to restore electricity to 445 customers that night; the rest didn't get their power back until 4:30 a.m.

On April 9, Batchelor provided an update about the two outages to the city's Utilities Advisory Commission.

"Our goal is to provide reliable service and we didn't do a very good job over the last two weeks with that," Batchelor said. "We understand that from the utility's perspective, reliability and safety is high on our list."

Four days later, on the evening of April 13, about 2,000 customers in the Old Palo Alto and Ventura neighborhoods lost power for about three hours, an outage that the Utilities Department attributed to a failed underground cable.

Stay informed

Get daily headlines sent straight to your inbox.

Sign up for free

Stay informed

Get daily headlines sent straight to your inbox.

Sign up for free

For a city that often touts the reliability of its municipal utilities and that is banking on electrification of cars and buildings to achieve its aggressive emission-reduction goals, the spate of outages has offered both a reality check and an opportunity for self-reflection and troubleshooting. In an April 16 post on the city's Medium page, Batchelor detailed the city's plan to address the problems that the recent outages had exposed.

The plan includes conducting a third-party review of existing systems and procedures; an evaluation of the electrical system to identify any unexpected issues; and hiring additional system operators.

Batchelor also noted that the city is working to address the limitations of the call system, including adding more capacity to receive calls and ensuring that calls won't get dropped.

"We are also working on ways to share and confirm an initial outage, in the hope of reducing customer calls into the Utilities phone system," Batchelor wrote.

According to the Utilities Department, the city averages about 25 outages a year, with recent causes including downed tree branches, floating mylar balloons and, in one recent case, a goose that flew into a power line. The most significant outage in recent years occurred on Feb. 17, 2010, when a small plane attempting a liftoff at Palo Alto Airport hit high-power lines and crashed. All three plane passengers died in the accident and all 28,000 utility customers lost power for more than 10 hours as the plane damaged all three transmission lines that the city relies on to connect to the power grid.

Since then, utility officials have explored adding an additional transmission line. City staff had spent years negotiating with Stanford University and SLAC National Accelerator Laboratory but could not reach an agreement on interconnection. In February, utilities staff reported that with the option exhausted, it is looking to explore an interconnection between the city's electric substation at Adobe Creek and PG&E's Ames substation.

"Although this connection is at the same transmission voltage level as the City’s existing interconnection, and therefore does not offer a reduction in electric transmission costs, it does significantly increase reliability and resiliency," the report states.

Reliable electricity is key to the council's goal of reducing carbon emissions by 80% by 2030, with 1990 as the baseline. According to a new study conducted by Utilities and Public Works staff, as well as the consulting firm Aecom, reaching the goal would require a wholesale conversion of gas appliances to electrical ones in the city's single-family homes as well as widespread adoption of electric vehicles.

At the April 9 meeting of the Utilities Advisory Commission, David Coale, a member of the group Carbon Free Palo Alto, urged city staff and commissioners to reconsider their effort to pursue an interconnection and to instead spend their time and resources on local resiliency, which the city hasn't quite mastered.

"If we're going to do electrification, the utility needs to go a lot further in making the system more robust to make our residents believe that electrification is going to be a good idea," Coale said.

Frank Flynn, who lives on Matadero Avenue, alluded to the March 27 outage in a recent letter to the council, noting that the electric utility "appears to be the least reliable utility we have." In the 30 years that he has lived in his home, Flynn wrote in the April 7 letter, he does not recall ever seeing his gas service go off.

"If we do want the user to switch to electric service then our electric service has to be better," Flynn wrote. "Otherwise residents will simply buy portable generators and such that are significantly worse for pollution.

"There is no (good) reason that our electrical service cannot be more reliable but it will never become more reliable unless we measure it, analyze each and every failure, and commit to fixing every issue," Flynn wrote.

Craving a new voice in Peninsula dining?

Sign up for the Peninsula Foodist newsletter.

Sign up now

Follow Palo Alto Online and the Palo Alto Weekly on Twitter @paloaltoweekly, Facebook and on Instagram @paloaltoonline for breaking news, local events, photos, videos and more.

After spate of outages, Palo Alto Utilities vows to improve reliability

City commissions third-party review of electric systems and procedures, upgrades to phone systems

by / Palo Alto Weekly

Uploaded: Thu, Apr 22, 2021, 9:05 am

For the Palo Alto Utilities, an organization that prides itself on reliability, March 27 marked the beginning of a brutal and humbling half-month stretch.

That was when a maintenance project went awry at the Park Boulevard substation, cutting off power just before 5 p.m. for about 7,000 customers, who were without service for about two hours. The power failure was accompanied by a communication breakdown, as the city struggled to get the word out, the Utilities Department's outage map malfunctioned and hundreds of customers who called to report the outage were greeted with automatic messages, placed into queues or hung up on.

Even days later, customers and members of the media struggled to understand what caused the outage. City Manager Ed Shikada and utilities staff repeatedly referred to a "preventive maintenance project" that went wrong at the substation without defining what that means.

Ultimately, it was determined that the March 27 outage occurred when the circuit breakers were replaced at an aging substation. The project involved changing settings on substation relays — devices in power systems that disconnect an element from the system from service when it suffers a short circuit or starts to operate abnormally, according to Catherine Elvert, communications manager at the Utilities Department. Relays, she said, are designed to prevent shorts or abnormal operations from interfering with the rest of the system.

After the relays were put back into service at the substation, they initiated a circuit breaker to trip, causing the power outage.

"These are abnormal operating conditions at a substation and not an occurrence that customers should anticipate will happen on a regular or frequent basis," Elvert said in an email. "It was an unusual situation which hopefully should not occur again."

Even despite the wide scope of the outage and the communication struggles, the March 27 outage probably wouldn't have stood out if it hadn't been followed by another power outage on April 4, when a failed underground transformer knocked out power to about 570 customers around 9 p.m. According to Utilities Director Dean Batchelor, the city was able to restore electricity to 445 customers that night; the rest didn't get their power back until 4:30 a.m.

On April 9, Batchelor provided an update about the two outages to the city's Utilities Advisory Commission.

"Our goal is to provide reliable service and we didn't do a very good job over the last two weeks with that," Batchelor said. "We understand that from the utility's perspective, reliability and safety is high on our list."

Four days later, on the evening of April 13, about 2,000 customers in the Old Palo Alto and Ventura neighborhoods lost power for about three hours, an outage that the Utilities Department attributed to a failed underground cable.

For a city that often touts the reliability of its municipal utilities and that is banking on electrification of cars and buildings to achieve its aggressive emission-reduction goals, the spate of outages has offered both a reality check and an opportunity for self-reflection and troubleshooting. In an April 16 post on the city's Medium page, Batchelor detailed the city's plan to address the problems that the recent outages had exposed.

The plan includes conducting a third-party review of existing systems and procedures; an evaluation of the electrical system to identify any unexpected issues; and hiring additional system operators.

Batchelor also noted that the city is working to address the limitations of the call system, including adding more capacity to receive calls and ensuring that calls won't get dropped.

"We are also working on ways to share and confirm an initial outage, in the hope of reducing customer calls into the Utilities phone system," Batchelor wrote.

According to the Utilities Department, the city averages about 25 outages a year, with recent causes including downed tree branches, floating mylar balloons and, in one recent case, a goose that flew into a power line. The most significant outage in recent years occurred on Feb. 17, 2010, when a small plane attempting a liftoff at Palo Alto Airport hit high-power lines and crashed. All three plane passengers died in the accident and all 28,000 utility customers lost power for more than 10 hours as the plane damaged all three transmission lines that the city relies on to connect to the power grid.

Since then, utility officials have explored adding an additional transmission line. City staff had spent years negotiating with Stanford University and SLAC National Accelerator Laboratory but could not reach an agreement on interconnection. In February, utilities staff reported that with the option exhausted, it is looking to explore an interconnection between the city's electric substation at Adobe Creek and PG&E's Ames substation.

"Although this connection is at the same transmission voltage level as the City’s existing interconnection, and therefore does not offer a reduction in electric transmission costs, it does significantly increase reliability and resiliency," the report states.

Reliable electricity is key to the council's goal of reducing carbon emissions by 80% by 2030, with 1990 as the baseline. According to a new study conducted by Utilities and Public Works staff, as well as the consulting firm Aecom, reaching the goal would require a wholesale conversion of gas appliances to electrical ones in the city's single-family homes as well as widespread adoption of electric vehicles.

At the April 9 meeting of the Utilities Advisory Commission, David Coale, a member of the group Carbon Free Palo Alto, urged city staff and commissioners to reconsider their effort to pursue an interconnection and to instead spend their time and resources on local resiliency, which the city hasn't quite mastered.

"If we're going to do electrification, the utility needs to go a lot further in making the system more robust to make our residents believe that electrification is going to be a good idea," Coale said.

Frank Flynn, who lives on Matadero Avenue, alluded to the March 27 outage in a recent letter to the council, noting that the electric utility "appears to be the least reliable utility we have." In the 30 years that he has lived in his home, Flynn wrote in the April 7 letter, he does not recall ever seeing his gas service go off.

"If we do want the user to switch to electric service then our electric service has to be better," Flynn wrote. "Otherwise residents will simply buy portable generators and such that are significantly worse for pollution.

"There is no (good) reason that our electrical service cannot be more reliable but it will never become more reliable unless we measure it, analyze each and every failure, and commit to fixing every issue," Flynn wrote.

Comments

No heat
Registered user
Fairmeadow
on Apr 22, 2021 at 9:16 am
No heat, Fairmeadow
Registered user
on Apr 22, 2021 at 9:16 am

Just a reminder folks: when the electric goes out, your gas-burning heater stops working too. Gas-burning heaters depend on electricity to run pumps or fans to move the heat around your house, as well as electric thermostats and electric sensors to maintain safe operation. The same likely goes for your oven.

Whether you use electricity for heating, water heating, and cooking or not, it's important for all of us to have it be reliable.


Dilettante
Registered user
Greenmeadow
on Apr 22, 2021 at 10:46 am
Dilettante, Greenmeadow
Registered user
on Apr 22, 2021 at 10:46 am

Undergrounding all of the utilities would help harden the infrastructure. Smart-meters would improve the efficiency of meter-reading.


Len Ely
Registered user
Old Palo Alto
on Apr 22, 2021 at 10:47 am
Len Ely, Old Palo Alto
Registered user
on Apr 22, 2021 at 10:47 am

Solar also doesn't work either. I have been trying to get a permit for a backup generator and that is not going very well.


Online Name
Registered user
Embarcadero Oaks/Leland
on Apr 22, 2021 at 10:50 am
Online Name, Embarcadero Oaks/Leland
Registered user
on Apr 22, 2021 at 10:50 am

They should put their ambitious plans on hold until they can get their act together.

Re their "outreach" failures, last Friday they posted an update on NextDoor urging people to sign up for the Nixle alert and then on Monday the PA Police Department sent out a similar message.

Several of us had a good laugh about that, wondering why the alert wasn't coming from PA Utilities itself, esp. when they're over-charging us $20,000,000 a year to siphon money from our pockets into the General Fund to keep funding the ridiculous salaries for employees and to keep the consultant gravy train running and then using OUR money to appeal paying us the settlement ordered in the over-charge class action suit!

How about getting their jobs DONE effectively -- including COST-effectively -- instead of doing more virtue signaling on Climate Change??


eileen
Registered user
College Terrace
on Apr 22, 2021 at 10:57 am
eileen , College Terrace
Registered user
on Apr 22, 2021 at 10:57 am

Why can't we underground our utilities? Many neighborhoods look like developing nations with all the wires everywhere! Underground utilities will be better for the city tree canopy and the general look of our streets. We can spend less money trimming our street trees. Just a thought.


Palo Alto Green
Registered user
Community Center
on Apr 22, 2021 at 11:56 am
Palo Alto Green, Community Center
Registered user
on Apr 22, 2021 at 11:56 am

Undergrounding helps with storms but is much worse during an earthquake. During the Christchurch, NZ earthquakes, the underground cables suffered far more damage, loss of electricity than above ground. It's also hard to locate failures and requires you to dig up the cable. Plus its 10X more expensive to bury cables underground.


Bystander
Registered user
Another Palo Alto neighborhood
on Apr 22, 2021 at 12:59 pm
Bystander, Another Palo Alto neighborhood
Registered user
on Apr 22, 2021 at 12:59 pm

We are in need of a much better service, that is elementary. We can't just increase the number of customers (building more offices or homes) and expect customers to use more power for EVs and heating, without improving the service. It is laughable to hear the commercials for not using power between 4 and 9, the hours that everyone is arriving home, needs to charge devices including cars, do chores, cook dinner, do homework, etc. This may be something some may be willing to do on an occasional basis say when temps are in the 100s, but in the middle of winter, forget it.

Power usage is going to rise and our service can't cope.


PA Community Advocate
Registered user
Another Palo Alto neighborhood
on Apr 22, 2021 at 8:58 pm
PA Community Advocate, Another Palo Alto neighborhood
Registered user
on Apr 22, 2021 at 8:58 pm

There has to be 20,000,000 ways to improve the poor reliability and service of Palo Alto Utilities.

Let’s start by bringing in a more qualified and professional workforce.


Don't miss out on the discussion!
Sign up to be notified of new comments on this topic.

Post a comment

In order to encourage respectful and thoughtful discussion, commenting on stories is available to those who are registered users. If you are already a registered user and the commenting form is not below, you need to log in. If you are not registered, you can do so here.

Please make sure your comments are truthful, on-topic and do not disrespect another poster. Don't be snarky or belittling. All postings are subject to our TERMS OF USE, and may be deleted if deemed inappropriate by our staff.

See our announcement about requiring registration for commenting.