Home page logo

nanog logo nanog mailing list archives

RE: power failure causes and effects
From: Mark Segal <MSegal () Corporate FCIBroadband com>
Date: Fri, 15 Aug 2003 01:17:34 -0400

We have power in some pops and our co.. Genie just shut off.. Wow its quiet
now.. (fyi our co is in vaughan).


Mark Segal 
Director, Network Planning
FCI Broadband 
Tel: 905-284-4070 
Fax: 416-987-4701 

Futureway Communications Inc. is now FCI Broadband

-----Original Message-----
From: Mehmet Akcin [mailto:mehmet () akcin net] 
Sent: August 15, 2003 1:10 AM
To: nanog () merit edu
Subject: Re: power failure causes and effects

I have been hearing on the TV that some places that had power failure have
started getting their power back, reporters say hopefully by the morning all
of the places where had power failure will back online.

Mehmet Akcin

----- Original Message ----- 
From: "Marshall Eubanks" <tme () multicasttech com>
To: "Fred Heutte" <aoxomoxoa () sunlightdata com>; <nanog () merit edu>
Sent: Friday, August 15, 2003 12:51 AM
Subject: Re: power failure causes and effects

On Thu, 14 Aug 2003 19:12:04 -0700
 Fred Heutte <aoxomoxoa () sunlightdata com> wrote:

It looks like DC may have had a close call - the University of 
dark for about 1 hour, and I have heard several news reports here 
stating that the local (DC regional) power grid just managed to 
decouple from wider East Coast grid in time to avoid a collapse here.

Marshall Eubanks

Most of the early rumors about causes of the power failure have 
proven incorrect -- fire at a New York City power plant, etc.

Most likely is a congestion failure in the Niagara-Mohawk grid, 
which covers a large part of New York state and has feeders across 
into Canada.

Congestion failures happen when power flows through a particular 
switching station are high, and a component fails either directly or 
because of a power surge caused by a failure elsewhere.

The imbalance caused by an open or short circuit will then 
immediately spread through the rest of the grid unless action is 
taken to disconnect, or point failures occur (resulting in fires and 
explosions from degradation of power transmission equipment, 
transformers, etc., not a pretty outcome at all) -- or both.  In 
general, transmission grids are designed from a failsafe 
perspective, meaning that it is much safer to cause rolling 
brownouts or blackouts than to let key components such as 
transmission substations or power plants have a catastrophic 

Since the entire grid has to be in sync and supply and demand must 
be in relative parity at all times, the usual strategy in these 
cases is to isolate the affected area, "island it" by shutting down 
power ingress and egress (tripping safety breakers at major crossing 
points), and shutting down power plants in the vicinity that will 
have stress failures if they don't have sufficient load to balance 
their output.

The problem is that power travels faster than even the highest-speed 
switching equipment can operate, so the surges causing a cascading 
failure like this afternoon's can spread very quickly, like ripples 
in a pond.

The weather was hot and humid but completely within
range for the time of year, so this has to be counted as a "normal 
accident."  It's likely that whatever component initially failed and 
triggered the shutdown was within its usual tolerances and simply 
had an ordinary breakdown. Of course, to spread, a massive outage 
like this also exposes other weaknesses and hidden dependencies in 
the system, which might be other physical components, software,
operator error, etc.

The system is remarkably resilient in most circumstances, which is 
what five-nines or more is all about.  But rust never sleeps, and 
underinvestment in key transmission corridors in New England, New 
York north of Manhattan and in parts of the Midwest is no doubt an 
underlying cause.

As to the root cause of that engineering problem -- the answer is 
politics, some of it congressional, and I will say no more in this 

Fred Heutte

Portland, Oregon
energy policy analyst and net geek

  By Date           By Thread  

Current thread:
[ Nmap | Sec Tools | Mailing Lists | Site News | About/Contact | Advertising | Privacy ]