mailing list archives
Re: Verizon damage and service disruptions
From: Frank Coluccio <fcoluccio () dticonsulting com>
Date: Thu, 13 Sep 2001 06:37:23 GMT
re: the NYSE not having tridundancy, and the Broad Street CO Power uncertainties:
The Pearl Street Central Office [which sits adjacent to the Brooklyn Bridge on
the East Side of City Hall] is capable and does provide back up to both the Broad
Street and West Street Central Offices on a special assembly basis. This, in
addition to supporting its own subscribers, many of whom are residential and
small business with routes running into the Chinatown area.
Some firms near the NY Stock Exchange [I know of two on Broad Street] had elected
long ago to have tri-dundancy built into their local access networks, by adding
Pearl Street to the mix. So, the NYSE has to tread lightly when they look to why
they don't have the same. All they had to do is place an order. This does raise
an interesting question, though, concerning why their other local providers (of
which, I believe, there are at least four) can't pick up some of the slack.
As for the Broad Street Central Office having some power problems, I would remind
all here that the Seaport Substation [which is just south of the Pearl Street CO]
is also having its difficulties at this time. I believe that this substation
failed yesterday under the load when 7WTC collapsed. You might recall that the
Seaport substation was the cause of another of the Wall Street's major disasters
back in the early 1990s.
A substantial part of the financial district down Water Street and Broad Street
are fed directly from this substation. Major re-grooming of feeders in this area
was done back then when the failure occurred, in order to shore up switching to
alternate substations in the event that it failed again. But this situation is
different, because now it is the Seaport station itself that the West Side grid
would depend on, and neither is working at this time. Most of what is working at
this time in the area is actually on generator.
It's going to take a lot of doing to support trading when the Stock Exchanges are
re-opened on Friday, or Monday at the latest. Another thing that I remain wary of
is the integrity of many of the underground duct bank systems supporting newly-
run fiber routes in the area. Some of these sit atop older ducts and are only a
foot or two beneath street asphalt in many places, despite standards to the
contrary, and they run up and down Church Street, which becomes Tinity Place.
These will not be tested fully until the downtown trading floors and data centers
are reopened in a couple of days. Some rings in the area are now showing failed
links on remote surveillance systems in offsite data centers, but it is not known
whether this is due to carriers taking their nodes down temporarily, or due to
underground problems, or due to anomalies at the customers own premises that are
now almost entirely unpersonneled below 14th Street and working on generator.
First, 1 Verizon employee and 3 Genuity employees are still missing, my
prayers, and I'm sure everyone else's prayers go out to their families.
No one wants to distract or impair the continuing rescue efforts in the area.
As Verizon continues its assessment of the damage, it is starting to
look worse than original reports. Verizon hasn't filed an outage
report with the FCC, or the FCC has delayed its release. So this is
based on Verizon's news conference and news reports. The reports
aren't very clear in all the details.
The walls of the 140 West ST CO adjacent to the WTO Building 7 were breached
when Building 7 collapsed, and the building has flooded and the equipment
is covered with debris. 140 West St has no power. Verizon needs to pump
out the building because it is 5 stories below ground.
Also mentioned is the Broad Street CO, which is experiencing intermittent
power problems. The Broad Street CO supplies 80% of the New York Stock
Exchange lines, and the West Steet CO serves as the alternate CO supplying
20% of the NYSE lines.
The CO's supply 200,000 voice lines and 3 million separate private lines.
Although the NYSE had backup facilities, it looks like the disaster in
NYC is disrupting service from both CO's. The telecommunication directors
of the exchanges have been meeting during the day.
According to news reports, Verizon left it unclear why a backup facility
somewhere else wouldn't be an adequate substitute for Wall Street's
financial community and stock trading. According to one news report,
NYSE Chairman Grasso acknowledged that work is needed to come up with
a better telecommunications plan in the future.