Re: Zayo woes

[michael brooks - ESC <michael.brooks () adams12 org>]

If it makes you feel any better, we have been waiting for a simple BCI
Circuit from Comcast for over a year (not 100% their fault, they had to
permit a canal crossing). Two weeks ago, we received a construction
completion email from our account manager, and I scheduled a turnup for the
following week. Then the construction crew showed up the next day....




michael brooks
Sr. Network Engineer
Adams 12 Five Star Schools
michael.brooks () adams12 org
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
"flying is learning how to throw yourself at the ground and miss"



On Tue, Nov 14, 2023 at 11:04 AM Billy Earley via NANOG <nanog () nanog org>
wrote:

> Navigating the Zayo maze has become my five-month odyssey, attempting to
> breathe life into a new wave circuit. Yet, at every turn, the elusive
> Letter of Authorization (LOA) emerges as a formidable opponent, adorned
> with syntax problems or entangled in the chaos of already occupied ports.
>
> Securing new LOAs has evolved into a drawn-out saga, reminiscent of a
> never-ending loop that defies both reason and resolution. Weeks are spent
> coaxing these documents into existence, only to find myself caught in the
> clutches of the same cycle, ad infinitum.
>
> Communication with Zayo has been challenging. Phone calls always go
> unanswered. Persistent radio-silence despite emails promising regular
> updates. Zayo's website boldly touts "waves on demand" with a 24-hour
> turn-up, a tantalizing promise that seems more mirage than reality in my
> current struggle.
>
> Wish me luck as I continue my quest through the Zayo wilderness. May the
> waves finally align in my favour some day soon. If anyone at Zayo is
> reading this - I would love to hear from you off-list.
>
>
>
> On Sep 21, 2023, at 01:15, Richard Holbo <holbor () sonss net> wrote:
>
> Laughing out Loud, really, good views all...
> Having been through this a few times.. and being one of those who is now
> the one of the hated C level guys..
> Much truth is spoken here.  EBITA and size are the issues IMHO in our
> current system.
> Having been the owner of a few "smallish" retail ISPs in the west.. I can
> say with certainty that size is an issue.  in the world of today a
> small/medium ISP can do OK, but can never access the funds or resources
> necessary to actually be a long term survivor.  To that end we look at
> sources of money that require us to sell a majority interest in our company
> to make it to the next level, that and exhaustion from staying awake nights
> wondering how to make next payroll because we spent a couple million on new
> gear.
> This money seeking then comes with an interview system where we court the
> guys with money and they court us.. The outcome is always iffy.  You can
> partner with money that has a good plan or one that sucks (done both).
> Even if you get good money, you will suffer from culture issues.  Small
> ISP's tend to focus on the people and the customers, the larger you get the
> more important the money becomes (EBITA) which for a lot of the employees
> is, well, hard.
> But staying small in general = extinction so  you do what you do to keep
> employees working (at least that's what I tell myself).
> Good views all, and I totally agree with @Matthew Petach miracles
> statement.. I'm no longer that guy, but I like to think I was in the past,
> and I've got a bunch of them working for me now, so I try really hard to
> appreciate it and to recognize that that is what is happening, I fight for
> the money to do it as right as possible.. but I do depend on .... you
> miracle workers.
>
> /thanks
> /rh
>
> On Wed, Sep 20, 2023 at 9:30 AM Matthew Petach <mpetach () netflight com>
> wrote:
>
> > On Tue, Sep 19, 2023 at 12:21 PM Mike Hammett <nanog () ics-il net> wrote:
> >
> > > Well sure, and I would like to think (probably mistakenly) that just no
> > > one important enough (to the money people) made the money people that these
> > > other things are *REQUIRED* to make the deal work.
> > >
> > > Obviously, people lower on the ladder say it all of the time, but the
> > > important enough money people probably don't consider those people
> > > important enough to listen to.
> >
> >
> > Not quite.
> >
> > It's more of what Mark said:
> >
> > "  I blame this on the success of how well we have built the Internet
> > with whatever box and tool we have, as network engineers."
> >
> > I have worked time and time again with absolute miracle workers in the
> > networking field.
> > They say over and over again "to make this work, we need $X M to get the
> > right hardware", even directly to the CFO.
> >
> > They get handed a roll of duct tape, some baling wire, a used access
> > point and a $25 gift card from Office Depot, and they turn it into a
> > functional BGP-speaking backbone, because that's what they're good at.
> >
> > The CFO and the rest of the executives that said "no" to the request for
> > $X M to make the integration work properly pat themselves on the back,
> > saying "see, we knew they didn't really NEED that money to make it work."
> >
> > A year down the line, customers are posting to NANOG wondering why things
> > are going to hell in a handbasket at ISP A, as the BGP-speaking access
> > point with some duct tape, baling wire, and SFPs purchased from Office
> > Depot that ties the two networks together starts failing.
> >
> > As network engineers, we collectively set ourselves up for this by being
> > so damn good at pulling miracles out of our backside to keep things running.
> > We've effectively been training our executives that if they habitually
> > turn down our requests for resources, we'll still find some way of making
> > things work.
> >
> > We pride ourselves on being able to keep a dozen spinning plates going
> > like a circus performer, without letting any of them crash to the floor.
> >
> > It's a hard thing to do, but one lesson I've taught junior network
> > engineers of all ages is that sometimes, you have to step back, and watch a
> > plate smash into the floor, *even if you could have rescued it*, if it
> > seems like that's the only way your executive team will understand that if
> > requests for necessary resources are denied, there will be operational
> > impacts.
> >
> > Now, it's not something you should do lightly, and not something to do
> > without first working with the executives to understand why the resource
> > request is being denied.
> > If you are working at a startup, and the money is running out, and the
> > company is one step ahead of the creditors, probably not the time to put
> > the foot down and intentionally let things crash and burn.
> >
> > But if the company is doing well, has the money, and the executives just
> > want the numbers to look good for wall street analysts, then it's time to
> > pause the miracle working, and help them understand that they cannot simply
> > expect you to pull a miracle out of your backside every time, just so they
> > can look good.
> >
> > If we continue to pull off miracles after telling executives that
> > additional resources are required, it's no wonder they don't take the
> > requests as seriously as they should.  ^_^;
> >
> > Matt
> >
> >
> >
> >
> > > ------------------------------
> > > *From: *"Mark Tinka" <mark@tinka.africa>
> > > *To: *nanog () nanog org
> > > *Sent: *Tuesday, September 19, 2023 10:28:26 AM
> > > *Subject: *Re: Zayo woes
> > >
> > >
> > >
> > > On 9/19/23 16:48, Mike Hammett wrote:
> > >
> > > As someone that has been planning to be in the acquiring seat for a
> > > while (but yet to do one), I've consistently passed to the money people
> > > that there's the purchase price and then there's the % on top of that for
> > > equipment, contractors, etc. to integrate, improve, optimize future
> > > cashflow, etc. those acquisitions with the rest of what we have.
> > >
> > >
> > > I blame this on the success of how well we have built the Internet with
> > > whatever box and tool we have, as network engineers.
> > >
> > > The money people assume that all routers are the same, all vendors are
> > > the same, all software is the same, and all features are easily deployable.
> > > And that all that is possible if you can simply do a better job finding the
> > > cheapest box compared to your competition.
> > >
> > > In general, I don't fancy nuance when designing for the majority. But
> > > with acquisition and integration, nuance is critical, and nuance quickly
> > > shows that the acquisition was either underestimated, or not worth doing at
> > > all.
> > >
> > > Mark.

-- 
This is a staff email account managed by Adams 12 Five Star Schools.  This 
email and any files transmitted with it are confidential and intended 
solely for the use of the individual or entity to whom they are addressed. 
If you have received this email in error please notify the sender.