Re: puck not responding

[George Herbert]

If it wasn’t for how clunky they are with email sites, I’d suggest moving to a 
cloud somewhere.  But …

-George 

Sent from my iPhone

> On Feb 29, 2024, at 8:01 AM, Jared Mauch  wrote:
> 
> 
> 
>> On Feb 29, 2024, at 10:56 AM, Jay Acuna  wrote:
>> 
>>> On Thu, Feb 29, 2024 at 9:22 AM Jared Mauch  wrote:
>>> 
>> 
>> Apparently some of the most important email lists, Outages, etc, are
>> being kept online by 1 person's  Unix/Linux server.
>> 
> 
> There’s other people who have access etc, but when it comes to hardware that 
> is quite old, last substantive refresh was in 2011, it’s served its purpose 
> well.
> 
> Obligatory xkcd https://xkcd.com/2347/
> 
> 

Re: AWS WAF list

[George Herbert]

This is terrible advice, but you might need another netblock for the
eyeballs.  Possibly a small one with enterprise NAT, but something outside
the AWS list ranges...


-George

On Mon, Feb 19, 2024 at 7:35 PM Justin H.  wrote:

> That matches my experience with these types of problems in the past.
> Especially when the end-users don't have a process for white-listing.
> We actually got a response from one WAF user to "connect to another
> network to log in, then you should be able to use the site, because it's
> just the login page that's protected".
>
> I am working with someone off-list, so I have hope this can be resolved
> without account gymnastics. :)
>
> Justin H.
>
> Owen DeLong wrote:
> > The whole situation with these WAF as a service setups is a nightmare
> for the affected (afflicted) parties.
> >
> > I saw this problem from both sides when I was at Akamai. It’s not great
> from the service provider side, but it’s an absolute shit show for anyone
> on the wrong side of a block. There’s no accountability or process for
> redress of errors whatsoever. The impacted party isn’t a customer of the
> WAF publisher, so they cant get any traction there. The WAF subscriber
> blindly applies the WAF and it’s virtually impossible to track down anyone
> there who even knows that they subscribe to such a thing, let alone get
> them to take useful action.
> >
> > Best of luck.  The only thing I saw that worked while I was at Akamai
> was a few entities subscribed to the WAF service and then complained about
> getting blocked from their own web sites. Since they were then Akamai WAF
> customers, they could get Akamai to take action.
> >
> > Crazy.
> >
> > Owen
> >
> >
> >> On Feb 16, 2024, at 09:19, Justin H.  wrote:
> >>
> >> Justin H. wrote:
> >>> Hello,
> >>>
> >>> We found out recently that we are on the HostingProviderIPList (found
> here
> https://docs.aws.amazon.com/waf/latest/developerguide/aws-managed-rule-groups-ip-rep.html)
> at AWS and it's affecting our customers' access to various websites.  We
> are a datacenter, and a hosting provider, but we have plenty of enterprise
> customers with eyeballs.
> >>>
> >>> We're finding it difficult to find a technical contact that we can
> reach since we're not an AWS customer.  Does anyone have a contact or
> advice on a solution?
> >> Sadly we're not getting any traction from standard AWS support, and end
> users of the WAF list like Reddit and Eventbrite are refusing to whitelist
> anyone.  Does anyone have any AWS contacts that might be able to assist?
> Our enterprise customers are becoming more and more impacted.
> >>
> >> Justin H.
>
>

-- 
-george william herbert
george.herb...@gmail.com

Re: 365 Datacenters Tampa AC Failure

[George Herbert]

Oof.  Get ready to replace all spinning media you may have there.  

-George 

Sent from my iPhone

> On Jun 12, 2023, at 4:06 PM, Nick Olsen  wrote:
> 
> 
> Just a heads up to anyone else colo'd at 365 TPA1/TAMSFLDE. Currently seeing 
> floor temps of ~105F as reported by equipment. Started yesterday at ~5:30PM 
> eastern. 2nd AC failure in the last 30 days. They have not sent any advisory 
> notices as of yet.

Re: China Telecom in Hunan office tower fire

[George Herbert]

I think “the whole building burned” is a bit hyperbolic.  Building was covered 
in the now known to be spectacularly flammable exterior foam insulation panels. 
 Those panels are now largely banned because of several fires.

It had intact windows and fire sprinklers when the cladding ignited on one side 
at the lower floors pedestal (A/C equipment on pedestal roof?  Unknown).  It 
burned up one side and marginally around its corners.  Most windows eventually 
failed in the fire but sprinklers held and internal damage was only moderate.  
Building occupants safety measures held reasonably from what I have heard, 
evacuated safely.

It’s going to take a lot of repair but wasn’t that catastrophic.  No sign that 
it’s going to need to be demolished or anything crazy.

Should not have happend… but life safety systems worked.


-george 

Sent from my iPhone

Re: "Permanent" DST

[George Herbert]

> 
> On Mar 15, 2022, at 2:06 PM, Jay Ashworth  wrote:
> 
> It violates the international rule determining what your time zone should be 
> based on what your longitude is. 
> 
> That is not trivial.

It’s an informal convention, not “rule”, and it  not vaguely consistent in 
practice now.  You’re attributing a consensus to what’s practically chaos.  
Look at all the headaches the TZ people are dealing with now.  This simplifies 
things considerably.

-george 

Sent from my iPhone

Retracted: Re: RU evidently hijacked UA netblock

[George Herbert]

I don’t know about Scott’s situation but the original hijack report was shown 
to have an innocent explanation.  My apologies.


-george 

Sent from my iPhone

> On Mar 4, 2022, at 6:06 PM, Scott Weeks  wrote:
> 
> 
> 
> --- george.herb...@gmail.com wrote:
> 
> https://bgpstream.com/event/287556
> 
> Beware of further such activity…
> 
> ---
> 
> 
> I have a ticket open with my vendor, but I see strange NLRI buffer overflow 
> syslog messages about Khazkstan's AS21299 (TNSPLUS) announcements.  It looks 
> like a 'too many' AS prepends, but it is only 250 prepends.  Could be a 
> mistake or intentional. 
> 
> I get those from no other ASNs and I am sure some AS sent 250 AS path 
> prepends before.  Anyone else see stuff from them?
> 
> scott

RU evidently hijacked UA netblock

[George Herbert]

https://bgpstream.com/event/287556

Beware of further such activity…

-george 

Sent from my iPhone

Re: Ukraine request yikes

[George Herbert]

I don’t hear anyone in the networks field supporting doing it.

It was a yikes that the request was made, but not looking at all likely to 
happen IMHO.

-george  

Sent from my iPhone

> On Mar 1, 2022, at 2:12 PM, Brian R  wrote:
> 
> 
> The problem with all this talk, especially with trusted international neutral 
> organizations, is that once they bend they will never be trusted again.  
> Shutting off the routes, removing TLDs (or keeping them because of politics), 
> etc will cause irreparable damage to these organizations.  Bowing to 
> governments, politics, etc does not have a path back from future control.
> This is a recommendation that will only hurt people (China, North Korea, 
> [even the USA], etc all do this to control their people).  Governments will 
> get around whatever the limitations are, it may take them time and resources 
> but they will get around it.  Freedom of information is the only way to help 
> people understand the reality of what is going on in the world (galaxy, 
> universe, etc).
> 
> Brian
> Technological solutions for Sociological problems 
> 
> From: NANOG  on behalf of 
> Bryan Fields 
> Sent: Tuesday, March 1, 2022 1:23 PM
> To: nanog@nanog.org 
> Subject: Re: Ukraine request yikes
>  
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA256
> 
> 
> On 3/1/22 4:08 PM, David Conrad wrote:
> > See .SU.
> >
> > (SU was moved from allocated to "transitionally reserved” back when the
> > USSR broke up. My recollection is that an agreement was reached by which
> > .SU users would be migrated out to appropriate new ccTLDs, that is, the
> > ccTLDs based on ISO codes created for former Soviet republics, and no new
> > entries would be added to .SU. However, when ICANN tried to propose a plan
> > to finalize removing .SU from the root (around 2006 or so), the operators
> > of .SU reopened registrations and complained to the US Dept. of Commerce,
> > who were overseeing ICANN performance of the IANA Functions contract.
> > Eventually, the Russian government was able to convince ISO-3166/MA to move
> > SU to “exceptionally reserved” (like UK, EU, and a number of others) and
> > forward motion on removing .SU from the root essentially ceased.)
> 
> I know someone (non-Russian) using .su for a funny name ending in .su.  This
> is non-political and caters only to an English speaking audience.  These were
> registered in the last few years, so they are still open and taking the
> registrations.
> 
> I would ask what of .ly used for various URL shorteners, and .kp or .cn?  All
> these are representing evil countries too, why do they get a pass.  I'm
> certain they would argue .us should be revoked for the same.
> 
> This would break connectivity, and that's a bad thing.
> - -- 
> Bryan Fields
> 
> 727-409-1194 - Voice
> http://bryanfields.net
> -BEGIN PGP SIGNATURE-
> 
> iQIzBAEBCAAdFiEEaESdNosUjpjcN/JhYTmgYVLGkUAFAmIejrUACgkQYTmgYVLG
> kUA+QQ//Z9ovTSFqVEunql2guHAN3xWaNpCpuNJCGM68dJTBSWrPEY0zFXlmZG1k
> 0TWrSrRSoWogiJmRvaOuFx6KxkaADqZaZq6OFaCw3jvyFGULw+auyuATGlhnUL8p
> CV0AbovPUnoAef1qJdglFkqnfrGBxeBGsgRIM8tx2l/G+zq5MdMnCx9cM+JmmN1y
> b+jrV4oekgXRZLAMI/sA9clMAXUmlgReRvit8YBccunkmMP8naQ92vj9dvVGZld0
> hGguK2a7vFXpDiW5o0nFe5GRdGIqM0aWUz6p0qkB9JudkZkAyEqSpCePZky4LdAt
> ebh9544PZu/vllQjv3L6vENlCURcifTIRSevcwfKZtos7UG4mJI1UQ51OLTRjB7a
> nqYkVNJSQJ+dXZFLPoRHNUOu4+1MAyozpDeMJzMsr4a7Ru2lh0AOTiXxDaSRhOd+
> 2s3rQigh/l6cP/x9iM7+f+rInHzPihHfjbwcxhyqd12EFxgTe3hvi9JlRSe18RYw
> bnDKQg3xKp1eIk0sZMeLyIWDERjsMxIuEP9MuKHp+oTCrLq6MFSgUiFan7M5Pk2t
> mwB3sbFuwkVzfmDbbnbelll30ukXQM3d7KVp2AHbsvI6hNs6zHZgRb7ZgGrR9Ep5
> 6UlYqVqQOWtYNujNxYRgzemFI6lgJj8GHyDeh0wLRCP0aw/ATPg=
> =KK8e
> -END PGP SIGNATURE-

Ukraine request yikes

[George Herbert]

Posted by Bill Woodcock on Twitter… 
https://twitter.com/woodyatpch/status/1498472865301098500?s=21

https://pastebin.com/DLbmYahS

Ukraine (I think I read as) want ICANN to turn root nameservers off, revoke 
address delegations, and turn off TLDs for Russia.

Seems… instability creating…

-george

Sent from my iPhone

Re: massive facebook outage presently

[George Herbert]

And WhatsApp and Instagram.  Twitter users nationwide agree anecdotally.

What I’m getting is DNS failure. 

-George 

Sent from my iPhone

> On Oct 4, 2021, at 9:07 AM, Eric Kuhnke  wrote:
> 
> 
> https://downdetector.com/status/facebook/
> 
> Normally not worth mentioning random $service having an outage here, but this 
> will undoubtedly generate a large volume of customer service calls. 
> 
> Appears to be failure in DNS resolution.
> 

Re: Rack rails on network equipment

[George Herbert]

(Crying, thinking about racks and racks and racks of AT&T 56k modems strapped 
to shelves above PM-2E-30s…)

The early 90s were a dangerous place, man.

-George 

Sent from my iPhone

> On Sep 24, 2021, at 8:05 PM, Wayne Bouchard  wrote:
> 
> Didn't require any additional time at all when equipment wasn't bulky
> enough to need rails in the first place
> 
> 
> I've never been happy about that change.
> 
> 
>> On Fri, Sep 24, 2021 at 09:37:58AM -0700, Andrey Khomyakov wrote:
>> Hi folks,
>> Happy Friday!
>> 
>> Would you, please, share your thoughts on the following matter?
>> 
>> Back some 5 years ago we pulled the trigger and started phasing out Cisco
>> and Juniper switching products out of our data centers (reasons for that
>> are not quite relevant to the topic). We selected Dell switches in part due
>> to Dell using "quick rails'' (sometimes known as speed rails or toolless
>> rails).  This is where both the switch side rail and the rack side rail
>> just snap in, thus not requiring a screwdriver and hands of the size no
>> bigger than a hamster paw to hold those stupid proprietary screws (lookin
>> at your, cisco) to attach those rails.
>> We went from taking 16hrs to build a row of compute (from just network
>> equipment racking pov) to maybe 1hr... (we estimated that on average it
>> took us 30 min to rack a switch from cut open the box with Juniper switches
>> to 5 min with Dell switches)
>> Interesting tidbit is that we actually used to manufacture custom rails for
>> our Juniper EX4500 switches so the switch can be actually inserted from the
>> back of the rack (you know, where most of your server ports are...) and not
>> be blocked by the zero-U PDUs and all the cabling in the rack. Stock rails
>> didn't work at all for us unless we used wider racks, which then, in turn,
>> reduced floor capacity.
>> 
>> As far as I know, Dell is the only switch vendor doing toolless rails so
>> it's a bit of a hardware lock-in from that point of view.
>> 
>> *So ultimately my question to you all is how much do you care about the
>> speed of racking and unracking equipment and do you tell your suppliers
>> that you care? How much does the time it takes to install or replace a
>> switch impact you?*
>> 
>> I was having a conversation with a vendor and was pushing hard on the fact
>> that their switches will end up being actually costlier for me long term
>> just because my switch replacement time quadruples at least, thus requiring
>> me to staff more remote hands. Am I overthinking this and artificially
>> limiting myself by excluding vendors who don't ship with toolless rails
>> (which is all of them now except Dell)?
>> 
>> Thanks for your time in advance!
>> --Andrey
> 
> ---
> Wayne Bouchard
> w...@typo.org
> Network Dude
> http://www.typo.org/~web/

Re: Rack rails on network equipment

[George Herbert]

I’ve seen Dell rack equipment leap for safety (ultimately very very 
unsuccessfully…) in big earthquakes.  Lots of rack screws for me.

-George 

Sent from my iPhone

> On Sep 24, 2021, at 9:41 AM, Andrey Khomyakov  
> wrote:
> 
> 
> Hi folks,
> Happy Friday!
> 
> Would you, please, share your thoughts on the following matter?
> 
> Back some 5 years ago we pulled the trigger and started phasing out Cisco and 
> Juniper switching products out of our data centers (reasons for that are not 
> quite relevant to the topic). We selected Dell switches in part due to Dell 
> using "quick rails'' (sometimes known as speed rails or toolless rails).  
> This is where both the switch side rail and the rack side rail just snap in, 
> thus not requiring a screwdriver and hands of the size no bigger than a 
> hamster paw to hold those stupid proprietary screws (lookin at your, cisco) 
> to attach those rails.
> We went from taking 16hrs to build a row of compute (from just network 
> equipment racking pov) to maybe 1hr... (we estimated that on average it took 
> us 30 min to rack a switch from cut open the box with Juniper switches to 5 
> min with Dell switches)
> Interesting tidbit is that we actually used to manufacture custom rails for 
> our Juniper EX4500 switches so the switch can be actually inserted from the 
> back of the rack (you know, where most of your server ports are...) and not 
> be blocked by the zero-U PDUs and all the cabling in the rack. Stock rails 
> didn't work at all for us unless we used wider racks, which then, in turn, 
> reduced floor capacity.
> 
> As far as I know, Dell is the only switch vendor doing toolless rails so it's 
> a bit of a hardware lock-in from that point of view. 
> 
> So ultimately my question to you all is how much do you care about the speed 
> of racking and unracking equipment and do you tell your suppliers that you 
> care? How much does the time it takes to install or replace a switch impact 
> you?
> 
> I was having a conversation with a vendor and was pushing hard on the fact 
> that their switches will end up being actually costlier for me long term just 
> because my switch replacement time quadruples at least, thus requiring me to 
> staff more remote hands. Am I overthinking this and artificially limiting 
> myself by excluding vendors who don't ship with toolless rails (which is all 
> of them now except Dell)?
> 
> Thanks for your time in advance!
> --Andrey

RIP Dan Kaminsky

[George Herbert]

Reported widely on Twitter by his personal friends, Dan Kaminsky passed away 
yesterday.  The DNS community has lost an immense contributor.


-George 

Sent from my iPhone

Re: OVH datacenter SBG2 in Strasbourg on fire 🔥

[George Herbert]

Sent from my iPhone

> On Mar 10, 2021, at 7:45 AM, Andy Ringsmuth  wrote:
> 
> Sad to see of course, but also a little surprising that fire suppression 
> systems didn’t, well, suppress the fire.
> 
> Unless they didn’t exist?


I am assuming you haven’t had a real datacenter fire before.

I’ve had one fire, seen another, and had an accidental system firing of the gas 
system.  

In the actual fire, caused by cooling system partial failure, there was no gas 
and it turned out the system to disable power on sprinkler discharge failed, 
then the power circuit breakers stayed live despite significant electrical 
short circuiting in the room as sprinklers fired, and then after 5 minutes the 
fire department arrived and sprayed water in for over 10 minutes without 
reducing arcing and fire before building power was successfully disabled and 
they could put the rest of the fire out.  This one could easily have destroyed 
its building, also the building Palo Alto Frys was in.

Fun fact: a rack of burning servers displaces sprinkler water around the rack, 
if it has a top on it, and even if not a vertical stack of burning servers 
pushes water down the front and back and slightly to the sides of burning 
servers, not through the systems themselves.

Fun fact: motherboards burn, as does chip encapsulation epoxy and all the 
wiring, the fan frames, board standoffs, essentially all the RAM and PCI board 
slots, some capacitors and surface mount devices... hard drives melt and the 
aluminum casings burn, SSD plastic casings generally burn.  GBICs and other 
laser diodes smell awful after they catch fire, though it can be hard to tell 
with everything else that smells.

Fun fact: It does not necessarily take many burning servers to put the room 
integrity at risk, even with sprinklers going and the fire department spraying 
water in. 

Fun fact: All the servers that get wet but don’t burn will rust.  And 
everything in the room near the sprinklers that are going off will get wet.  
This is very sad as you watch many millions of dollars of brand new 42U racks 
prestuffed with HP enterprise servers oxidizing away while you wait for the 
garbage truck.

Fun fact: Combustion soot is conductive and even things that didn’t get wet 
probably are dead.

Fun fact: late era Sun Microsystems server boxes were very nice waxed 
cardboard, very well made, apparently fire resistant more than anyone would 
have thought but that were also water resistant enough that you’d have a new in 
box server submerged in a box still full to the brim with water days later.

Fun engineering advice: The window for critical data recovery from hard drives 
that are visibly corroding from water damage short of immersion is probably 
48-72 hours, but run them outside any casing on a fire resistant table and have 
all of CO2 and dry chemical fire extinguishers ready... and a fire hose, and 
handy building fire alarm.  Corroding water damaged SSDs are lower power draw 
and somewhat less likely to start another fire, but take the same precautions.


The fire I saw but that wasn’t mine burned a building with halon and pre action 
sprinklers more or less to the steel columns and roof beams, despite the fire 
department arriving in 5 minutes and trying to actively suppress.  Not enough 
openings for them to safely get to the fire before it was out of control, and 
not enough water flow available in all the sprinklers once it took off.

That one had evidently burned aluminum rack posts by the time it was over, not 
just melted them...


Systems and datacenters aren’t built to eliminate fire risk.  They just aren’t. 
 You can contain or control most office fires with sprinklers, and certainly 
evacuate.  Datacenters with emergency power batteries in the envelope often 
have enough stored energy to set the room on fire despite sprinklers.  If AC 
cutoffs fail and circuit breakers don’t trip mains power will as well.  Too 
many systems and storage and networking hardware components can burn.  


-george

Re: Famous operational issues

[George Herbert]

Northridge quake.  I was #2 and on call at CRL.  That One Guy on dialup in 
Atlanta playing MUDs 23x7 pages that things are down.  I wander out to my 
computer to dial in and see what’s up, turned on TV walking past it, sat down 
and turned computer on, as it was booting on comes a live helicopter shot over 
Northridge showing the 1.5 remaining floors of the 3-story Cable and Wireless 
building our east coast connector went through.

Took a second to listen and make sure I understood what was happening, changed 
channels to verify it wasn’t a stunt, logged  on and pinged our router there to 
confirm nothing there, call & wake up Jim: “East coast’s down because 
earthquake in Northridge and the C&W center fell down.”

“oh.”

And then there was the Sidekick outage...


-George 

Sent from my iPhone

> On Feb 18, 2021, at 4:37 PM, Patrick W. Gilmore  wrote:
> 
> On Feb 18, 2021, at 6:10 PM, Karl Auer  wrote:
>> 
>> I think it was Macchiavelli who said that one should not ascribe to
>> malice anything adequately explained by incompetence…
> 
> https://en.wikipedia.org/wiki/Hanlon%27s_razor
>Never attribute to malice that which is adequately explained by stupidity.
> 
> I personally prefer this version from Robert A. Heinlein:
>Never underestimate the power of human stupidity.
> 
> And to put it on topic, cover your EPOs
> 
> In 1994, there was a major earthquake near the city of Los Angeles. City hall 
> had to be evacuated and it would take over a year to reinforce the building 
> to make it habitable again. My company moved all the systems in the basement 
> of city hall to a new datacenter a mile or so away. After the install, we 
> spent more than a week coaxing their ancient (even for 1994) machines back 
> online, such as a Prime Computer and an AS400 with tons of DASD. Well, tons 
> of cabinets, certainly less storage than my watch has now.
> 
> I was in the DC going over something with the lady in charge when someone 
> walked in to ask her something. She said “just a second”. That person took 
> one step to the side of the door and leaned against the wall - right on an 
> EPO which had no cover.
> 
> Have you ever heard an entire row of DASD spin down instantly? Or taken 40 
> minutes to IPL an AS400? In the middle of the business day? For the second 
> most populous city in the country?
> 
>Me: Maybe you should get a cover for that?
>Her: Good idea.
> 
> Couple weeks later, in the same DC, going over final checklist. A fedex guy 
> walks in. (To this day, no idea how he got in a supposedly locked DC.) She 
> says “just a second”, and I get a very strong deja vu feeling. He takes one 
> step to the side and leans against the wall.
> 
>Me: Did you order that EPO cover?
>Her: Nope.
> 
> -- 
> TTFN,
> patrick
> 

Re: Nice work Ron

[George Herbert]

> On Jan 21, 2021, at 12:59 PM, Eric Kuhnke  wrote:
> 
> > How many other Belize defuncts do they have?  How many offshore countries 
> > like Belize are there in the region?
> 
> Based on my cursory knowledge of offshore corporate registrations in Belize, 
> Panama and the Cayman Islands, identifying those locations which are only 
> mailboxes versus actual business office addresses should not be overly 
> complicated or difficult.
> 
> In the era of Google Street View for most major urban areas the initial 
> search process can be done remotely, such as when it appears that dozens of 
> companies occupy one street address of a very small office building.

That will basically fail in Belize; nobody has run a Google streets camera 
around down there.  I was planning to try to start that last September with 
their volunteer loaner cameras program and a SUV for a couple of weeks but 
there was a pandemic instead of a vacation.

Not even all of the English speaking world...


-George 

Sent from my iPhone

>> 

Re: NDAA passed: Internet and Online Streaming Services Emergency Alert Study

[George Herbert]

I've already had to spike one widely announced WAN UDP protocol that
someone had proposed without thinking through security and DDOS features.
Please don't let's try that trick again.

We have perfectly good approaches that don't involve insecure
untraceable transport layers.  This isn't 1985.  TCP and something SSL
encrypted - HTTPS comes to mind, even if it gets its own port (11911 is
available...).


-george

On Sat, Jan 2, 2021 at 10:02 PM Mark Foster  wrote:

>
> On 3/01/2021 2:41 am, Masataka Ohta wrote:
> > Sean Donelan wrote:
> >
> >> the Commission shall complete an
> >> inquiry to examine the feasibility of updating the Emergency
> >> Alert System to enable or improve alerts to consumers provided
> >> through the internet, including through streaming services.
> >
> > It is trivially easy to have a dedicated UDP port to receive
> > broadcast packets for such purposes, as "through streaming
> > services" is not the requirement.
>
> but "including" is...
>
> And I don't see that opening up a UDP port on every end-user device to
> receive some sort of broadcast (unicast?) is going to be great security.
> Someone will find away to exploit it.
>
>
> >
> > As streaming services are often offered from distant places
> > including foreign locations, generations of emergency alert
> > packets *MUST* be responsibility of *LOCAL* ISPs.
> >
> > A problem is that home routers may filter the broadcast
> > packets from ISPs, but the routers may be upgraded or
> > some device to snoop the alert packets may be placed between
> > ISPs and the routers.
> >
> I think you're overthinking this.
>
> In my mind it's simple.  The streaming companies need to have a channel
> within their streaming system to get a message to a 'currently active
> customer' (emergency popup notification that appears when their app is
> open or their website is active with an authenticated user).  The
> streaming company will also know the location of their customer (billing
> information) so will know what geographic locations are relevant to that
> customer.
>
> Local Authorities can feed emergency broadcast information to the
> streaming companies tagged with a geolocation and the streaming company
> will only rebroadcast it to those customers who are interested in that
> geolocation.
>
> Providing for network-layer alerts of this nature is overcomplicated and
> unnecessary - as was pointed out there are existing means to do this
> (cellphone emergency broadcasts, weather radio service, etc) and the
> intent appears to be to simply add another channel for those who might
> not be able to receive the other. Asking the likes of Netflix to be able
> to channel an brief emergency notifcation across a relevantly-located
> customers streaming service doesn't actually seem that complex, and
> because it's all 'in band' it requires no specific intervention from the
> underlying network operator.
>
> Mark.
>
>

-- 
-george william herbert
george.herb...@gmail.com

Re: {Disarmed} Re: Asus wifi AP re-writing DNS packets

[George Herbert]

This is annoying behavior, because unless you are doing something weird
with actually signing DNS or TCP DNS, the router can just inject a fake
response for their one DNS name they need into any UDP DNS stream with a
tiny bit of inspection.  Hijacking all of DNS is the DUMB way to do it.

And either way you go, it should be configuration flaggable on/off.


On Wed, Nov 4, 2020 at 11:34 AM Tony Wicks  wrote:

> I had a similar discussion with another vendor recently while testing
> their mesh wireless systems. This vendor’s units are actually re-writing
> dhcp requests that clients make to point DNS to the primary mesh unit. This
> even happened when the mesh platform was in pure bridge mode (as opposed to
> router mode). The vendor said this was to make sure their app worked
> reliably. I’d say this sort of behaviour has quietly become common in the
> one app to rule it all world.
>
>
>
>
>
>
>
> *From:* NANOG  *On Behalf Of *Anurag
> Bhatia
> *Sent:* Thursday, 5 November 2020 7:03 am
> *To:* NANOG Mailing List 
> *Subject:* {Disarmed} Re: Asus wifi AP re-writing DNS packets
>
>
>
> Hello
>
>
>
>
>
> An update on this issue:
>
>
>
> Going through (long) Asus support channel, they first agreed that this was
> intentional to make router.asus.com work but did take my request to make
> that optional. They have issued me a test firmware which so far seems to be
> working perfectly with no-rewriting rules. Hoping that it doesn't bring any
> side effects and they eventually put it in their public release after
> testing.
>
>
>
>
>
>
>


-- 
-george william herbert
george.herb...@gmail.com

Re: att or sonic "residential" fiber service at a "nontraditional" residence.

[George Herbert]

Sonic both has their own FTTH and layers on top of ATT FTTH with Fusion IPBB I 
think it’s called.  I don’t know the resale agreement details in place but it’s 
openly advertised as such on Sonic’s site.

Waiting for the true deal to land in my neighborhood ...


-George 

Sent from my iPhone

> On Nov 1, 2020, at 8:55 PM, Matt Corallo  wrote:
> 
> 
> Their site is confusing - they were historically (and still are, in most 
> places) a DSL provider using AT&T for the last hop into the house. Over the 
> past few years they’ve built out their own fiber network which currently has 
> a much smaller footprint. Definitely by far the best residential internet 
> service in the Bay Area, by a mile. They sell both under similar/nearly 
> identical branding.
> 
> Matt
> 
>>> On Nov 1, 2020, at 22:03, Mark Seiden  wrote:
>>> 
>> 
>> 
>>> On Nov 1, 2020, at 5:32 PM, Fletcher Kittredge  wrote:
>>> 
>>> 
>>> Sonic builds their own fiber; they are insurgents. This is a good thing and 
>>> society would be better off with more competition among infrastructure 
>>> providers. It needs to be funded somehow.
>>> 
>>> You can cheat, but if you are a nonprofit doesn't that kinda go against 
>>> mission?
>>> 
>>> 
>> 
>> according to
>> 
>> https://www.sonic.com/residential
>> 
>> for the offering 
>> 
>> “fusion IP Broadband”
>> 
>> 
>> Delivered over AT&T’s IP network using Fiber-to-the-Home or 
>> Fiber-to-the-Node Technology (technology based on location)
>> 
>> This service uses AT&T infrastructure and is installed by an AT&T 
>> technician, you are required to use an AT&T supplied modem. This will be 
>> provided during your installation. 
>> 
>> 
>> 
>>> -- 
>>> Fletcher Kittredge
>>> GWI
>>> 207-602-1134
>>> www.gwi.net
>> 

Re: DHS letters for fuel and facility access

[George Herbert]

The SF Bay Area shelter in place rules specifically exempt news media, 
telecommunications and internet including infrastructure services thereof 
(presumably large internet companies, network and security vendors, etc), fuel 
deliveries.

I could use infrastructure vendors excuse but $current_client_company is on 
mandatory WFH for next five weeks and team had filtered out doing it informally 
before it became official.  

I’d name the company but someone might contact me for an emergency and I have 
nothing to do with the customer incidents team.  I don’t even know who to 
forward stuff to.  Suffice it to say that everyone doing network security infra 
at all the vendors is being as safe as possible under the circumstances.  We’re 
trying to keep all the lights on for you.


-George 

Sent from my iPhone

> On Mar 16, 2020, at 1:21 PM, Sean Donelan  wrote:
> 
> 
> On some other mailing lists, FCC licensed operators are reporting they have 
> received letters from the Department of Homeland Security authorizing 
> "access" and "fuel" priority.
> 
> Occasionally, DHS issues these letters after natural disasters such as 
> hurricanes for hospitals and critical facilities.  I haven't heard of them 
> issued for pandemics.
> 

Re: south bay ops channel

[George Herbert]

Not that I specifically recall since late 90s.  All the local problems
became nationwide.

If you want to start one, sign me up.

On Mon, Nov 18, 2019 at 6:53 PM Randy Bush  wrote:

> > dear lazynet.  is there a list, irc, slack, ... for ops in the
> > southern bay area?  need to find/discuss colo, hands, brains, ...
>
> fwiw, in seattle, the SIX chatter list would be a good example.
>
> randy
>


-- 
-george william herbert
george.herb...@gmail.com

Re: Asset management recommendations

[George Herbert]

Do you really want asset management tools, or configuration management
tools with asset discovery / inventory capability?

Juniper supports Chef configuration management pretty extensively, and is
widely used for systems management and patch management on Linux.  Scales
to multisite well.  There are tie-ins to be able to export monitoring and
alerting tool configurations based on server and network inventories, etc.

https://www.juniper.net/documentation/en_US/junos-chef11.10/topics/concept/chef-overview.html

There are also Puppet, Ansible, and Saltstack in this product space,
slightly less well supported with Juniper as I understand it (haven't
looked extensively, someone else may have better info).

On Fri, Aug 23, 2019 at 9:10 PM Mehmet Akcin  wrote:

> Hey there
>
> I am looking for a tool recommendation for network and server asset
> management which can scale in multiple sites and integrate with other
> platforms like nagios, librenms. Being able to do patch management is plus.
> Mostly linux and juniper shop
>
> Any recommendations?
>
>
> --
> Mehmet
> +1-424-298-1903
>


-- 
-george william herbert
george.herb...@gmail.com

Re: 240/4 (Re: 44/8)

[George Herbert]

Most importantly, if you're running out of 1918 space is a totally
different problem than running out of global routable space.

If you patch common OSes for 240/4 usability but a significant fraction of
say unpatched OSes, IOT, consumer routers, old random net cruft necessary
for infrastructure aren't patched... it's not actually globally routable.
At some point you can write off the few stragglers but... really, get IPv6
everywhere.

On Mon, Jul 22, 2019 at 8:50 PM Owen DeLong  wrote:

>
>
> > On Jul 22, 2019, at 20:14 , Mikael Abrahamsson  wrote:
> >
> > On Mon, 22 Jul 2019, Owen DeLong wrote:
> >
> >>  2.  It was decided that the effort to modify each and every IP
> stack in order to facilitate use of this relatively small block (16 /8s
> being evaluated against a global
> >>  run rate at the time of roughly 2.5 /8s per month, mostly
> to RIPE and APNIC) vs. putting that same effort into modifying each and
> every IP stack to support
> >>  IPv6 was an equation of very small benefit for slightly
> smaller cost. (Less than 8 additional months of IPv4 free pool vs.
> hopefully making IPv6 deployable
> >>  before IPv4 ran out).
> >
> > Well, people are working on making 240/4 usable in IP stacks:
> >
> >
> https://raw.githubusercontent.com/dtaht/unicast-extensions/master/rfcs/draft-gilmore-taht-v4uniext.txt
> >
> > There have been patches accepted into some BSDs and into Linux
> tools/kernel and other operating systems to make 240/4 configurable and
> working as unicast space.
> >
> > I don't expect it to show up in DFZ anytime soon, but some people have
> dilligently been working on removing any obstacles to using 240/4 in most
> common operating systems.
> >
> > For controlled environments, it's probably deployable today with some
> caveats. I think it'd be fine as a compliment to RFC1918 space for some
> internal networks.
> >
> > --
> > Mikael Abrahamssonemail: swm...@swm.pp.se
>
> I guess people can do whatever they want. I personally consider it to be a
> sad sad waste of time that could be better spent deploying IPv6 to more
> places.
>
> Owen
>
>

-- 
-george william herbert
george.herb...@gmail.com

Re: Multi-day GNSS Galileo outage -- Civilization survives

[George Herbert]

Worthwhile noting however that they’re not reliably pushing notifications to 
people on their notifications list.

Worthwhile checking fundamentals you do depend on with your own low level 
monitoring.

-George

Sent from my iPhone

> On Jul 18, 2019, at 10:30 PM, Mikael Abrahamsson  wrote:
> 
>> On Fri, 19 Jul 2019, Sean Donelan wrote:
>> 
>> So much for the disaster scenarioes about a global clamity, planes falling 
>> out the sky, the end of civil society because a global navigation satellite 
>> system fails.  The European Galileo GNSS was down for days, and life went on.
> 
> It wasn't even in full production, and I am not aware of much equipment that 
> solely relies on Galileo.
> 
> A lot of devices today can use multiple GNSS and this is great, as this 
> incident shows that one of them can go offline. Relying on only one of them 
> is risky.
> 
> This outage and its lack of ramifications doesn't imply that if GPS went 
> offline there woulnd't be consequences. Galileo is just a few years old, and 
> wasn't even in production. If GPS would go offline, you'd see a lot different 
> fallout. Lots of things rely on GPS solely.
> 
> -- 
> Mikael Abrahamssonemail: swm...@swm.pp.se

Re: Escalation point at Google

[George Herbert]

If this is re os33.com where Alex emailed from, the front page is Lets
Encrypt.  Which is a strange choice for a financial SAAS?...

Alex, if your internal app site certs are Symantec that could well explain
it; check your cert locations.

On Mon, Nov 12, 2018 at 12:30 PM Guillaume Tournat 
wrote:

> Hello
>
> Problem with blacklisted CA of Symantec, that issued SSL certificates ?
>
>
>
> Le 9 nov. 2018 à 02:57, Alex Osipov  a écrit :
>
> Hello –
>
>
>
> Does anyone have an escalation point or a human to speak to on the Google
> escalations or  Google Safe Browsing team?  Our entire SaaS business, 15
> years in business, in a niche software industry with a good reputation has
> become blocked in ALL browsers.  We are impacting 30k+ enterprise users in
> the financial space and have tried everything but all roads lead to
> automated systems.
>
>
>
> Can anyone please reach out with a contact if you have one?
>
>
>
> Sorry to spam this list if this is inappropriate content.  Very desperate
> here.
>
>
>
> Thank you,
>
> Alex Osipov / CTO
>
>

-- 
-george william herbert
george.herb...@gmail.com

Re: Impacts of Encryption Everywhere (any solution?)

[George Herbert]

I’m confused.

People are using last hop (wireless) arguments against HTTPS Everywhere; that’s 
the part that requires full bandwidth either way (as your non-HTTPS cache is 
upstream somewhere).  The fiber links that are physically fixed and can handle 
in many cases better lasers, are the ongoing upgradable part.

If you’re complaining your fiber backhaul is too big a deal, you’re playing the 
wrong game to start with.


George William Herbert
Sent from my iPhone

> On Jun 19, 2018, at 7:53 AM, Lee Howard  wrote:
> 
> 
> 
>> On 06/17/2018 02:53 PM, Brad wrote:
>> While I agree there are unintended consequences every time advancements are 
>> made in relation to the security and stability of the Internet- I disagree 
>> we should be rejecting their implementations. Instead, we should innovate 
>> further.
> 
> I look forward to your innovations.
>> Just because end to end encryption causes bandwidth issues for a very small 
>> number users - then perhaps they could benefit the most by these changes 
>> with additional capacity.
> 
> I encourage you to invest billions of dollars in rural broadband capacity 
> worldwide. The rest of us will thank you for your sacrifice.
> 
> Lee
> 
>> -Brad
>> 
>>  Original message From: Michael Hallgren  
>> Date: 6/17/18  11:14  (GMT-07:00) To: na...@jack.fr.eu.org Cc: Matthew 
>> Petach , nanog@nanog.org Subject: Re: Impacts of Encryption 
>> Everywhere (any solution?)
>> Le 2018-06-17 12:40, na...@jack.fr.eu.org a écrit :
>>> Well, yes, there is, you simply have to break the end to end encryption
>> Yes, (or) deny service by Policy (remains to evaluate who's happy with
>> that).
>> 
>> Cheers,
>> mh
>> 
 On 06/17/2018 03:09 AM, Matthew Petach wrote:
 Except that if websites are set to HTTPS only, there's no option for
 disabling encryption on the client side.
 
 Matt
 
 
> On Sat, Jun 16, 2018, 14:47  wrote:
> 
>> On 06/16/2018 10:13 PM, Mike Hammett wrote:
>> Sadly, it's just falling on deaf ears. Silicon Valley will continue
>> to
> think they know better than everyone else and people outside of that
> bubble
> will continue to be disadvantaged.
> 
> What, again ?
> Encryption is what is best for the most people.
> The few that will not use it can disable it.
> 
> No issue then.
> 
> 
> 

Re: Craigslist Blocks

[George Herbert]

...Anne's contact is better placed for abuse incidents but if they fail I have 
an alternate contact who has also indirectly helped before.  He's a programmer 
not abuse ops guy but does know the other teams well and has helped.


George William Herbert
Sent from my iPhone

> On Feb 26, 2018, at 10:44 AM, Anne P. Mitchell Esq.  
> wrote:
> 
> If someone wants to send me a copy of the block message, and at least one IP 
> that is blocked, I'll see what we can do.
> 
> Anne
> 
> Anne P. Mitchell, 
> Attorney at Law
> CEO/President, 
> SuretyMail Email Reputation Certification and Inbox Delivery Assistance
> http://www.SuretyMail.com/
> http://www.SuretyMail.eu/
> 
> Attorney at Law / Legislative Consultant
> Author: Section 6 of the CAN-SPAM Act of 2003 (the Federal anti-spam law)
> Author: The Email Deliverability Handbook
> Legal Counsel: The CyberGreen Institute
> Legal Counsel: The Earth Law Center
> Member, California Bar Cyberspace Law Committee
> Former Chair, Asilomar Microcomputer Workshop
> Ret. Professor of Law, Lincoln Law School of San Jose
> 
> 
> 
>> 
>> Same thing here. I've had unresolved issues for months. Tried multiple ways 
>> of contact, including email in block message. No luck.
>> 
>> Joshua Stump
>> Network Admin
>> Fourway.NET
>> 800-733-0062
>> 
>> -Original Message-
>> From: NANOG [mailto:nanog-boun...@nanog.org] On Behalf Of Brett A Mansfield
>> Sent: Monday, February 26, 2018 9:03 AM
>> To: Chris Gross 
>> Cc: NANOG 
>> Subject: Re: Craigslist Blocks
>> 
>> I’ve been having the same problem. I’d also like a contact off list from 
>> someone who can do something about it. 
>> 
>> Thank you,
>> Brett A Mansfield
>> 
>>> On Feb 26, 2018, at 5:54 AM, Chris Gross  wrote:
>>> 
>>> Is there anyone from Craigslist here or anyone have a better way to deal 
>>> with their blocks? There's a contact e-mail in the block messages when 
>>> trying to visit, but there's never gets a response back when we try it. 
>>> Please hit me up off list.
>> 
>> 
>> 
> 
> 

Re: Level 3 issues?

[George Herbert]

Yes; you should subscribe to outa...@outages.org for better reports.  (Short 
summary - yes, no root cause/TTR yet).

George William Herbert
Sent from my iPhone

> On May 16, 2016, at 12:49 PM, David Hubbard  
> wrote:
> 
> Anyone seeing issues with Level 3 networking right now?  We’re seeing huge 
> latency and loss on traffic coming inbound (to us, AS33260) but it seems to 
> be at the peering points with other major ISP’s and Level 3.  Comcast for 
> example:
> 
>  333 ms21 ms70 ms  te-3-5-ur01.hershey.pa.pitt.comcast.net 
> [68.85.42.29]
>  4 *   33 ms   106 ms  162.151.48.173
>  5   214 ms54 ms41 ms  162.151.21.229
>  6   561 ms   764 ms   459 ms  4.68.71.133
> 
> Thanks,
> 
> David

Re: NIST NTP servers

[George Herbert]

> On May 11, 2016, at 6:31 AM, Leo Bicknell  wrote:
> ...
> You're replacing one single point of failure with another.
> 
> Personally, my network gets NTP from 14 stratum 1 sources right now.
> You, and the hacker, do not know which ones.  You have to guess at least
> 8 to get me to move to your "hacked" time.  Good luck.

...except for people who think that N internet only servers is enough 
redundancy.

Pretty much anything with unfiltered outbound could put out enough forged UDP 
to effectively jam ALL the Stratum 1 servers for a given endpoint.


George William Herbert
Sent from my iPhone

Re: Why the US Government has so many data centers

[George Herbert]

The last time I checked, the US CIO office was understaffed and fighting the 
bureaucratic hydra and mostly losing, but competent and doing things like 
providing IGs with relevant ammo.

If not true in this case then the audit should be redone with relevant criteria.

George William Herbert
Sent from my iPhone

> On Mar 22, 2016, at 11:36 AM, Sean Donelan  wrote:
> 
>> On Tue, 22 Mar 2016, George Herbert wrote:
>> Come on, the audit requirements should have diversity/redundancy concerns in 
>> them.
>> 
>> That's standard in all the audits I have done or participated in.
>> 
>> If these ones don't I have a marketing opportunity to teach a HA seminar and 
>> followon consulting to the IG.
> 
> Turn on C-SPAN and watch any random congressional oversight hearing.
> 
> Reasonable, rational or logical thoughts are rare. You may be making 
> assumptions that aren't supported.  Just ask Flint Michigan about saving
> money on cheaper water supplies.
> 

Re: Why the US Government has so many data centers

[George Herbert]

Come on, the audit requirements should have diversity/redundancy concerns in 
them.

That's standard in all the audits I have done or participated in.

If these ones don't I have a marketing opportunity to teach a HA seminar and 
followon consulting to the IG.

George William Herbert
Sent from my iPhone

> On Mar 22, 2016, at 10:59 AM, valdis.kletni...@vt.edu wrote:
> 
> On Tue, 22 Mar 2016 12:11:11 -0400, Sean Donelan said:
>> Why do you have two circuits with only 40% utilization. The auditor says
>> that's waste, and you only need one circuit at 80% utilization for half
>> the cost.
> 
> And of course, said auditor is probably near impervious to the very real
> and valid reasons you have 2 circuits.  Because as Upton Sinclair wrote
> around a century ago:
> 
> "You cannot make a man understand something when his paycheck depends
> on him not understanding it".

Re: Craiglist blocked

[George Herbert]

My guy (who is coder team not ops) confirmed he got the forwarded email and is 
passing it to the right ops folks, but those ops folks will have to reach back 
out again to Chris.

You might try Michael's contacts if you don't hear anything in a few hours at 
most.

George William Herbert
Sent from my iPhone

> On Mar 16, 2016, at 2:29 PM, "Michael J Wise"  wrote:
> 
> 
>> I know someone (not ops but ha can forward internally);  forwarding to
>> him.
> 
> If George's contact doesn't pan out, I have a name that I can forward your
> concern to.
> Ping me at work (address in the Cc:) with details if there's no response?
> 
>> George William Herbert
>> Sent from my iPhone
>> 
>>> On Mar 16, 2016, at 2:18 PM, Christopher Tyler
>>>  wrote:
>>> 
>>> Does anyone have a contact at Craigslist?
>>> Some of our IP addresses got blocked and we are getting no response from
>>> the email address listed when attempting to visit their site. Our
>>> customers are threatening mutiny.
>>> 
>>> --
>>> Christopher Tyler
>>> MTCRE/MTCNA/MTCTCE/MTCWE
>>> Total Highspeed Internet Services
>>> 417.851.1107
> 
> 
> Aloha mai Nai`a.
> -- 
> " So this is how Liberty dies ...  http://kapu.net/~mjwise/
> " To Thunderous Applause.
> 
> 

Re: Craiglist blocked

[George Herbert]

> On Mar 16, 2016, at 2:51 PM, "Michael J Wise"  wrote:
> 
> Let's try that again, once more with feeling.

Put that tablet away
I'm asking you, please, no
It isn't right, it isn't fair!
There were firewalls everywhere
I think that exploit wasn't there...



George William Herbert
Sent from my iPhone

Re: Why the US Government has so many data centers

[George Herbert]

So...

Before I go on, I have not been in Todd's shoes, either serving nor directly 
supporting an org like that.

However, I have indirectly supported orgs like that and consulted at or 
supported literally hundreds of commercial and a few educational and nonprofit 
orgs over the last 30 years. 

There are corner cases where distributed resilience is paramount, including a 
lot of field operations (of all sorts) on ships (and aircraft and spacecraft), 
or places where the net really is unstable.  Any generalizations that wrap 
those legitimate exceptions in are overreaching their valid descriptive range.

That said, the vast bulk of normal world environments, individuals make 
justifications like Todd's and argue for distributed services, private servers, 
etc.  And then do not run them reliably, with patches, backups, central 
security management, asset tracking, redundancy, DR plans, etc.

And then they break, and in some cases are and will forever be lost.  In other 
cases they will "merely" take 2, 5, 10, in one case more than 100 times longer 
to repair and more money to recover than they should have.

Statistically these are very very poor operational practice.  Not so much 
because of location (some) but because of lack of care and quality management 
when they get distributed and lost out of IT's view.

Statistically, several hundred clients in and a hundred or so organizational 
assessments in, if I find servers that matter under desks you have about a 2% 
chance that your IT org can handle supporting and managing them appropriately.

If you think that 98% of servers in a particular category being at high risk of 
unrecoverable or very difficult recovery when problems crop up is acceptable, 
your successor may be hiring me or someone else who consults a lot for a very 
bad day's cleanup.

I have literally been at a billion dollar IT disaster and at tens of smaller 
multimillion dollar ones trying to clean it up.  This is a very sad type of 
work.

I am not nearly as cheap for recoveries as for preventive management and 
proactive fixes. 


George William Herbert
Sent from my iPhone

> On Mar 18, 2016, at 9:28 PM, Todd Crane  wrote:
> 
> I was trying to resist the urge to chime in on this one, but this discussion 
> has continued for much longer than I had anticipated... So here it goes
> 
> I spent 5 years in the Marines (out now) in which one of my MANY duties was 
> to manage these "data centers" (a part of me just died as I used that word to 
> describe these server rooms). I can't get into what exactly I did or with 
> what systems on such a public forum, but I'm pretty sure that most of the 
> servers I managed would be exempted from this paper/policy.
> 
> Anyways, I came across a lot of servers in my time, but I never came across 
> one that I felt should've been located elsewhere. People have brought up the 
> case of personal share drive, but what about the combat camera (think public 
> relations) that has to store large quantities (100s of 1000s) of high 
> resolution photos and retain them for years. Should I remove that COTS 
> (commercial off the shelf) NAS underneath the Boss' desk and put in a data 
> center 4 miles down the road, and force all that traffic down a network that 
> was designed for light to moderate web browsing and email traffic just so I 
> can check a box for some politician's reelection campaign ads on how they 
> made the government "more efficient"
> 
> Better yet, what about the backhoe operator who didn't call before he dug, 
> and cut my line to the datacenter? Now we cannot respond effectively to a 
> natural disaster in the Asian Pacific or a bombing in the Middle East or a 
> platoon that has come under fire and will die if they can't get air support, 
> all because my watch officer can't even login to his machine since I can no 
> longer have a backup domain controller on-site
> 
> These seem very far fetched to most civilian network operators, but to 
> anybody who has maintained military systems, this is a very real scenario. As 
> mentioned, I'm pretty sure my systems would be exempted, but most would not. 
> When these systems are vital to national security and life & death 
> situations, it can become a very real problem. I realize that this policy was 
> intended for more run of the mill scenarios, but the military is almost 
> always grouped in with everyone else anyways. 
> 
> Furthermore, I don't think most people realize the scale of these networks. 
> NMCI, the network that the Navy and Marine Corps used (when I was in), had 
> over 500,000 active users in the AD forest. When you have a network that 
> size, you have to be intentional about every decision, and you should not 
> leave it up to a political appointee who has trouble even checking their 
> email. 
> 
> When you read how about much money the US military hemorrhages, just 
> remember 
> - The multi million dollar storage array combined with a complete network 
> overhaul, and 

Re: Craiglist blocked

[George Herbert]

I know someone (not ops but ha can forward internally);  forwarding to him.

George William Herbert
Sent from my iPhone

> On Mar 16, 2016, at 2:18 PM, Christopher Tyler  
> wrote:
> 
> Does anyone have a contact at Craigslist? 
> Some of our IP addresses got blocked and we are getting no response from the 
> email address listed when attempting to visit their site. Our customers are 
> threatening mutiny.
> 
> -- 
> Christopher Tyler 
> MTCRE/MTCNA/MTCTCE/MTCWE 
> Total Highspeed Internet Services 
> 417.851.1107
> 

Re: Why the US Government has so many data centers

[George Herbert]

> On Mar 14, 2016, at 12:19 PM, George Metz  wrote:
> 
> Based on the "standard" (per the Windows admins) file storage space of 700 
> meg, that sounds like 3TB for user storage. Even if it were 30TB, I still 
> can't see a proper setup costing more than the OC-12 after a period of two 
> years.
> 
> Org is within the Federal Government, so they're not allowed to buy 
> non-top-line anything.

Million-plus dollar NetApps or EMC units are not at all unusual.

This is a terrible pity if a small NAS from Imation/Nexsan would work 
redundantly for $150k or less.

> I agree we should check how much bandwidth is storage, but since there's a 
> snowball's chance in hell of them actually making a change, it's almost 
> certainly not worth the paperwork.

This is the kind of thing whoever runs it needs to know, proves my point, and 
argues against local datacenters where nobody bothers to even collect 
performance metrics much of the time.

George William Herbert
Sent from my iPhone

Re: Why the US Government has so many data centers

[George Herbert]

At enterprise storage costs, that much storage will cost more than the OC-12, 
and then add datacenter and backups.  Total could be 2-3x OC-12 annual costs.

If your org can afford to buy non-top-line storage then it would probably be 
cheaper to go local.

However, you should check how much of the bandwidth is actually storage.  I see 
multimillion dollar projects without basic demand / needs analysis or 
statistics more often than not.


George William Herbert
Sent from my iPhone

> On Mar 14, 2016, at 10:01 AM, George Metz  wrote:
> 
>> On Mon, Mar 14, 2016 at 12:44 PM, Lee  wrote:
>> 
>> 
>> Yes, *sigh*, another what kind of people _do_ we have running the govt
>> story.  Altho, looking on the bright side, it could have been much
>> worse than a final summing up of "With the current closing having been
>> reported to have saved over $2.5 billion it is clear that inroads are
>> being made, but ... one has to wonder exactly how effective the
>> initiative will be at achieving a more effective and efficient use of
>> government monies in providing technology services."
>> 
>> Best Regards,
>> Lee
> 
> That's an inaccurate cost savings though most likely; it probably doesn't
> take into account the impacts of the consolidation on other items. As a
> personal example, we're in the middle of upgrading my site from an OC-3 to
> an OC-12, because we're running routinely at 95+% utilization on the OC-3
> with 4,000+ seats at the site. The reason we're running that high is
> because several years ago, they "consolidated" our file storage, so instead
> of file storage (and, actually, dot1x authentication though that's
> relatively minor) being local, everyone has to hit a datacenter some 500+
> miles away over that OC-3 every time they have to access a file share. And
> since they're supposed to save everything to their personal share drive
> instead of the actual machine they're sitting at, the results are
> predictable.
> 
> So how much is it going to cost for the OC-12 over the OC-3 annually? Is
> that difference higher or lower than the cost to run a couple of storage
> servers on-site? I don't know the math personally, but I do know that if we
> had storage (and RADIUS auth and hell, even a shell server) on site, we
> wouldn't be needing to upgrade to an OC-12.

Re: Why the US Government has so many data centers

[George Herbert]

I really don't care about AWS sales (customer, but not investor or employee).  
But...

If it's not highly loaded, cloud is cheaper.

If it's not in a well run datacenter / machine room, cloud is FAR more reliable.

The cost of blowing up hardware in less than well run machine rooms / 
datacenters can be immense.  At a now defunct cell provider, we lost a badly 
maintained machine room to fire, only about 24 racks, $2.1 million damage.  And 
nearly burned down the Frys Palo Alto building.  And that's just the worst 
catastrophe; had more losses than that in smaller clusters / onsies.

George William Herbert
Sent from my iPhone

> On Mar 13, 2016, at 2:15 PM, Sean Donelan  wrote:
> 
>> On Sun, 13 Mar 2016, Roland Dobbins wrote:
>>> On 13 Mar 2016, at 3:03, George Herbert wrote:
>>> 
>>> It's a symptom of trying to save a few cents at the risk of dollars.
>> 
>> Concur 100%.
>> 
>> Not to mention the related security issues.
> 
> Just remember, no exceptions, no waivers.
> 
> I understand why cloud vendors want 100% of government IT dollars.  But
> requiring all test and development to be done solely in cloud data centers... 
>  there is your 100%
> 

Re: Why the US Government has so many data centers

[George Herbert]

> On Mar 11, 2016, at 11:57 AM, "Mark T. Ganzer"  wrote:
> 
> but I will instead ask this for your consideration:  Do servers in "test, 
> stage, development, or any other environment" really need to have the same 
> environmental, power and connectivity requirements that "production" servers 
> have?


Why would you think otherwise?

It's a symptom of trying to save a few cents at the risk of dollars.

George William Herbert
Sent from my iPhone

Re: AWS Direct Connect - Peering VPCs to Tier 1's and MPLS

[George Herbert]

If you're asking if one can get a provider's router to handle the outside 
physical part of a DC connection... As an ISP service so you don't need your 
own router hardware...

I was working on this for a recent ex client and asked Level 3 exactly that 
question.  I believe I had the right network guy on the phone and it was a firm 
no.

I was going to check all the other Direct Connect providers but client ran out 
of $$.

If anyone does do that, I would like to know and pass it along to ex client for 
their information.


George William Herbert
Sent from my iPhone

> On Mar 1, 2016, at 9:16 AM, "Jay R. Ashworth"  wrote:
> 
> Just got this dropped on my desk an hour ago, and I'm not finding as much
> material online as I might have hoped for...
> 
> It looks like the easiest solution is to just hang a router/firewall at
> Equinix Ashburn and AWS-DC to that, and then peer it to carriers both IP and
> MPLS; is there a "native" way to do that from an AWS VPC instead?
> 
> Any public or private replies cheerfully accepted; will summarize what I
> can to the list.
> 
> Cheers,
> -- jra
> 
> -- 
> Jay R. Ashworth  Baylink   
> j...@baylink.com
> Designer The Things I Think   RFC 2100
> Ashworth & Associates   http://www.bcp38.info  2000 Land Rover DII
> St Petersburg FL USA  BCP38: Ask For It By Name!   +1 727 647 1274

Sonatel?

[George Herbert]

https://bgpstream.com/event/19524

Second Sonatel hijack in last half hour-ish. Anyone on NANOG?...

George William Herbert
Sent from my iPhone

Re: IP-Echelon Compliance

[George Herbert]

You guys aren't devious enough.

These guys are in violation of CAN-SPAM.  To the tune of exceeding the 
statutory maximum $1,000,000 per ISP last *month* for some of you, much less in 
the statute of limitations period.  You could probably point to refusal to 
remove as justifying the triple damages claim.

Everyone on this list just earned your companies $3 million.

Call your attorneys.


George William Herbert
Sent from my iPhone

On Oct 14, 2015, at 5:20 AM, Randy Bush  wrote:

>>> http://www.procmail.org/
>> I wouldn't necessarily recommend that approach.  There is no
>> obligation for victims of spammers to continue providing Internet
>> services to them, including SMTP services.
> 
> computers are cheap.  my time is finite and i value it highly.  what is
> the minimal action i can take to see that idiots do not take my time?
> 
> randy

Re: Cloud backups versus lightning strikes

[George Herbert]

My read on the situation is Yet Another Intermediate Cacheing Fail in storage, 
a well known problem.  Yes, do a pull the power test on your storage so you 
KNOW what's committed...

George William Herbert
Sent from my iPhone

> On Aug 19, 2015, at 5:44 PM, Sean Donelan  wrote:
> 
> 
> As the saying goes, cloud computing is just someone else's computer. Always 
> backup your cloud backups... in your backup.
> 
> Google's spokesperson used the percentage statistic to avoid how
> much data was lost.  Other cloud providers have also lost customer
> data due to various problems.  While a well-run cloud service provider
> is more reliable than keeping data under your mattress (just like a well-run 
> bank is better than keeping cash under your mattress), its
> not magic.
> 
> Nature is still more powerful than even Google.
> 
> 
> http://www.bbc.com/news/technology-33989384
> 
> Google says data has been wiped from discs at one of its data centres in 
> Belgium - after it was struck by lightning four times.
> 
> Some people have permanently lost access to their files as a result.
> 

Re: Cisco Routers Vulnerability

[George Herbert]

A whole pile of new vulnerabilities including remote code exploit were
revealed against specific models about 3 weeks ago; I had not heard of any
exploits, but, ...

Which is why the models and IOS versions would be very useful.

On Mon, Apr 13, 2015 at 2:59 PM, Rashed Alwarrag 
wrote:

> Still I don't have full information from them as it has been reported by
> different customers and all almost in the same time , I am trying to get
> some information about , I was just checking if there is known
> vulnerability has been announced recently regarding this
>
> Thanks you guys
>
>
> On Tuesday, April 14, 2015, Nick Hilliard  wrote:
>
> > On 13/04/2015 23:48, Rashed Alwarrag wrote:
> > > It's reported by different customers in different locations so I don't
> > > think it's password compromised
> >
> > Have you checked?  If the routers had vty access open (ssh or telnet) and
> > the passwords were easy to guess, then it's more likely that this was a
> > password compromise.  You can test this out by getting a copy of one of
> the
> > configs and decrypting the access password.  Or by asking your customers
> > whether their passwords were dictionary or simple words.
> >
> > It's possible that there was a remotely accessible vulnerability, but ios
> > isn't known for this.
> >
> > Nick
> >
> >
> >
>
> --
>
> *Rashed Alwarrag *
>



-- 
-george william herbert
george.herb...@gmail.com

Re: Cisco Nexus

[George Herbert]

> Brandon Ewing  wrote:
> 
>> David Bass wrote:
>> The n2k ToR is not a great design for user or storage interfaces if most of 
>> your traffic is east/west.  It is great as a low cost ilo/drac/choose your 
>> oob port, or if most of your traffic is north/south.  Biggest thing to 
>> remember is that it is not a switch, and has limitations such as not 
>> connecting other switches to it. Like anything else you have to understand 
>> the product so that you don't engineer something that it wasn't designed to 
>> do.
> 
> And remember -- The Nexus 2K performs absolutely ZERO local switching -- all
> frames received from client ports are just copied to the upstream device, so
> it can handle the frame/packet forwarding logic.  
>


What this really does is force you to consider how much of your East-West is 
rack-local, versus off rack.

Rack-local-heavy hurts as badly as off rack, with FEX.

If you want to / can localize E/W tighter than that then you want real TOR 
switching.  If the average E-W is cross rack then the FEX are performance 
equivalent.  For random distributions this comes at a few racks.  For 
intentional distributions it's probably better to TOR switch from day one.


George William Herbert
Sent from my iPhone

Re: Cisco Nexus

[George Herbert]

I wasn't the implementing engineer but I've been at two places that did that, a 
larger game company and a network gear manufacturer in their engineering 
support computational hubs.  I was there during planning and rollout at the 
game company, very early in the Nexus lifespan.

Both sites brought the FEXes back to 5500s; one used a 6-something for core, 
the other a pair of 7ks.

Game company was more east-west, telco eqpt was very heavy east west.

In both cases it's working fine.

George William Herbert
Sent from my iPhone

> On Feb 2, 2015, at 10:17 AM, "Herman, Anthony" 
>  wrote:
> 
> Nanog,
> 
> I would like to poll the collective for experiences both positive and 
> negative with the Nexus line. More specifically I am interested in hearing 
> about FEX with N2K at the ToR and if this has indeed made any impact on Opex 
> as well as non-obvious shortcomings to using the fabric extenders. Also if 
> anyone is using any of the Nexus line for I/O convergence (FCoE) I would be 
> interested in hearing your experience with this as well.
> 
> Thank you in advance,
> 
> -A

Re: gamer "lag" dashboard

[George Herbert]

Cruel, cruel man.

George William Herbert
Sent from my iPhone

> On Jan 19, 2015, at 6:56 PM, Charles N Wyble  wrote:
> 
> SSL is no problem. We just had a whole thread about breaking it. :-) 
> 
> 
>> On January 19, 2015 5:16:43 PM CST, George Herbert 
>>  wrote:
>> Emulating game traffic...  Good luck with that.  You'll probably have to 
>> figure it out and build your own models per service, though a lot is 
>> encapsulated in https.
>> 
>> In terms of showing it to the public, look at Zabbix and Zenoss; both do 
>> dashboards and managing multiple realtime monitoring / performance info 
>> feeds well.
>> 
>> George William Herbert
>> Sent from my iPhone
>> 
>>>  On Jan 19, 2015, at 2:10 PM, Michael O Holstein 
>>>  wrote:
>>>  
>>>  ?Can someone point me in the right direction for something that allows 
>>> creation of a "dashboard" with current and statistical latency to the 
>>> various game servers (PC, Xbox, PS4, etc) ? .. I'm in the education space 
>>> and we get lots of questions/complains about this and would like a way to 
>>> make the stats public.
>>>  
>>>  
>>>  I could roll
>>> something with RRD and Smokeping but with all the packet-shaping crapola 
>>> (including that which we use here) I need something that emulates the 
>>> actual game traffic as would be classified by all the network crap that 
>>> endeavors to mess with it.
>>>  
>>>  
>>>  (not intended to be an argument about QoS and prioritization, responses 
>>> addressing either --or the politics thereof-- really aren't helpful).
>>>  
>>>  
>>>  TIA,
>>>  
>>>  
>>>  Michael Holstein
>>>  
>>>  Network & Data Security
>>>  
>>>  Cleveland State University
>> 
>> !DSPAM:54bd909e175152519182214!
> 
> -- 
> Sent from my Android device with K-9 Mail. Please excuse my brevity.

Re: gamer "lag" dashboard

[George Herbert]

Emulating game traffic...  Good luck with that.  You'll probably have to figure 
it out and build your own models per service, though a lot is encapsulated in 
https.

In terms of showing it to the public, look at Zabbix and Zenoss; both do 
dashboards and managing multiple realtime monitoring / performance info feeds 
well.

George William Herbert
Sent from my iPhone

> On Jan 19, 2015, at 2:10 PM, Michael O Holstein 
>  wrote:
> 
> ?Can someone point me in the right direction for something that allows 
> creation of a "dashboard" with current and statistical latency to the various 
> game servers (PC, Xbox, PS4, etc) ? .. I'm in the education space and we get 
> lots of questions/complains about this and would like a way to make the stats 
> public.
> 
> 
> I could roll something with RRD and Smokeping but with all the packet-shaping 
> crapola (including that which we use here) I need something that emulates the 
> actual game traffic as would be classified by all the network crap that 
> endeavors to mess with it.
> 
> 
> (not intended to be an argument about QoS and prioritization, responses 
> addressing either --or the politics thereof-- really aren't helpful).
> 
> 
> TIA,
> 
> 
> Michael Holstein
> 
> Network & Data Security
> 
> Cleveland State University

Re: Craigslist hacked?

[George Herbert]

> On Nov 24, 2014, at 4:18 PM, Randy Epstein  wrote:
> 
> Actually, he didn’t hack its records either.  He exploited a bug in BIND.


...returned a legit response plus a tacked-on glue record for www.internic.net 
anytime you queried his nameserver, which he tricked people into doing with 
mixtures of sending you mail, hitting open DNS servers with queries for his 
domain, and another thing I still don't want to talk about.


Paul was more widely quoted and knew his BIND vulnerability better; he can 
always out-pedant me on this one.

I did get a few press quotes, though.

Your fu is weak, Randyhopper.  Train harder!   ;-)

George William Herbert
Sent from my iPhone

Re: Craigslist hacked?

[George Herbert]

And that was July 1997 not 96, though that does nothing to make me feel younger 
...

George William Herbert
Sent from my iPhone

> On Nov 24, 2014, at 4:16 PM, George Herbert  wrote:
> 
> 
> He didn't hack the registry, he hijacked its records.  And this is far from 
> the first time a registry account was hacked.  But, yeah, *still* not secure 
> enough.
> 
> 
> George William Herbert
> Sent from my iPhone
> 
>>> On Nov 24, 2014, at 2:17 PM, Randy Epstein  wrote:
>>> 
>>> On 11/24/14, 5:08 PM, "Michael T. Voity"  wrote:
>>> 
>>> I hate to say this, But I think that Network Operators have not see the
>>> last of of this DNS Hijacking. Craigslist might have been a test to see
>>> how far they could get and how long it would take for it to be
>>> discovered.   I hope the FBI and the other Federal agencies out there
>>> are involved with Craigslist to determine how this happened and put in
>>> safeguards in place to help prevent this from happening again.
>>> 
>>> -Mike
>>> 
>>> Michael T. Voity
>>> Network Engineer
>>> University of Vermont
>> 
>> Anyone heard from Eugene Kashpureff lately?
>> 
>> Hello 1996.  :)
>> 
>> 

Re: Craigslist hacked?

[George Herbert]

He didn't hack the registry, he hijacked its records.  And this is far from the 
first time a registry account was hacked.  But, yeah, *still* not secure enough.


George William Herbert
Sent from my iPhone

> On Nov 24, 2014, at 2:17 PM, Randy Epstein  wrote:
> 
>> On 11/24/14, 5:08 PM, "Michael T. Voity"  wrote:
>> 
>> I hate to say this, But I think that Network Operators have not see the
>> last of of this DNS Hijacking. Craigslist might have been a test to see
>> how far they could get and how long it would take for it to be
>> discovered.   I hope the FBI and the other Federal agencies out there
>> are involved with Craigslist to determine how this happened and put in
>> safeguards in place to help prevent this from happening again.
>> 
>> -Mike
>> 
>> Michael T. Voity
>> Network Engineer
>> University of Vermont
> 
> Anyone heard from Eugene Kashpureff lately?
> 
> Hello 1996.  :)
> 
> 

Re: cheap laptop with 32G or 64G recommendations

[George Herbert]

"Nobody will ever need more than 64K...M...G..."

George William Herbert
Sent from my iPhone

> On Nov 10, 2014, at 4:24 PM, Izaac  wrote:
> 
>> On November 10, 2014 4:49:08 PM EST, lobna gouda  
>> wrote:
>> Hello,
>> Any recommendation, not looking for anything fantasy,  my understanding
>> it should be quardcore, with more than DIMM0 slot so each can have 8G.
>> wind7-64bits to work. I want to use it as a server or practice logical
>> routers   
> 
> "Cheap" and "64GiB of RAM" are incompatible concepts in laptops.
> 
> There is no earthly reason you should need to carry a machine like that 
> anyway. If for some reason you need something so equipped, get yourself a 
> cloud instance and connect to it. That's how you save money.
> 
> If you're stuck working in a completely isolated environment, then work it 
> into the contract. That's the cost of being on an island.
> 
> -- 
> Izaac

Re: Linux: concerns over systemd [OT]

[George Herbert]

Ok.  As a highly on- list-topic example of why distrust is called for...

Without referring to the systemd source code*, does anyone know what systemd 
uses to select between networking subsystems (i.e. NetworkManager, the new 
standard as of RHEL 7, vs /etc/ sysconfig/network-scripts/, etc.).  
NetworkManager is default but disableable and it magically falls back to 
network-scripts dir, but the fallback is nearly undocumented and the selection 
behavior appears completely undocumented.

If by some chance you do know this, where did you come by that knowledge?  
Hopefully with URLs.

(* don't bother telling me to read the source.  I'm reading...)

If I cannot find credible documentation of this, as networking person as well 
as enterprise sysadmin, this is a Problem.)


George William Herbert
Sent from my iPhone

Re: Linux: concerns over systemd adoption and Debian's decision to switch

[George Herbert]

> On Oct 22, 2014, at 9:30 AM, Jeffrey Ollie  wrote:
> 
> The people that like systemd (like myself) have wisely learned that
> the people that hate systemd, hate it mostly because it's different
> from what came before and don't want to change.  There's no way to
> argue rationally with that.

I think you are monumentally misreading the situation.

A) Change is the constant in IT. Staying relevant and employable has put me 
through five or more generational shifts in enterprise OS, plus diversions to 
Mach, Plan 9, MacOS, etc.  Change is normal.  

B) Systemd and the Solaris SMF that it conceptually followed have a number of 
technical flaws, ranging from obscure interfaces (sometimes requiring source 
code to understand) to lack of human readable configs to (at least with SMF, 
and as far as I can tell systemd) a lack of ability to even print/dump out the 
current dependencies and ordering tree.

C) In both systemd and SMF a tremendous unpreparedness of training and 
documentation accompanied rollout.  These were not reasonably enterprise ready 
at launch, or now.

D) The architectural case that the services adopted in systemd over time belong 
there or are safe there is not proven, and not that I see well argued or 
documented.  Conglomerated services are at least to be eyed skeptically.

I did not closely follow systemd's development but it is evident from a 
distance that operator feedback in the community and to Sun regarding SMF flaws 
was somehow missed in systemd's development as they did the same wrong things.

A change this big deserves architectural clarity and justification.  We get 
snide comments about change being good and core developers Linus  evidently 
feels are unsafe.


George William Herbert
Sent from my iPhone

Re: Linux: concerns over systemd [OT]

[George Herbert]

> On Oct 21, 2014, at 6:03 PM, Jay Ashworth  wrote:
> 
> GNOME is probably the linchpin.
> 
> But it's not just RH.  It's Debian, and by extension *buntu, and SuSE, and 
> at least one other major independent parent distro that I can't think of
> just now...
> 
> And as far as I know, it's done; SuSE packages already largely don't even
> include initscripts.

Enough to make a grown man fork RHEL (or, CentOS).


George William Herbert
Sent from my iPhone

Re: Major California Faults Ready To Rupture | IFLScience

[George Herbert]

Loma Prieta, very little; the UCSC line was a non-redundant T1 from San Jose 
BARRNET, and the other leaf nodes off that were down.  As I recall the San Jose 
/ SF to LA links were all golden.

Phone service to Santa Cruz was down, then spotty, then up over the course of a 
day, but every line was jammed with people checking in so connect rates sucked. 
 The UCSC point to point T1 had to be manually repaired I think.  The telco 
lines had alternate routes for calls and made it work, in a bit.

Northridge a few years later more or less flattened a C&W center just about at 
ground zero.  CRL's pager-happy 24x7 MUD customer in Atlanta woke me up a 
minute later, and our lines through LA (and many others' lines) were down for a 
while.  Dynamic routing was a little less dynamic then; I don't know what 
others did in great detail.

CIX lists buzzed etc.  I think that predates nanog as a list by a few months, 
but memory is fuzzy.


George William Herbert
Sent from my iPhone

> On Oct 18, 2014, at 3:42 PM, "Bill Woodcock"  wrote:
> 
> Nothing that I recall.  Sean might know better. 
> 
> 
> -Bill
> 
> 
>> On Oct 19, 2014, at 6:19, "Jay Ashworth"  wrote:
>> 
>> How widespread were the effects on backbone communication circuits from 
>> those quakes? 
>> 
>>> On October 18, 2014 3:22:58 PM EDT, Bill Woodcock  wrote:
>>> 
>>>> On Oct 19, 2014, at 2:20 AM, George Herbert  
>>>> wrote:
>>>> 
>>>>  You should restate the "predates"; I was on console on 
>>>> earthquake.berkeley.edu at the time Loma Prieta let go, using among other 
>>>> things (then) Forumnet (now) ICB in a chat, and one of the immediate 
>>>> damage indications was that everyone at UC Santa Cruz dropped offline.
>>> 
>>> …and I was one of those people at UCSC, who had an interesting little 
>>> adventure driving home to Berkeley the next day.
>>> 
>>> Also, there are probably people in Northridge and Napa who might dispute 
>>> your definition of “major,” but yes,a  I take your point.
>>> 
>>> http://en.wikipedia.org/wiki/1994_Northridge_earthquake
>>> 
>>> http://en.wikipedia.org/wiki/2010_Baja_California_earthquake
>>> 
>>> http://en.wikipedia.org/wiki/2014_South_Napa_earthquake
>>> 
>>> -Bill
>> 
>> -- 
>> Sent from my Android phone with K-9 Mail. Please excuse my brevity.

Re: Major California Faults Ready To Rupture | IFLScience

[George Herbert]

You should restate the "predates"; I was on console on earthquake.berkeley.edu 
at the time Loma Prieta let go, using among other things (then) Forumnet (now) 
ICB in a chat, and one of the immediate damage indications was that everyone at 
UC Santa Cruz dropped offline.

Topic important, though, I live near the Hayward Fault now, and all my 
customers and most of their data are in the shake zone.


George William Herbert
Sent from my iPhone

> On Oct 18, 2014, at 9:02 AM, Jay Ashworth  wrote:
> 
> Since the last time we had a really major earthquake in California predates 
> the rise of the Internet, this will be the first time for us. What happens 
> when the fault lets go, folks?
> 
> http://www.iflscience.com/environment/Major-California-Faults-Ready-To-Rupture
> -- 
> Sent from my Android phone with K-9 Mail. Please excuse my brevity.

Re: EFF gets into the CPE router software business..

[George Herbert]

Any idea how well CeroWRT stands up to nation-state level intrusion efforts?


George William Herbert
Sent from my iPhone

> On Jul 24, 2014, at 10:24 AM, char...@thefnf.org wrote:
> 
>> On 2014-07-24 12:04, Valdis Kletnieks wrote:
>> So the EFF is pushing development of an open CPU router
>> https://www.eff.org/deeplinks/2014/07/building-open-wireless-router
>> https://openwireless.org/
>> It's currently targeting WNDR3800's and based on the CeroWRT software
>> (which works pretty well in my own experience).
>> What will possibly be interesting in this forum is that it's explicitly
>> targeting having open guest wireless access (unlike the stuff being pushed
>> by some ISPs, where you can roam but only to other customers of the same
>> ISP).
>> !DSPAM:53d13dc965333732154236!
> 
> The Free Network Foundation (which I co founded and am CTO of) has been 
> helping several groups in the USA do this for ~1 year now. EFF is simply 
> rebranding/respinning community networking, but they are pretty new to the 
> USA Free Networks party overall. They just have a bigger budget/brand 
> recognition (though FreedomTower has become a pretty resilient brand based on 
> the e-mails we get on a daily basis). Also I'm not sure of the level of 
> support/hand holding/documentation etc EFF will provide for folks wanting to 
> build a network off this setup (I'm guessing not much).  Also most incumbent 
> carriers prevent sharing (where FNF 
> supported/assisted/collaborative/affiliated US based efforts back haul (over 
> high capacity wifi or VPN over incumbent circuits) to wholesale colocation 
> facilities POP and do things like monitor abuse@ contacts etc. (Ya know, 
> actually responsibly run an ISP).
> 
> I'd rather of seen them partner with FNF, (or actually much more preferable 
> would be upstream wrt projects like QMP) and not spin YET ANOTHER FIRMWARE.
> 
> I'm glad they picked CeroWRT though.
> 

Re: Inevitable death, was Re: Verizon Public Policy on Netflix

[George Herbert]

> On Jul 17, 2014, at 5:19 AM, Jared Mauch  wrote:
> 
> The problem is partly a technological one.  If you have a fiber span from 
> east<-> west it doesn't make sense to OEO when you can just plop in a bidi 
> amplifier.

Almost certainly, most of the fiber going through the building just hits an 
amplifier (or nothing and isn't broken out there).  Yes.

But they quoted a price for access, and some research turned up signs other 
people are doing big fiber out of that location, so my assumption at this point 
is that at least one pair each direction down the fiber is terminating in some 
router there.  Possibly a fiber level wave device but seems more likely a 
router.

Unless that assumption is not true, this comes down to "We don't want your 
antenna on our roof*, come in via fiber like everyone else" and not having met 
the right Layer 3 reseller yet.  It's not sounding at all like "we have to 
break open a fiber for you and put in a router".

(The rest of this indirectly aimed back at Brett, not Jared )

It's not 1995.  Even little ISPs need to get aware and step their game up.  
Treating transit or uplink like a 1995 problem IS a short road to damnation now.

Seriously.  The net is changing. The customers are changing, the customers uses 
and expectations are changing.  Change with it, or step out of the way.  You 
are not an exception because you're rural. You've just got a density and size 
lag.  That is temporary at best.  Keep up.  This is critical national 
telecommunications infrastructure.  Modern teens have mostly never used 
landline phones and are not OK with inadequate bandwidth at home or on the road.

Being in Laramie is not a shield against change.


* probably expands to "...you aren't big enough for me to bother working with 
my facility staff and filling out the paperwork to get an exception or lease 
amendment or permit and let you put an antenna on our roof, sorry", but this is 
an educated guess not informed.


George William Herbert
Sent from my iPhone

Re: Inevitable death, was Re: Verizon Public Policy on Netflix

[George Herbert]

> On Jul 15, 2014, at 5:02 PM, Brett Glass  wrote:
> 
> At 05:10 PM 7/15/2014, George Herbert wrote:
> 
>> Layer3 runs right through Laramie. With a redundant run slightly south.  
>> What conversations have you had with them?...
> 
> At first, Level3 completely refused us. Then, they quoted us a rate several 
> times higher than either of our existing upstreams for bandwidth. Even at 
> that price, they refused to let us link to them via wireless (requiring us to 
> either buy easements or buy land adjacent to their building, which sits on 
> rented land).
> 
> --Brett Glass
> 

Local fiber provider?  How does everyone else tie in to Layer3 in Laramie?

And, find a Layer3 reseller who can handle the cost problem.  There are a 
bunch.  I can recommend one privately if you can't find one.

Buying retail markups from the vendor who wants to sell wholesale only does not 
scale.


George William Herbert
Sent from my iPhone

Re: Inevitable death, was Re: Verizon Public Policy on Netflix

[George Herbert]

> On Jul 15, 2014, at 8:03 AM, Brett Glass  wrote:
> 
> At 06:49 AM 7/15/2014, Baldur Norddahl wrote:
> 
>> Ah but they are charging you for it. You are paying approximately 40x as
>> much for your bandwidth as you should be (you said you paid 20 USD/Mbps -
>> an outrageous rate). You have a link to a place where you can buy 1 Gbps
>> flatrate for USD 500 per month, so why aren't you?
> 
> Because I'd be charged at least as much per Mbps for raw transport as I
> am paying now. (I look at pricing every quarter to see if I can do
> better. Because I'm rural it has not happened.)
> 
> --Brett Glass
> 

Layer3 runs right through Laramie. With a redundant run slightly south.  What 
conversations have you had with them?...


George William Herbert
Sent from my iPhone

Re: Inevitable death, was Re: Verizon Public Policy on Netflix

[George Herbert]

> On Jul 14, 2014, at 10:41 AM, Matthew Petach  wrote:
> 
> Brett's concerns seem to center around his
> ability to be cost-competitive with the big
> guys in his area...which implies there *are*
> big guys in his area to have to compete with.


He 's running wireless links, from web and prior info as I recall.  His key 
business seems to be outside the cable tv / DSL wire loop ranges from wire 
centers.  The bigger services seem to have fiber into Laramie, and Brett seems 
to have fiber to that Denver exchange pointlet .

Why he's not getting fiber to a bigger exchange point or better transit is 
unclear.

There are bandwidth reseller / BGP / interconnect specialist ISPs out there who 
live to fix these things, if there's anything like a viable customer base...


George William Herbert
Sent from my iPhone

Re: Verizon Public Policy on Netflix

[George Herbert]

> On Jul 14, 2014, at 6:03 AM, Jared Mauch  wrote:
> 
> In my experience the bandwidth is typically the lowest part of the cost 
> equation.
> 
> Why transcode on 1k nodes when you can do it once and distribute it at lower 
> cost,
> including in electricity to run the host CPU.
> 
> Centralized transcoding on dedicated hardware makes sense.
> 
> - Jared

Except perhaps for the (current discussion) small rural ISP.

The bandwidth scaling equations out in Ruralistan have never been the same as 
in large metros.  You see this in wireless delivered performance as well.  
Netflix is probably not the straw that broke the camel's back, but it's The 
Thing Du Jour which one can point at and criticize, so it 's becoming a focal 
point.


George William Herbert
Sent from my iPhone

Re: Verizon Public Policy on Netflix

[George Herbert]

> On Jul 11, 2014, at 10:31 PM, Owen DeLong  wrote:
> 
> 
> On Jul 11, 2014, at 8:18 PM, Randy Bush  wrote:
> 
 And, for the record, it's pretty widely acknowledge that "The World" 
 (Barry Shein) was the world's first commercial ISP - offering shell 
 access in 1989, and at some point started offering PPP dial-up 
 services.  As I recall, they were a UUnet POP.
>>> yep.  and uunet and psi were hallucinations.  can we please not rewrite
>>> well-known history?
>>> or are you equating shell access with isp?  that would be novel.  unix
>>> shell != internet.
>> 
>> btw, not do denigrate what barry did.  a commercial unix bbs connected
>> to the real internet was significant.  the left coasties were doing free
>> stuff, the well, community memory, ...  and barry created a viable bbs
>> commercial service which still survives (i presume).  a significant
>> achievement.
>> 
>> randy
> 
> Not to take away from Barry, but around that same time, some of us left 
> coasts were also helping to build Netcom as a viable commercial entity 
> providing shell and later PPP and dedicated line access (DS0, T1).
> 
> Owen


...and CRL, and shortly after Netcom came Scruznet, and  ...

(Still giggling at how many times CRL got the intersection of 
Market/Geary/Kearny dug up in the early 90s bringing fiber in...).


George William Herbert
Sent from my iPhone

Re: Verizon Public Policy on Netflix

[George Herbert]

> On Jul 11, 2014, at 9:44 AM, Owen DeLong  wrote:
> 
> Would it really be plausible for a small ISP to host caching clusters for
> every streaming content supplier out there?

No, but if you have typical internet user streaming uptake, Netflix and Akamai 
and then...  

Short list, most of the demand.

If you can't handle 2-3 then you have a scale problem.


George William Herbert
Sent from my iPhone

Re: Requirements for IPv6 Firewalls

[George Herbert]

As long as the various stateful firewalls and IDS systems offer hostile
action detection and blocking capabilities that raw webservers lack, there
are certainly counterarguments to the "port filter only" approach being
advocated here.

Focusing only on DDOS prevention from one narrow range of attack vectors
targeting the firewalls themselves is narrowminded.  The security threat
envelope is pretty wide.  Vulnerabilities of similar nature exist on the
webservers themselves, and on load balancer devices you will likely need
anyways.

Any number of enterprises have chosen that if a DDOS or other advanced
attack is going to be successful, to let that be successful in bringing
down a firewall on the external shell of the security envelope rather than
having penetrated to the servers level.

Smart design can also handle transparently failing over should such a
vendor-specific attack succeed.  The idea that anyone doing real, big
complex networks would or has to accept any SPOF is ludicrous.  The
question is, how important is avoiding SPOFs, and how committed you are.
 If the answer is "absolutely must, and we have enough budget to do so"
then it's entirely doable.





On Tue, Apr 22, 2014 at 1:28 PM, Doug Barton  wrote:

> On 04/22/2014 01:15 PM, Matthew Huff wrote:
>
>> I wouldn't manage a corporate network without a centrally managed
>> firewall (stateful; or not).
>>
>
> Matthew,
>
> No one is saying that. What Roland is saying, and the position that I
> agree with, is that putting a firewall in front of a system _that is
> intended to be ON the Internet, serving external users_, is a bad idea.
>
> I think it's a given that you'd want to protect your internal systems with
> a firewall (except for the aforementioned IPv6 illuminati, of whom I am not
> one).
>
> Doug
>
>
>


-- 
-george william herbert
george.herb...@gmail.com

Re: Requirements for IPv6 Firewalls

[George Herbert]

On Mon, Apr 21, 2014 at 9:32 AM, Lee Howard  wrote:
>
> You're describing best practice.  Yes, of course, you should have well
> documented technical and business needs for what's open and what's closed
> in firewalls, and should have traceability from the rules in place to the
> requirements, and be able to walk the rules and understand them and
> reinterpret them from v4 to v6, to a new firewall vendor, etc etc.
>
>
> Yes.  Any publicly-traded company will have this because their auditors
> require it.
> I would think that companies without this documentation are probably not
> ready to deploy a new protocol.
> I concede that tracing the rules to the requirements is a hard one in
> practice (and a PITA in operational practice), but I don't think it's
> required to be able to map IPv4 rules to IPv6 rules.
>
>
You would think that any publicly-traded or sufficiently large or high
profile company would have that because their auditors should require that.
 Yes, that's a reasonable assertion and hope.

I regret to inform the discussion that it's a forlorn hope in a number of
actual real world organizations.

> I'm not making noise to be remembered on the lists as a pissed off
troublemaker.  I've been doing enterprise IT consulting since the early
1990s, and am relaying what the state of reality is, and attempting to get
people at various levels to deal with that rather than assume higher levels
of competence than are really out there...


-- 
-george william herbert
george.herb...@gmail.com

Re: Requirements for IPv6 Firewalls

[George Herbert]

Lee Howard:
>
> So, yeah, you have to give your firewall administrator time to walk
> through the rules and figure out what they ought to be in IPv6.  Your
> firewall administrator has been wanting to clean up the rules for the last
> two years, anyway.



The arrogance in this assertion is amazing.

You're describing best practice.  Yes, of course, you should have well
documented technical and business needs for what's open and what's closed
in firewalls, and should have traceability from the rules in place to the
requirements, and be able to walk the rules and understand them and
reinterpret them from v4 to v6, to a new firewall vendor, etc etc.

Again - THE INERTIA IN REAL ENTERPRISE ENVIRONMENTS SAYS OTHERWISE.

Policymakers baldly asserting that it should be otherwise does not change
reality on the ground in numerous enterprise customers.

You and the others are ascribing to me and William blame for this.  Shoot
the messenger all you want; all we're doing it relaying on why we've failed
to convert all our customers.  It's not because we don't understand
firewalls or v6.  It's because the real world is substantially messier,
often man-decades of work messier than you all assert it could possibly be.

Again - policy community blinders on understanding what real systems are
like out in the world has repeatedly shot the conversion in the legs.  If
you're going to start floating standards for this kind of stuff, then
listen to feedback on why things are failing.




On Fri, Apr 18, 2014 at 3:36 PM, Lee Howard  wrote:

>
>
> On 4/17/14 4:45 PM, "George Herbert"  wrote:
> >
> >> There's a fair argument to be made which says that kind of NAT is
> >> > unhealthy. If its proponents are correct, they'll win that argument
> >> > later on with NAT-incompatible technology that enterprises want. After
> >> > all, enterprise security folk didn't want the Internet in the
> >> > corporate network at all, but having a web browser on every desk is
> >> > just too darn useful. Where they won't win that argument is in the
> >> > stretch of maximum risk for the enterprise security folk.
> >> >
> >> >
> >> Any technology has associated risks, it's a matter of how you
> >> reduce/mitigate them.
> >> This paranoia thingie about IPv6 is getting a bit old.
> >> Just because you don't (seem to) understand how it works, it doesn't
> >>mean
> >> no one else should use it.
> >
> >
> >
> >You are missing the point.
> >
> >Granted, anyone who is IPv6 aware doing a green-field enterprise firewall
> >design today should probably choose another way than NAT.
> >
> >What you are failing is that "redesign firewall rules and approach from
> >scratch along with the IPv6 implementation" usually is not the chosen
> >path,
> >versus "re-implement the same v4 firewall rules and technologies in IPv6
> >for the IPv6 implementation", because all the IPv6 aware net admins are
> >having too much to do dealing with all the other conversion issues, vendor
> >readiness all across the stack, etc.
>
> One of the things we (operator hat) like about IPv6 is that we get to
> clean up the mess we made in IPv4. In many cases we've significantly
> reduced the number of firewall rules or ACL lines, because we don't have
> disaggregate blocks we have to stack up.
>
> On my enterprise firewalls, I had a couple of DMZs, a couple of internal
> networks, and policies for what could get where.  Firewalls referred to
> objects of various kinds, some of which had multiple addresses listed;
> putting servers with similar policies in a single /64 (or a /60 if I
> needed separate VLANs) would have simplified things.  And the policy/rule
> difference between net 10 addresses internally and GUA prefixes internally
> is null.
>
> So, yeah, you have to give your firewall administrator time to walk
> through the rules and figure out what they ought to be in IPv6.  Your
> firewall administrator has been wanting to clean up the rules for the last
> two years, anyway.
>
> Even if the above doesn't apply to you, what rules do you have that you
> can't copy?
> * deny ICMP to any.  Can't do that.  Must allow at least some messages.
> * deny (public address range) to (private address range) unless initiated
> form inside.  Substitute external and internal prefixes.
> * deny (outside) to (DMZ) except (port ranges).  Same in IPv6.
> * deny (inside) to (DMZ) except (port ranges).  Same in IPv6.
>
> As I recall, the rules were in place even when we used NAT.  If "no
> thinking required"

Re: Requirements for IPv6 Firewalls

[George Herbert]

On Fri, Apr 18, 2014 at 10:15 AM, Timothy Morizot wrote:

> On Apr 18, 2014 10:04 AM, "William Herrin"  wrote:
> > That's correct: you don't understand. Until you do, just accept: there
> > are more than a few folks who want to, intend to and will use NAT for
> > IPv6. They will wait until NAT is available in their preferred
> > products before making any significant deployment efforts.
>
> Actually, the few like you will hold off until they are behind the curve,
> then scramble to catch up. Good luck with that strategy!
>


Again.  You're speaking down to William as if he's not IPv6 aware, which is
wrong, and ascribing to him misunderstandings and resistance that he (and
I) are trying to communicate to explain why customers in real life are
lagging so badly.

The reason the IPv6 market penetration is so poor right now is because of
antagonistic attitudes like this when actual implementers in the field try
to feed back what the actual, real objections are that are slowing things
down.  "That shouldn't happen," is not acceptable as a response to an
actual user saying "No, not until I get NAT.".

If William and I fight that fight, lose it, and come back and tell you
"They won't go because insufficient NAT" you need to listen.  I've fought
this in a dozen places and lost 8 of them, not because I don't know v6, but
because the clients have inertia and politics around security posture
changes (and in some cases, PCI compliance regs).


-- 
-george william herbert
george.herb...@gmail.com

Re: Requirements for IPv6 Firewalls

[George Herbert]

On Thu, Apr 17, 2014 at 11:32 AM, Eugeniu Patrascu wrote:

> ...
> It's a bigger risk to think that NAT somehow magically protects you against
> stuff on the Internet.
> Also, if your problem is that someone can screw up firewalls rules, then
> you have bigger issue in your organization than IPv6.



> There's a fair argument to be made which says that kind of NAT is
> > unhealthy. If its proponents are correct, they'll win that argument
> > later on with NAT-incompatible technology that enterprises want. After
> > all, enterprise security folk didn't want the Internet in the
> > corporate network at all, but having a web browser on every desk is
> > just too darn useful. Where they won't win that argument is in the
> > stretch of maximum risk for the enterprise security folk.
> >
> >
> Any technology has associated risks, it's a matter of how you
> reduce/mitigate them.
> This paranoia thingie about IPv6 is getting a bit old.
> Just because you don't (seem to) understand how it works, it doesn't mean
> no one else should use it.



You are missing the point.

Granted, anyone who is IPv6 aware doing a green-field enterprise firewall
design today should probably choose another way than NAT.

What you are failing is that "redesign firewall rules and approach from
scratch along with the IPv6 implementation" usually is not the chosen path,
versus "re-implement the same v4 firewall rules and technologies in IPv6
for the IPv6 implementation", because all the IPv6 aware net admins are
having too much to do dealing with all the other conversion issues, vendor
readiness all across the stack, etc.

Variations on this theme are part of why it's 2014 and IPv6 hasn't already
taken over the world.  The more rabid IPv6 proponents have in fact shot the
transition in the legs repeatedly, and those of us who have been on the
front lines would like you all to please shut up and get out of the way so
we can actually finish effecting v6 deployment and move on to mopping up
things like NAT later.

This is why listening to operators is important.


-- 
-george william herbert
george.herb...@gmail.com

Re: Recommendation on NTP appliances/devices

[George Herbert]

On Thu, Apr 3, 2014 at 8:46 PM, Rob Seastrom  wrote:

>
> Chris Adams  writes:
>
> > Once upon a time, Rob Seastrom  said:
> >> Along the same lines I'm troubled by the lack of divergent sources
> >> these days - everything seems slaved to GPS either directly or
> >> indirectly (might be nice to have stuff out there that got its time
> >> exclusively via Galileo or Glonass).
> >
> > Since you mentioned GLONASS: it had a 10+ hour outage yesterday,
> > apparently due to a bad ephemeris upload.  Did anybody have a
> > GLONASS-using NTP server experience problems?
>
> It would be the height of arrogance to think that this couldn't happen to
> GPS.
>
> I want redundancy.
>


Sadly, right now that either means your own real clock, or WWV.  The
cellphone time is (as far as I know, for the networks I saw data on) all
coming off GPS.

Fortunately real clocks are coming way down in cost.

So the question is, if you want redundancy, what do your failure modes look
like.  Is some low level drift if GPS goes away and stays away for an
extended period OK?  In that case, redundancy probably would be a single
local high grade clock.  Do you want
multi-vendor-common-mode-failure-resistant low drift if GPS goes away?  In
that case, you probably need 3 local clocks.  Possibly 4, if you want to be
able to down one for maintenance and still have 3 operating when the fit
hits the shan, so that if one of the remaining ones drifts you know which
of the 3 is out of whack and to exclude from the "live source".  Just two
operating and you're SOL on figuring out which one is off.

This is why spacecraft and aircraft often have 3 or 4 of each critical
thing; 3 gets you "only fly with all 3 working" and the ability to detect
the bad instrument; 4 lets you fly with one down for maintenance and still
have safe redundant operation, increasing dispatch reliability.


-- 
-george william herbert
george.herb...@gmail.com

Re: misunderstanding scale

[George Herbert]

On Mon, Mar 24, 2014 at 8:02 PM, Owen DeLong  wrote:

>
> On Mar 24, 2014, at 9:21 AM, William Herrin  wrote:
>
> > On Sun, Mar 23, 2014 at 11:07 PM, Naslund, Steve 
> wrote:
> >> I am not sure I agree with the basic premise here.   NAT or Private
> addressing does not equal security.
> >
> > Hi Steve,
> >
> > It is your privilege to believe this and to practice it in the
> > networks you operate.
> >
> > Many of the folks you would have deploy IPv6 do not agree. They take
> > comfort in the mathematical impossibility of addressing an internal
> > host from an outside packet that is not part of an ongoing session.
> > These folks find that address-overloaded NAT provides a valuable
> > additional layer of security.
>
> Which impossibility has been disproven multiple times.
>
> > Some folks WANT to segregate their networks from the Internet via a
> > general-protocol transparent proxy. They've had this capability with
> > IPv4 for 20 years. IPv6 poorly addresses their requirement.
>
> Actually, there are multiple implementations of transparent proxies
> available
> for IPv6. NAT isn't the same thing at all.
>
> If you want to make your life difficult in IPv6, you can. Nobody prevents
> you from
> doing so. It is discouraged and non-sensical, but quite possible at this
> point.
>
> Owen
>
>
>
Right.  fc00::/7 exists.  If you want to emulate your internal use of
10.0.0.0/8 plus NAT (or, proxies or load balancers or whatever) in your
IPv6 implementation go ahead.  Putting in some robust filtering that if the
fc00::/7 ever appears outside the internal gateway the traffic goes poof
should be as easy as the equivalents for 10, 172.16, 192.168 ...


-- 
-george william herbert
george.herb...@gmail.com

Re: L6-20P -> L6-30R

[George Herbert]

Crap, was looking at the non-locking ones.  Ignore that.


On Tue, Mar 18, 2014 at 3:54 PM, George Herbert wrote:

>
> https://www.21cii.com/ITStudio/Content/Resources/Images/Appendix/Plug%20&%20Power/SB%202P-3W_505x447.png
>
> I think the 250 v 15 amp plugs fit in the 20 amp sockets, but the 20s
> don't fit in the 30 sockets.
>
> This sort of thing is usually an adapter, a little cylinder with a L6-20R
> on one end and a L6-30P on the other, since the loads are safe.  Either
> that, or a short jumper cable wired the same way.
>
>
> On Tue, Mar 18, 2014 at 3:46 PM, Mike Hale wrote:
>
>> They're different.  You can't force them.
>>
>> On Tue, Mar 18, 2014 at 12:24 PM, Randy  wrote:
>> > I have a situation where a 208v/20A PDU (L6-20P) is supposedly hooked
>> to a
>> > 208v/30A circuit (L6-30R).   Before I order the correct PDU's and whip
>> > cords...sanity check...are connectors 'similar' enough that this is
>> possible
>> > (with force) or am I going to find we've actually got L6-20R's on the
>> > provider side?
>> >
>> > --
>> > ~Randy
>> >
>>
>>
>>
>> --
>> 09 F9 11 02 9D 74 E3 5B D8 41 56 C5 63 56 88 C0
>>
>>
>
>
> --
> -george william herbert
> george.herb...@gmail.com
>



-- 
-george william herbert
george.herb...@gmail.com

Re: L6-20P -> L6-30R

[George Herbert]

https://www.21cii.com/ITStudio/Content/Resources/Images/Appendix/Plug%20&%20Power/SB%202P-3W_505x447.png

I think the 250 v 15 amp plugs fit in the 20 amp sockets, but the 20s don't
fit in the 30 sockets.

This sort of thing is usually an adapter, a little cylinder with a L6-20R
on one end and a L6-30P on the other, since the loads are safe.  Either
that, or a short jumper cable wired the same way.


On Tue, Mar 18, 2014 at 3:46 PM, Mike Hale wrote:

> They're different.  You can't force them.
>
> On Tue, Mar 18, 2014 at 12:24 PM, Randy  wrote:
> > I have a situation where a 208v/20A PDU (L6-20P) is supposedly hooked to
> a
> > 208v/30A circuit (L6-30R).   Before I order the correct PDU's and whip
> > cords...sanity check...are connectors 'similar' enough that this is
> possible
> > (with force) or am I going to find we've actually got L6-20R's on the
> > provider side?
> >
> > --
> > ~Randy
> >
>
>
>
> --
> 09 F9 11 02 9D 74 E3 5B D8 41 56 C5 63 56 88 C0
>
>


-- 
-george william herbert
george.herb...@gmail.com

Re: random dns queries with random sources

[George Herbert]

Right.  Nonzero chances that you (Joe's site) are the target...

Also, check if you have egress filtering of spoofed addresses below these
DNS resources, between them and any user objects.  You could be sourcing
the spoofing if not...


On Tue, Feb 18, 2014 at 7:44 PM, Dobbins, Roland  wrote:

>
> On Feb 19, 2014, at 10:08 AM, Joe Maimon  wrote:
>
> > What is the purpose of this?
>
> Resource-exhaustion attack against the recursive DNS?
>
> ---
> Roland Dobbins  // 
>
>   Luck is the residue of opportunity and design.
>
>-- John Milton
>
>
>


-- 
-george william herbert
george.herb...@gmail.com

Re: latest Snowden docs show NSA intercepts all Google and Yahoo DC-to-DC traffic

[George Herbert]

On Fri, Nov 1, 2013 at 4:37 PM, Randy Bush  wrote:

> > Anyone familiar with secure organizations
>
> there are such things?
>
> we should be more cautious with absolutes, usually :)
>


Nothing is absolute, but there are certainly "white" organizations which
have no attempt to be secure, and much greyer ones where it's a big deal in
organizational process and ethos.

A Snowden once a decade or so is not a bad record.  Unfortunately, we ...
hoped ... they were the good guys, not the bad guys.


-- 
-george william herbert
george.herb...@gmail.com

Re: latest Snowden docs show NSA intercepts all Google and Yahoo DC-to-DC traffic

[George Herbert]

On Fri, Nov 1, 2013 at 4:01 PM, Masataka Ohta <
mo...@necom830.hpcl.titech.ac.jp> wrote:

> Anthony Junk wrote:
>
> > It seems as if both Yahoo and Google assumed that since they were
> > private circuits that they didn't have to encrypt.
>
> According to Snowden, there are government agents at key
> positions for managing security.
>
> When they declare the private circuits are secure, no one
> else in the companies can argue against.
>
> Unless they are fired and all the backdoors installed by
> them are removed, neither Yahoo and Google are secure.
>

This is probably not entirely true, however...

There is certainly enough in the Snowden docs to render this a valid
question, and there is enough to assume some truth to the statement.

Anyone familiar with secure organizations will realize this as the internal
witch hunt problem.  You now have serious reason to believe that you have
been compromised.  If security needs to be absolute, then the degree of
response needed to succeed at attaining that will require very serious
vetting of all the staff, of the nature of what national security
organizations do (background checks, polygraphs, detailed personal
histories, intrusive random monitoring of employee actions in and outside
the office, etc).

Most of "us" will not put up with that.  However, most of "us" also desire
reasonably secure services (both those of us who work for those services,
and those of us who use them).

The prior default setting was to assume there was nobody trying hard enough
to penetrate those services that the internal witch hunt degree of internal
security was necessary.  It was "reasonable" to hope that someone with
nation-state / superpower level resources was not actively Trying To Get
In.  Now that's not a safe assumption.

The NSA has just put the entire profession in a horrible bind.  By going
beyond the foggy-but-legally-documented FISA warrant activities into active
hostile actions against US providers we have to wonder about what degree of
paranoia is necessary.

Do we now just stick our heads back in the sand?  Identify key security
groups with override authority within our organizations, vet them and
monitor them like the CIA and NSA vet and monitor their employees?  Try to
establish that level of review of all our staffs?

Bruce Schneier has tiptoed around this some, but the thread from his blog
last week of "How do we know we can trust Bruce" is terrifying when we have
to consider applying that question to everyone on this list (and who should
be on this list).


-- 
-george william herbert
george.herb...@gmail.com

Re: latest Snowden docs show NSA intercepts all Google and Yahoo DC-to-DC traffic

[George Herbert]

On Fri, Nov 1, 2013 at 3:26 PM, Niels Bakker  wrote:

> * mi...@stillhq.com (Michael Still) [Fri 01 Nov 2013, 05:27 CET]:
>
>  Its about the CPU cost of the crypto. I was once told the number of CPUs
>> required to do SSL on web search (which I have now forgotten) and it was a
>> bigger number than you'd expect -- certainly hundreds.
>>
>
> False: 
> https://www.imperialviolet.**org/2010/06/25/overclocking-**ssl.html
>
> "On our production frontend machines, SSL/TLS accounts for less than 1% of
> the CPU load, less than 10KB of memory per connection and less than 2% of
> network overhead. Many people believe that SSL takes a lot of CPU time and
> we hope the above numbers (public for the first time) will help to dispel
> that."


That was *front end* SSL/TLS - not internal / back end SSL/TLS.

One could assert that the per-activity SSL/TLS overhead might be the same
for internal services accessed to answer a front-end request, but that's
not necessarily true.  The code/request ratios and external/internal
SSL/TLS startup costs are going to vary wildly from service to service.


-- 
-george william herbert
george.herb...@gmail.com

Re: Sudan disconnected from the Internet

[George Herbert]

http://abcnews.go.com/International/wireStory/sudan-security-clashes-subsidy-protesters-20360418


On Wed, Sep 25, 2013 at 5:34 PM, Tammy Firefly wrote:

> On 9/25/13 18:29:58, Jeff Kell wrote:
> > On 9/25/2013 8:25 PM, Tammy Firefly wrote:
> >> with the old fashioned pair of diagonal cutters applied to fiber?
> >
> > Yes, interesting to know if it was cut fiber, pressure on the inside
> > providers (or their feeds), or pressure on the outside providers.
> >
> > Traceroutes lend any clue?
> >
> > Jeff
> >
>
> If the government did it, I guarantee it was cut fiber.  That makes it
> difficult to quickly restore.  One has to wonder whats going on there
> right now that they dont want the world to know about?
>
>
>
>


-- 
-george william herbert
george.herb...@gmail.com

Re: What to expect after a cooling failure

[George Herbert]

Numbers from memory and filed off a bit for anonymity, but

A site I was consulting with had statistically large numbers of x86 servers 
(say, 3000), SPARC enterprise gear (100), NetApp units (60) and NetApp drives 
(5000+) go through a roughly 42C excursion.  It was much hotter at ceiling 
level but fortunately high (20 foot) ceilings.  Within about 1C of the (wet 
pipes) sprinkler system head fuse temp... (shudder)

Both NetApp and X86 server PSUs had significantly increased failure rates for 
the next year.  Say in rough numbers 10% failed in the year.  About 2% were 
instant fails.

Hard drives had a significantly higher fail rate for the next year, also in the 
10% range.

No change in rate of motherboard or CPU or RAM failures was noted that I recall.


George William Herbert
Sent from my iPhone

On Jul 9, 2013, at 8:28 PM, "Erik Levinson"  wrote:

> As some may know, yesterday 151 Front St suffered a cooling failure after 
> Enwave's facilities were flooded. 
> 
> One of the suites that we're in recovered quickly but the other took much 
> longer and some of our gear shutdown automatically due to overheating. We 
> shut down remotely many redundant and non-essential systems in the hotter 
> suite, and transferred remotely some others to the cooler suite, to ensure 
> that we had a minimum of all core systems running in the hotter suite. We 
> waited until the temperatures returned to normal, and brought everything back 
> online. The entire event lasted from approx 18:45 until 01:15. Apparently 
> ambient temperature was above 43 degrees Celcius at one point on the cool 
> side of cabinets in the hotter suite. 
> 
> For those who have gone through such events in the past, what can one expect 
> in terms of long-term impact...should we expect some premature component 
> failures? Does anyone have any stats to share?
> 
> Thanks
> 
> --
> Erik Levinson
> CTO, Uberflip
> 416-900-3830
> 1183 King Street West, Suite 100
> Toronto ON  M6K 3C5
> www.uberflip.com
> 
> 
> 

Re: Fwd: [Filtering of NTP-access to swisstime.ethz.ch as of July 1st, 2013]

[George Herbert]

On Tue, Jun 25, 2013 at 4:38 PM, Larry Sheldon  wrote:
>
> What is it about people that makes them free-load on services like NTP
> chimes and DNSBLS but refuse to stay in contact with(or at least
> contactable by) the providers when important stuff is pending?
>

Several generations of employees past the ones who made the settings to use
them, and nobody realizes or audits where they are pointed or what they
depend on.


-- 
-george william herbert
george.herb...@gmail.com

Re: This is a coordinated hacking. (Was Re: Need help in flushing DNS)

[George Herbert]

I know how we got here, but perhaps we can take corporate parentage and how big 
.com is now to -discuss?

What happened with the registry data that caused the outage and what can / 
should be done about it / to prevent it happening again still seem to me to be 
operational topics.


George William Herbert
Sent from my iPhone

Re: Need help in flushing DNS

[George Herbert]

The indications and claim are that the root cause was registrar internal
goof, not hostile action against name servers.

The story is not yet detailed enough to add up; getting from point A to
point B requires steps that so far don't really make sense.  A more
detailed explanation is hopefully to be forthcoming...



On Fri, Jun 21, 2013 at 5:22 PM, Glen Kent  wrote:

> Hi,
>
> Do we know which DNS server started leaking the poisoned entry?
>
> Being new to this, i still dont understand how could a hacker gain access
> to the DNS server and corrupt the entry there? Wouldnt it require special
> admin rights, etc. to log in?
>
> Glen
>
>
> On Thu, Jun 20, 2013 at 11:32 AM, Paul Ferguson  >wrote:
>
> > Hanlon's razor? Misconfiguration. Perhaps not done in malice, but I
> > have no idea where the poison leaked in, or why. :-)
> >
> > - ferg
> >
> > On Wed, Jun 19, 2013 at 10:49 PM, Alex Buie 
> > wrote:
> >
> > > Anyone have news/explanation about what's happening/happened?
> > >
> > >
> > > On Wed, Jun 19, 2013 at 10:34 PM, Paul Ferguson <
> fergdawgs...@gmail.com
> > >wrote:
> > >
> > >> Sure enough:
> > >>
> > >>
> > >>
> > >>  ; <<>> DiG 9.7.3 <<>> @localhost yelp.com A
> > >>  ; (1 server found)
> > >>  ;; global options: +cmd
> > >>  ;; Got answer:
> > >>  ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 53267
> > >>  ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0
> > >>
> > >>  ;; QUESTION SECTION:
> > >>  ;yelp.com. IN A
> > >>
> > >>  ;; ANSWER SECTION:
> > >>  yelp.com. 300 IN A 204.11.56.20
> > >>
> > >>  ;; Query time: 143 msec
> > >>  ;; SERVER: 127.0.0.1#53(127.0.0.1)
> > >>  ;; WHEN: Thu Jun 20 07:33:13 2013
> > >>  ;; MSG SIZE  rcvd: 42
> > >>
> > >>
> > >>
> > >>
> > >>
> > >> NetRange: 204.11.56.0 - 204.11.59.255
> > >> CIDR: 204.11.56.0/22
> > >> OriginAS: AS40034
> > >> NetName: CONFLUENCE-NETWORKS--TX3
> > >> NetHandle: NET-204-11-56-0-1
> > >> Parent: NET-204-0-0-0-0
> > >> NetType: Direct Allocation
> > >> Comment: Hosted in Austin TX.
> > >> Comment: Abuse :
> > >> Comment: ab...@confluence-networks.com
> > >> Comment: +1-917-386-6118
> > >> RegDate: 2012-09-24
> > >> Updated: 2012-09-24
> > >> Ref: http://whois.arin.net/rest/net/NET-204-11-56-0-1
> > >>
> > >> OrgName: Confluence Networks Inc
> > >> OrgId: CN
> > >> Address: 3rd Floor, Omar Hodge Building, Wickhams
> > >> Address: Cay I, P.O. Box 362
> > >> City: Road Town
> > >> StateProv: Tortola
> > >> PostalCode: VG1110
> > >> Country: VG
> > >> RegDate: 2011-04-07
> > >> Updated: 2011-07-05
> > >> Ref: http://whois.arin.net/rest/org/CN
> > >>
> > >> OrgAbuseHandle: ABUSE3065-ARIN
> > >> OrgAbuseName: Abuse Admin
> > >> OrgAbusePhone: +1-917-386-6118
> > >> OrgAbuseEmail: ab...@confluence-networks.com
> > >> OrgAbuseRef: http://whois.arin.net/rest/poc/ABUSE3065-ARIN
> > >>
> > >> OrgNOCHandle: NOCAD51-ARIN
> > >> OrgNOCName: NOC Admin
> > >> OrgNOCPhone: +1-415-462-7734
> > >> OrgNOCEmail: n...@confluence-networks.com
> > >> OrgNOCRef: http://whois.arin.net/rest/poc/NOCAD51-ARIN
> > >>
> > >> OrgTechHandle: TECHA29-ARIN
> > >> OrgTechName: Tech Admin
> > >> OrgTechPhone: +1-415-358-0858
> > >> OrgTechEmail: ipad...@confluence-networks.com
> > >> OrgTechRef: http://whois.arin.net/rest/poc/TECHA29-ARIN
> > >>
> > >>
> > >> #
> > >> # ARIN WHOIS data and services are subject to the Terms of Use
> > >> # available at: https://www.arin.net/whois_tou.html
> > >> #
> > >>
> > >> - ferg
> > >>
> > >>
> > >>
> > >> On Wed, Jun 19, 2013 at 10:30 PM, Grant Ridder <
> shortdudey...@gmail.com
> > >
> > >> wrote:
> > >>
> > >> > Yelp is evidently also affected
> > >> >
> > >> > On Wed, Jun 19, 2013 at 10:19 PM, John Levine 
> wrote:
> > >> >
> > >> >> >Reaching out to DNS operators around the globe. Linkedin.com has
> had
> > >> some
> > >> >> issues with DNS
> > >> >> >and would like DNS operators to flush their DNS. If you see
> > >> >> www.linkedin.com resolving NS to
> > >> >> >ns1617.ztomy.com or ns2617.ztomy.com then please flush your DNS.
> > >> >> >
> > >> >> >Any other info please reach out to me off-list.
> > >> >>
> > >> >> While you're at it, www.usps.com, www.fidelity.com, and other well
> > >> >> known sites have had DNS poisoning problems.  When I restarted my
> > >> >> cache, they look OK.
> > >> >>
> > >> >>
> > >> >>
> > >>
> > >>
> > >>
> > >> --
> > >> "Fergie", a.k.a. Paul Ferguson
> > >>  fergdawgster(at)gmail.com
> > >>
> > >>
> >
> >
> >
> > --
> > "Fergie", a.k.a. Paul Ferguson
> >  fergdawgster(at)gmail.com
> >
> >
>



-- 
-george william herbert
george.herb...@gmail.com

Re: This is a coordinated hacking. (Was Re: Need help in flushing DNS)

[George Herbert]

At the DNS Servers or service provider level, one can (and I often do) have 
redundant providers.

At the registrar level?  ...

Not with our current infrastructure, as far as I know how.

The Internet:  Discovering new SPOF since 1969!


George William Herbert
Sent from my iPhone

On Jun 20, 2013, at 3:28 PM, Randy Bush  wrote:

> netsol screwed up.  they screwed up bigtime.  they are shoveling kitty
> litter over it as fast as they can, and they have a professional kitty
> litter, aka pr, department.
> 
> but none of this is surprising.
> 
> and dnssec did not save us.  is there anything which could have?
> 
> randy
> 
> 

Re: This is a coordinated hacking. (Was Re: Need help in flushing DNS)

[George Herbert]

Poisoning a domain's NS records with localhost will most certainly DOS the
domain, yes.

I have not yet seen the source of this; if anyone has a clue where the
updates are coming from please post the info.

Is there anything about ztomy.com that has been seen that's supicious as in
they might be the origin?  This could be them, or could be a joe-job
against them.  I do not want to point a finger lacking any sort of actual
data dump of the poisoning activity...




On Thu, Jun 20, 2013 at 1:02 PM, jamie rishaw  wrote:

> I'm rechecking realtime ns1620/2620 DNS right now and, looking at the
> output, I see an odd number of domains (that have changed) with a listed
> nameserver of "localhost.".
>
> Is this some sort of tactic I'm unaware of?
>
>
> On Thu, Jun 20, 2013 at 2:57 PM, Jared Mauch 
> wrote:
>
> > It seems there may be a need for some sort of 'dns-health' check out
> there
> > that can be done in semi-realtime.
> >
> > I ran a report for someone earlier today on a domain doing an xref
> against
> > open resolver data searching for valid responses vs invalid ones.
> >
> > Is this of value?  Does it need to be automated?
> >
> > - Jared
> >
> > On Jun 20, 2013, at 3:53 PM, jamie rishaw  wrote:
> >
> > > This is most definitely a coordinated and planned attack.
> > >
> > > And by 'attack' I mean hijacking of domain names.
> > >
> > > I show as of this morning nearly fifty thousand domain names that
> appear
> > > suspicious.
> > >
> > > I'm tempted to call uscentcom and/or related agencies (which agencies,
> > who
> > > the hell knows, as ICE seems to have some sort of authority over
> domains
> > > (nearly two hundred fifty of them as I type this in COM alone and
> another
> > > thirty-some in NET).
> > >
> > > Anyone credentialed (credentialed /n/., "I know you or know of you,")
> > > wanting data, e-mail me off-list for some TLD goodness.
> > >
> > >
> > >
> > >
> > >
> > >
> > > On Thu, Jun 20, 2013 at 12:29 PM, Phil Fagan 
> > wrote:
> > >
> > >> Agree'd in these "smaller" scenario's I just wonder if in a larger
> scale
> > >> scenario, whatever that might look like, if its necessary. Whereby
> many
> > >> organizations who provide "services" are effected. Perhaps the result
> > of a
> > >> State led campaign topic for another day.
> > >>
> > >>
> > >>
> > >>
> > >> On Thu, Jun 20, 2013 at 11:25 AM, Paul Ferguson <
> fergdawgs...@gmail.com
> > >>> wrote:
> > >>
> > >>> I am betting that Netsol doesn't need any more "coordination" at the
> > >>> moment -- their phones are probably ringing off-the-hook. There are
> > >>> still ~400 domains still pointing to the ztomy NS:
> > >>>
> > >>>
> > >>> ; <<>> DiG 9.7.3 <<>> @foohost parsonstech.com NS
> > >>> ; (1 server found)
> > >>> ;; global options: +cmd
> > >>> ;; Got answer:
> > >>> ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 49064
> > >>> ;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 0
> > >>>
> > >>> ;; QUESTION SECTION:
> > >>> ;parsonstech.com.INNS
> > >>>
> > >>> ;; ANSWER SECTION:
> > >>> parsonstech.com.172800INNSns2617.ztomy.com.
> > >>> parsonstech.com.172800INNSns1617.ztomy.com.
> > >>>
> > >>> ;; Query time: 286 msec
> > >>> ;; SERVER: 127.0.0.1#53(127.0.0.1)
> > >>> ;; WHEN: Thu Jun 20 19:16:25 2013
> > >>> ;; MSG SIZE  rcvd: 81
> > >>>
> > >>> - ferg
> > >>>
> > >>> On Thu, Jun 20, 2013 at 10:13 AM, Phil Fagan 
> > >> wrote:
> > >>>
> >  I should caveat.coordinate the "recovery" of.
> > 
> > 
> >  On Thu, Jun 20, 2013 at 11:10 AM, Brandon Butterworth
> >  wrote:
> > 
> > >> Is there an organization that coordinates outages like this
> amongst
> > >>> the
> > >> industry?
> > >
> > > No, usually they are surprise outages though Anonymous have tried
> > > coordinating a few
> > >
> > > brandon
> > >
> > 
> > 
> > 
> >  --
> >  Phil Fagan
> >  Denver, CO
> >  970-480-7618
> > >>>
> > >>>
> > >>>
> > >>> --
> > >>> "Fergie", a.k.a. Paul Ferguson
> > >>> fergdawgster(at)gmail.com
> > >>>
> > >>
> > >>
> > >>
> > >> --
> > >> Phil Fagan
> > >> Denver, CO
> > >> 970-480-7618
> > >>
> > >
> > >
> > >
> > > --
> > > Jamie Rishaw // .com.arpa@j <- reverse it. ish.
> > > [Impressive C-level Title Here], arpa / arpa labs
> >
> >
>
>
> --
> Jamie Rishaw // .com.arpa@j <- reverse it. ish.
> [Impressive C-level Title Here], arpa / arpa labs
>



-- 
-george william herbert
george.herb...@gmail.com

Re: If you thought you had wire management issues in your facilities...

[George Herbert]

That's nothing.

I was in a business office colo facility in San Jose in the 2001 timeframe,
that had a (as I recall) 12-rack long patch panel setup for the 2 or 3
floors they occupied.  All the phones and LANs used the same panels.

They'd used red cable for everything.  There was no - zero - cable
management.  There was a literally hand-deep (tip of my fingers to my
wrist) spaghetti mess of wire from side to side, top to bottom, across the
whole set of racks.  Going in every direction.  No cable in the entire room
had a label on either end.

The LAN switches didn't properly handle spanning tree, so if you looped it,
under the tangle of wires the whole room's switches would all start
blinking in unison, which was your sign to unplug what you just plugged in
and figure out what went wrong.

I walked in, examined the situation, went to Frys, purchased green and blue
cables (for phone and net, respectively, did my new switch, gateway, and
phone hookup, labeled both ends of all my cables, and fled.

New owners took over as we were leaving for our permanent office six months
later.  They had a crew in to rewire it.  I walked in and was pulling my
switch and gateway out, and they commented that mine were the only properly
done cables, and profusely thanked us for giving them at least a few ports
they could identify both ends of...



On Wed, Jun 19, 2013 at 10:04 AM, Tom Morris  wrote:

> Radio Free Asia, Washington DC.
>
> https://www.facebook.com/photo.php?fbid=485799631503312&set=gm.536342003094118&type=1
>
> Just remember, you're probably in better shape than them. If you look
> carefully on the right side you can see where some cables were left
> abandoned in place because they'd become unremovable from that giant set of
> dreadlocks.
>
> --
> --
> Tom Morris, KG4CYX
> Mad Scientist For Hire
> Chairman, South Florida Tropical Hamboree / Miami Hamfest
> Engineer, WRGP Radiate FM, Florida International University
> 786-228-7087
> 151.820 Megacycles
>



-- 
-george william herbert
george.herb...@gmail.com

Re: 10gig coast to coast

[George Herbert]

Also, what are reliability and redundancy requirements.

10 gigs of bare naked fiber is one thing, but if you need extra paths
redundancy, figure that out now and specify.

Is this latency, bandwidth, both?  Mission critical, business critical,
less priority?  24x7x365, or subset of that, or intermittent only?


On Mon, Jun 17, 2013 at 6:48 PM, Carlos Alcantar  wrote:

> It's typically that the last mile portion of the circuit is going to cost
> you the most, so it's important to know those details.
>
> Carlos Alcantar
> Race Communications / Race Team Member
> 1325 Howard Ave. #604, Burlingame, CA. 94010
> Phone: +1 415 376 3314 / car...@race.com / http://www.race.com
>
>
>
>
>
> -Original Message-
> From: eric clark 
> Date: Monday, June 17, 2013 3:22 PM
> To: "valdis.kletni...@vt.edu" 
> Cc: "nanog@nanog.org" 
> Subject: Re: 10gig coast to coast
>
> Fair enough
>
> Seattle to Boston is the general route, real close.
>
> On Monday, June 17, 2013, wrote:
>
> > On Mon, 17 Jun 2013 12:51:28 -0700, eric clark said:
> >
> > > I may be needing  10 gig from the West Coast to the East Coast
> >
> > Might want to be more specific.  Catalina Island, CA to Buxton, NC
> > (home of Cape Hatteras High School) will probably be way different
> > than downtown LA to downtown Boston.
> >
>
>
>
>


-- 
-george william herbert
george.herb...@gmail.com

Re: Cat-5 cables near 200 Paul, SF

[George Herbert]

+1 ; go Graybar.


On Fri, May 31, 2013 at 11:49 AM, Majdi S. Abbas  wrote:

> On Fri, May 31, 2013 at 06:25:54PM +, Warren Bailey wrote:
> > We walked up the counter all the time, however that was in Alaska so the
> > rules may be different down here.
>
> You can walk up with a credit card, terms just make it easier
> to place orders in advance for pickup.
>
> Anyway, as noted, from 200P, Graybar is your closest and best
> bet, Central Computer doesn't always have the quantities that people
> on this list sometimes require.
>
> --msa
>
>


-- 
-george william herbert
george.herb...@gmail.com

Re: Data Center Installations

[George Herbert]

Central Computers is ok on no-name server components, but not at all for
rack / cabling / power / management / etc.  Micro Center was right next to
places I go to eat over there, but all gone.

I can almost see Frys off Lawrence/Scott from here, and there's a Graybar 3
miles the other direction.  They no longer welcome me at that Graybar with
my first name, I spent too much time ordering online for delivery and / or
doing datacenters up in SF / the Peninsula, but there were a few years in
the 90s...

BTW, if you're sweating the cost on your cable wrap velcro, you're missing
something.  Your time is more valuable than all the above.



On Wed, May 8, 2013 at 5:29 PM, Jeroen van Aart  wrote:

> On 05/01/2013 10:05 PM, shawn wilson wrote:
>
>> I'm more impressed with MicroCenter than Frys (at least the Frys south if
>> SF).
>>
>
> Too bad the Micro Center in Santa Clara along hwy 101 closed shop a year
> or so ago. According to them the owner of the building raised the lease
> price too much. The closest one for the bay Area now is LA... But I too
> liked them better than frys. It looks like in frys most time I spend
> dodging pushy sales people. You can't look at a thing for more than 10
> seconds before some creepster walks over asking if you need help.
>
> A good alternative for the Bay Area is Central Computers. They even have a
> healthy selection of server hardware, including cases and motherboards:
> http://www.centralcomputers.**com/commerce/catalog/**
> spcategory.jsp?category_id=**1573
>
> Greetings,
> Jeroen
>
> --
> Earthquake Magnitude: 4.4
> Date: Wednesday, May  8, 2013 14:10:48 UTC
> Location: Kuril Islands
> Latitude: 44.1198; Longitude: 147.1659
> Depth: 76.00 km
>
>


-- 
-george william herbert
george.herb...@gmail.com

Re: CenturyLink Outage?

[George Herbert]

Widely discussed on outa...@outages.org list (hint!) but for those not yet list 
members over there, 13 or more states in southeast US affected, reportedly 
routing / layer 3 issue, possibly BGP to outside but not clear.  Some service 
restorations discussed.


George William Herbert
Sent from my iPhone

On May 7, 2013, at 9:54 AM, Jason Lester  wrote:

> Does anyone know what is going on with the nationwide CenturyLink outage?
> Their NOC recording says it is a BGP routing issue with their upstream
> peers affecting Internet traffic and traffic between regions.  Our outside
> connectivity with them has basically been down since about 4:00AM (EDT)
> this morning.  The prefixes we were receiving from them were fluctuating
> between a few hundred and a few thousand all morning.  We are getting the
> full BGP table from them now (for about the last hour), but still not
> seeing any incoming traffic.  Seems like a major issue since it has been
> almost 9 hours now.
> 
> Thanks,
> Jason
> -- 
> 
> Jason Lester
> Administrator for Instructional Technology
> Washington County Public Schools
> Tel: 276-739-3060
> Fax: 276-628-1893
> http://www.wcs.k12.va.us

Re: Data Center Installations

[George Herbert]

Seconded Graybar.  If necessary, in the absence of Graybar or for tiny
stuff, a Frys or Home Depot or Lowes.


On Wed, May 1, 2013 at 12:32 PM, Joe Hamelin  wrote:

> Graybar.
>
> --
> Joe Hamelin, W7COM, Tulalip, WA, 360-474-7474
>
>
> On Wed, May 1, 2013 at 12:23 PM, Warren Bailey <
> wbai...@satelliteintelligencegroup.com> wrote:
>
> > Do any of you have a "go to" resource for materials used in
> installations?
> > Tie wraps, cable management, blahblahblah?
> >
> > I have found several places, but I'm curious to know what the nanog
> > ninja's have to say.
> >
> > //warren
> >
> >
>



-- 
-george william herbert
george.herb...@gmail.com

Re: RFC 1149

[George Herbert]

In europe?  He probably was thinking of a Volvo 245...


On Wed, Apr 3, 2013 at 4:40 AM, Jamie Bowden  wrote:

> > From: Jay Ashworth [mailto:j...@baylink.com]
> > - Original Message -
> > > From: "TJ" 
>
> > > On Tue, Apr 2, 2013 at 3:41 PM, Owen DeLong 
> > wrote:
>
> > > > "Never underestimate the bandwidth of a 747 full of DLT cartridges."
>
> > > XKCD is all over this: http://what-if.xkcd.com/31/
> > > :)
>
> > I have always wondered what kind of station wagon Andy had in mind; the
> > SRT-8 Magnum didn't exist when he said that...
>
> No, but the Caprice Classic wagon was very common at the time.
>
> Jamie
>



-- 
-george william herbert
george.herb...@gmail.com

Re: RFC 1149

[George Herbert]

Packets, shmackets.  I'm just upset that my BGP over Semaphore Towers
routing protocol extension hasn't been experimentally validated yet.

Whoever you are who keeps flying pigeons between my test towers, you can't
deliver packets without proper routing updates!  Knock it off long enough
for me to converge the #@$#$@ routing table...



On Mon, Apr 1, 2013 at 7:19 PM, Jeff Kell  wrote:

> On 4/1/2013 10:15 PM, Eric Adler wrote:
> > Make sure you don't miss the QoS implementation of RFC 2549 (and make
> sure
> > that you're ready to implement RFC 6214).  You'll be highly satisfied
> with
> > the results (presuming you and your packets end up in one of the higher
> > quality classes).
> > I'd also suggest a RFC 2322 compliant DHCP server for devices inside the
> > hurricane zone, but modified by implementing zip ties such that the C47s
> > aren't released under heavy (wind or water) loads.
>
> Actually, given recent events, I'd emphasize and advocate RFC3514
> (http://www.ietf.org/rfc/rfc3514.txt) which I think is LONG overdue for
> adoption.  The implementation would forego most of the currently debated
> topics as related to network abuse or misuse :)
>
> Jeff
>
>
>


-- 
-george william herbert
george.herb...@gmail.com

Re: glass fiber @ 0.997 c

[George Herbert]

On Tue, Mar 26, 2013 at 9:36 AM, Eugen Leitl  wrote:
>
> http://www.newscientist.com/article/dn23309-information-superhighway-approaches-light-speed.html
>
> Information superhighway approaches light speed
>
> 18:00 24 March 2013 by Jacob Aron Nothing moves faster than light in a
> vacuum, but large volumes of data can now travel at 99.7 per cent of this
> ultimate speed limit.

Now I guess we find out exactly how much the various financial firms
are willing to pay to shave 0.3 of C off travel time from London to
NYC...


-- 
-george william herbert
george.herb...@gmail.com

Re: Is multihoming hard? [was: DNS amplification]

[George Herbert]

On Mar 23, 2013, at 7:47 PM, Kyle Creyts  wrote:

> Will they really demand ubiquitous, unabridged connectivity?

Let's back up.  End users do not as a rule* have persistent inbound 
connections.  If they have DSL and a Cable Modem they can switch manually (or 
with a little effort automatically) if one goes down.

* Servers-at-home-or-small-office is the use case for Owen's magic BGP box.  
Which is true for many of us and other core geeks but not an appreciable 
percent of the populace.

I believe that full BGP to end user is less practical for this use case than a 
geographically dispersed BGP external facing intermediary whose connectivity to 
the "end user servers" is full-mesh multi-provider-multi-physical-link VPNs. 

It's a lot easier to manage and has less chance of a config goof blowing up 
bigger network neighbors.

Every time I look at productizing this, though, the market's too small to 
support it.  Which probably means it's way too small for home BGP...


George William Herbert
Sent from my iPhone

Re: Class E addresses in the wild

[George Herbert]

On Thu, Mar 21, 2013 at 5:10 PM, cb.list6  wrote:
> I am pretty sure Class E is completely defunct and not used anywhere
> since Cisco and Juniper routers do not forward the packets (circa 2008
> testing) and no known host accept it as a valid address, AFAIK.

Both the net and host sides of this are trivially repairable problems,
even for crazy cellphone network operators.  As long as you have host
source code and a network vendor you can demand custom patches
from


-- 
-george william herbert
george.herb...@gmail.com

Re: Class E addresses in the wild

[George Herbert]

It is (or was) fairly commonly in use among internal nets which
overflowed RFC 1918 or have to internetwork with other heavy users of
RFC 1918 space.  I know of at least two service providers and one cell
network who were using it for that 3 years ago.

Someone leaking internal routes for such?  Or attempt to hijack the space?

Only the Shadow knows...


On Thu, Mar 21, 2013 at 11:17 AM, Donald Eastlake  wrote:
> No authorized IETF use that I know of. See
> http://www.iana.org/assignments/ipv4-address-space/ipv4-address-space.xml
>
> Thanks,
> Donald
> =
>  Donald E. Eastlake 3rd   +1-508-333-2270 (cell)
>  155 Beaver Street, Milford, MA 01757 USA
>  d3e...@gmail.com
>
>
> On Thu, Mar 21, 2013 at 2:09 PM, Buz Dale  wrote:
>> Is anyone else seeing a lot of Class E address space (240.0.0.0/4) at their
>> borders?  Has this space been reinstated in some as yet unknown to me RFC?
>> Thanks,
>> Buz
>>
>> --
>> Buz Dale
>> buzd...@gmail.com
>> GMT -5
>> --
>>
>>
>>
>> --
>> Buz Dale
>> buzd...@gmail.com
>> GMT -5
>> --
>



-- 
-george william herbert
george.herb...@gmail.com

Re: What Should an Engineer Address when 'Selling' IPv6 to Executives?

[George Herbert]

On Wed, Mar 6, 2013 at 12:30 PM, david raistrick  wrote:
> On Wed, 6 Mar 2013, George Herbert wrote:
>
>> The mindshare shift is happening, but the change won't snowball until
>> IT admins - in bulk - really get it.
>
>
> and keeping in mind that the bulk still don't "get" ipv4, either, (how many
> times a day do I explain to someone what a /xx is, and how you'd fill that
> out for just a single ip addresssigh), the snowball really won't happen
> until it Just Works(tm).  impe and all that.

I had to check something now, but the current client site is first
time my laptop's come up on the client's internal net finding IPv6
addresses in use.

10 clients in the last 4 years, mostly SF Bay Area tech firms of some sort.

This client is a subsidiary of a network hardware vendor.

Your mileage may vary, but ...


-- 
-george william herbert
george.herb...@gmail.com

Re: What Should an Engineer Address when 'Selling' IPv6 to Executives?

[George Herbert]

On Tue, Mar 5, 2013 at 8:20 PM, Owen DeLong  wrote:
>Matthew wrote:
>>[...]
>>>  1.  Decreased application complexity:
>>
>> Yeah. After IPv4 goes entirely away. Which is a long, long, LONG time from 
>> now. Until then…
>>
> I don't think so. I think IPv4's demise as a supported internet protocol is 
> certainly less than 10 years away and likely less than 5. I say this because 
> IPv6 deployment is a bit of a variable here and we're faced with one of two 
> outcomes as a result:

I'm probably biased because I'm a fulltime consultant off in
EndUserLand, but I don't believe this argument for a moment.

I'm sorry, but a lot of organizations' response to IPv6 has been "Ok,
desktops will need an overlay of it for some websites in AP next year,
so we'll do that.  And we need an IPv6 front end visibility for our
website.  But we don't REALLY need to change to using it primarily."
And a fair number are still "What six?".

A very small sliver of end-user networks are truly fully functionally
dual-stacking internally now.

A fair number of IT admins still don't know anything useful about how
to implement it, and are going to pray for translating gateways, and
are having pain and suffering getting their heads around what's needed
for the minimal IPv6 front end for their corporate web presence.

Adoption in big network providers, and in big network services, and in
big telco (both broadband and mobile users) are much further along
than the average "enterprise".

The mindshare shift is happening, but the change won't snowball until
IT admins - in bulk - really get it.


-- 
-george william herbert
george.herb...@gmail.com

Re: What Should an Engineer Address when 'Selling' IPv6 to Executives?

[George Herbert]

On Wed, Mar 6, 2013 at 9:20 AM, Cameron Byrne  wrote:
>
> So, your position, which is substantiated my Microsoft's / Windows
> Phone's / Skype's lack of IPv6 support , is that "nobody cares" until
> we "run out of IPv4".

That is clearly reducto ad absurdum and does not resemble Matthew's
detailed and nuanced argument.  Please try again.


-- 
-george william herbert
george.herb...@gmail.com