subject:"Re\: Common operational misconceptions"

 George Bonser wrote:
 
  It is seemingly working well means there is not much PMTU changes,
  which means we had better assumes some PMTU (1280B, for example) and
  use it without PMTUD.
 
  It depends on the OS and the method being used.  If you set the
 option
  to 2 on Linux, it will do MTU probing constantly and react to MTU
  changes.
 
 It actually does nothing.


Must be magic then, because it works for me.  I've got a few dozen servers with 
MTU 7500 that aren't having a bit of trouble talking to anyone.

Re: Common operational misconceptions

George Bonser wrote:

 Must be magic then, because it works for me.

Yes, but magicians always use tricks.

 I've got a few dozen servers with MTU 7500 that aren't
 having a bit of trouble talking to anyone.

Your trick is that your routers at the border between MTUs
7500 and 1500 (or maybe, 1400 or so) generate ICMP packet
too big packets to your servers and no intermediate
entities filter them, isn't it?

Masataka Ohta

RE: Common operational misconceptions

 
 Your trick is that your routers at the border between MTUs
 7500 and 1500 (or maybe, 1400 or so) generate ICMP packet too big
 packets to your servers and no intermediate entities filter them, isn't
 it?
 
   Masataka Ohta

I am saying that MTU probing works just fine, even with a link in between that 
has a shorter MTU and doesn't pass ICMP.  I actually have one of those. 

It actively probes with packets of varying sizes and learns what the path MTU 
is.  It does not rely on ICMP.  Again, it actively probes with varying sizes of 
packets until it believes it has converged.  There are two modes with linux.  
1: where it only probes if there is a problem and 2: where it probes all the 
time.



   The initial value for search_high SHOULD be the largest possible
   packet that might be supported by the flow.  This may be limited by
   the local interface MTU, by an explicit protocol mechanism such as
   the TCP MSS option, or by an intrinsic limit such as the size of a
   protocol length field.  In addition, the initial value for
   search_high MAY be limited by a configuration option to prevent
   probing above some maximum size.  Search_high is likely to be the
   same as the initial Path MTU as computed by the classical Path MTU
   Discovery algorithm.

   It is RECOMMENDED that search_low be initially set to an MTU size
   that is likely to work over a very wide range of environments.  Given
   today's technologies, a value of 1024 bytes is probably safe enough.
   The initial value for search_low SHOULD be configurable.

...

   The probe may have a size anywhere in the probe size range
   described above.  However, a number of factors affect the selection
   of an appropriate size.  A simple strategy might be to do a binary
   search halving the probe size range with each probe.  However, for
   some protocols, such as TCP, failed probes are more expensive than
   successful ones, since data in a failed probe will need to be
   retransmitted.  For such protocols, a strategy that raises the probe
   size in smaller increments might have lower overhead.  For many
   protocols, both at and above the Packetization Layer, the benefit of
   increasing MTU sizes may follow a step function such that it is not
   advantageous to probe within certain regions at all.

   As an optimization, it may be appropriate to probe at certain common
   or expected MTU sizes, for example, 1500 bytes for standard Ethernet,
   or 1500 bytes minus header sizes for tunnel protocols.

   Some protocols may use other mechanisms to choose the probe sizes.
   For example, protocols that have certain natural data block sizes
   might simply assemble messages from a number of blocks until the
   total size is smaller than search_high, and if possible larger than
   search_low.

   Each Packetization Layer MUST determine when probing has converged,
   that is, when the probe size range is small enough that further
   probing is no longer worth its cost.  When probing has converged, a
   timer SHOULD be set.  When the timer expires, search_high should be
   reset to its initial value (described above) so that probing can
   resume.  Thus, if the path changes, increasing the Path MTU, then the
   flow will eventually take advantage of it.  The value for this timer
   MUST NOT be less than 5 minutes and is recommended to be 10 minutes,
   per RFC 1981.

The timer for Linux is 5 minute by default but you can change it.

Re: Common operational misconceptions

George Bonser wrote:

 I am saying that MTU probing works just fine, even with a
 link in between that has a shorter MTU and doesn't pass
 ICMP.

And I have been saying your statement is unfounded.

 I actually have one of those.

I can't see any.

 It actively probes with packets of varying sizes and learns
 what the path MTU is.

First, it sets eff_pmtu to 1400B. OK?

 It does not rely on ICMP.  Again, it actively probes with
 varying sizes of packets until it believes it has
 converged.

 The initial value for search_high SHOULD be the largest possible
 packet that might be supported by the flow.  This may be limited by
 the local interface MTU, by an explicit protocol mechanism such as
 the TCP MSS option,

OK, so, even with local MTU of 7500B and without ICMP PTB, if
local MTU of your peer is 1500B, TCP MSS makes search_high
1500B.

As eff_pmtu of 1400B is close enough to search_high, you are
done.

Eff_pmtu of 1400B will be used forever with no probe packets
sent.

 The timer for Linux is 5 minute by default but you can change it.

Timer timeouts do not affect TCP MSS.

Masataka Ohta

PS

Your lengthy quotation means you don't see the point.

Re: Common operational misconceptions

2012-02-20 Thread Steven Bellovin

 
 
 The timer for Linux is 5 minute by default but you can change it.
 
 Timer timeouts do not affect TCP MSS.
 

RFC 2923:
  TCP should notice that the connection is timing out.  After
  several timeouts, TCP should attempt to send smaller packets,
  perhaps turning off the DF flag for each packet.  If this
  succeeds, it should continue to turn off PMTUD for the connection
  for some reasonable period of time, after which it should probe
  again to try to determine if the path has changed.

It's Informational, not standards track, but the problem -- and the fix
-- have been known for a very long time.


--Steve Bellovin, https://www.cs.columbia.edu/~smb

Re: Common operational misconceptions

2012-02-20 Thread Jimmy Hess

On Sat, Feb 18, 2012 at 1:19 AM, Bob Vaughan tec...@w6yx.stanford.edu wrote:
 Ethernet/Token Ring/Cisco Console/whatever uses an RJ45 connector
  RJ45 defines a keyed 8P8C type connector, wired in a specific
  manner, for a specific 2 wire telco service. Incompatible with the
  above on several levels.  RJxx == specific connector/wiring pattern
  for specific telco applications. Non-telco uses need not apply.

RJ45 is really an example of what was originally a misconception
became so widespread, so universal, that reality has actually shifted
so the misconception became reality.   When was the last time you ever
heard anyone say 8P8C connector?

Joe public caught on to RJ45,  so now that word means something
different in common usage than what it was specified to be. When
was the last time you heard someone say 8P8C connector in reference to
Ethernet?

Nowadays it is technically ambiguous to say RJ45;  are you talking about
[a] The original standard, Registered Jack 45,  which was a specific
connector together with a specific pinout   (which is not Ethernet
over UTP)? Usage of the connector is exceedingly rare,  and will
hardly ever be referred to.

Or
[b] Ethernet connector; The generic 8P8C connector (which has a
certain resemblance to RJ 45) is specified for use with TIA/EIA 568
compliant cable termination ?


Now instead of [a]  being  correct and [b] being always the
misconception. [b] is correct in common usage.

And you have to decide based on context of the conversation which
defintion of RJ45 is intended,  but  [b] will almost always be the
correct definition.

--
-JH

RE: Common operational misconceptions

 -Original Message-
 From: Masataka Ohta

 First, it sets eff_pmtu to 1400B. OK? 

Where did you get 1400 from?  Are you talking specifically with the linux 
implementation?  

As eff_pmtu of 1400B is close enough to search_high, you are done.

I suppose that depends on a specific implementation of close enough is.  I 
haven't looked at the specific code linux uses to implement this and close 
enough is fairly subjective and can be interpreted in different ways by 
different people.  But even 1400 on, say, a 1420 MSS ICMP black hole is one 
heck of a lot better than running no PMTUD at all and running at something 
under 600 bytes.


 PS
 
 Your lengthy quotation means you don't see the point.

I am wondering where you got this magic 1400 value from.  It should basically 
zero in on a number pretty close to the real path MSS in a few iterations.  
Maybe that one specific implementation stops at the first successful value, but 
that isn't the way the recommendation is written.

Did I say it was perfect?  No, but the notion that PMTUD is broken or 
doesn't work isn't exactly true.  With the old mechanism, such a connection 
would simply hang or force people to turn off PMTUD completely.  The new 
mechanism actually seems to perform rather well in the face of an ICMP black 
hole.

Re: Common operational misconceptions

George Bonser wrote:

 First, it sets eff_pmtu to 1400B. OK?
 
 Where did you get 1400 from?

Read the RFC. PERIOD.

Masataka Ohta

RE: Common operational misconceptions

I, in fact, HAVE read the RFC.  

   The initial value for search_high SHOULD be the largest possible
   packet that might be supported by the flow.  This may be limited by
   the local interface MTU, by an explicit protocol mechanism such as
   the TCP MSS option, or by an intrinsic limit such as the size of a
   protocol length field.

So the initial probe would be the reported MSS of the remote system in this 
case 1460 which would fail.  I believe you might be getting confused by this 
paragraph:

   The general strategy is for the Packetization Layer to find an
   appropriate Path MTU by probing the path with progressively larger
   packets.  If a probe packet is successfully delivered, then the
   effective Path MTU is raised to the probe size.

And believing that as soon as a value is found that passes, the process stops.  
That isn't the case.

   PLPMTUD uses a searching technique to find the Path MTU.  Each
   conclusive probe narrows the MTU search range, either by raising the
   lower limit on a successful probe or lowering the upper limit on a
   failed probe, converging toward the true Path MTU.  For most
   transport layers, the search should be stopped once the range is
   narrow enough that the benefit of a larger effective Path MTU is
   smaller than the search overhead of finding it.

The issue here is that the judgement of once the range is narrow enough that 
the benefit of a larger effective Path MTU is smaller than the search overhead 
of finding it and one might well argue that someone decided that the 
difference between 1400 and 1420 MSS (20 bytes) isn't worth the extra overhead 
of finding it those 20 bytes.  But in the case of a 7500 MTU and a 4500 MTU 
black hole link in between, it certainly does NOT go to 1400 and stay there.



 -Original Message-
 From: Masataka Ohta [mailto:mo...@necom830.hpcl.titech.ac.jp]
 Sent: Monday, February 20, 2012 6:43 PM
 To: George Bonser
 Cc: nanog@nanog.org
 Subject: Re: Common operational misconceptions
 
 George Bonser wrote:
 
  First, it sets eff_pmtu to 1400B. OK?
 
  Where did you get 1400 from?
 
 Read the RFC. PERIOD.
 
   Masataka Ohta

Re: Common operational misconceptions

George Bonser wrote:

 I, in fact, HAVE read the RFC.

You don't, at all.

 The initial value for search_high SHOULD be the largest possible
 packet that might be supported by the flow.  This may be limited by
 the local interface MTU, by an explicit protocol mechanism such as
 the TCP MSS option, or by an intrinsic limit such as the size of a
 protocol length field.

It is a section on search_high, while your question in your
previous mail was on eff_pmtu:

 First, it sets eff_pmtu to 1400B. OK?
 Where did you get 1400 from?

Note that rest of your mail also contains a lot of
misunderstandings on the RFC. But, as you don't read
the RFC, collecting them is a waste of time.

Read the RFC thoroughly again and again, 10 times a day
for 30 days. Then, I may reply your further questions if
asked off list.

PERIOD.

Masataka Ohta

Re: Common operational misconceptions

2012-02-20 Thread Steven Bellovin


On Feb 20, 2012, at 10:27 PM, Masataka Ohta wrote:

 Steven Bellovin wrote:
 
 Timer timeouts do not affect TCP MSS.
 
 RFC 2923:
   TCP should notice that the connection is timing out.  After
   several timeouts, TCP should attempt to send smaller packets,
   perhaps turning off the DF flag for each packet.  If this
   succeeds, it should continue to turn off PMTUD for the connection
   for some reasonable period of time, after which it should probe
   again to try to determine if the path has changed.
 
 So?
 
 It's Informational, not standards track, but the problem
 -- and the fix -- have been known for a very long time.
 
 I'm not sure what, do you think, is the problem, because the
 paragraph of RFC2923 you quote has nothing to do with TCP
 MSS.

Sure it does.  That's in 2.1; the start of it discusses PMTUD
failing for various reasons including firewalls.  
 
 The relevant section of the RFC (relevant to MSS) should be:
 
   The MSS should be determined based on the MTUs of the interfaces on
   the system, as outlined in [RFC1122] and [RFC1191].
 
 which means MSS is constant.

The text I quoted says, in so many words, send smaller packets.
I don't know how it's possible to be more explicit than that.
 
 Note also that the next paragraph (next to the paragraph you
 quote) of the RFC eventually says to use PMTU of 1280B for
 IPv6 if there are black holes. It is not a very good thing
 to do especially for IP over IP tunnels, because 1280B
 packets are always fragmented if they are carried over a
 tunnel with MTU of 1280B.

Please cite in context.  The text I quoted says that one option
is to try turning off DF; the next paragraph notes that you can't
do that on v6.  It also doesn't say to to use PMTU of 1280, it
says that that's a good fallback, and notes that v6 support requires
that.  Although it doesn't say so, I'll note that IP in IP makes the
outer IP effectively a link layer for the inner IP; as such, it has
to preserve all of the relevant properties including a link MTU of
1280.  If that doesn't work -- though it most likely will, since
the most common hardware MTU is from the ancient 1500 byte Ethernet
size -- the outer IP endpoint has to deal with it appropriately,
such as by intentional fragmentation. just as is done for IP over
ATM with its 53-byte cell size (RFC 2225).
 
 As implosion cause by multicast PMTUD of IPv6 requires ICMP
 PTB black holed, you can expect a lot of black holes.
 
   Masataka Ohta
 


--Steve Bellovin, https://www.cs.columbia.edu/~smb

Re: Common operational misconceptions

Steven Bellovin wrote:

 I'm not sure what, do you think, is the problem, because the
 paragraph of RFC2923 you quote has nothing to do with TCP
 MSS.
 
 Sure it does.  That's in 2.1; the start of it discusses PMTUD
 failing for various reasons including firewalls.

Firewalls?

Though I have never assumed existence of firewalls, if you are
saying IPv6 will be even less operational because of firewalls,
I have no counter argument.

 The text I quoted says, in so many words, send smaller packets.
 I don't know how it's possible to be more explicit than that.

No disagreement.

I have been keep saying that IPv6 can't depend on PMTUD and
must send packets of 1280B or less.

It's George Bonser, not me, who said there were other ways.

 Please cite in context.  The text I quoted says that one option
 is to try turning off DF; the next paragraph notes that you can't
 do that on v6.

I thought the context is whether IPv6 is operational or not.

Or, is it whether PMTUDv4 operational or not?

Or, is your problem completely different from the above two?

Your clarification is helpful

Masataka Ohta

Re: Common operational misconceptions

2012-02-19 Thread Owen DeLong


On Feb 18, 2012, at 3:31 AM, Masataka Ohta wrote:

 David Barak wrote:
 
 From: Owen DeLong o...@delong.com
 
 Sigh... NAT is a horrible hack that served us all too well in
  address conservation. Beyond that, it is merely a source of pain.
 
 I understand why you say that - NAT did yeoman's work in address
  conservation. However, it also enabled (yes, really) lots of
  topologies and approaches which are *not* designed upon the
  end-to-end model. Some of these approaches have found their way
  into business proceses.
 
 I'm afraid both of you don't try to understand why NAT was
 harmful to destroy the end to end transparency nor the end
 to end argument presented in the original paper by Saltezer
 et. al:
 
  The function in question can completely and correctly be
  implemented only with the knowledge and help of the application
  standing at the end points of the communication system. Therefore,
  providing that questioned function as a feature of the
  communication system itself is not possible.
 
 While plain NAT, which actively hide itself from end systems,
 which means there can be no knowledge and help of the
 application expected, is very harmful to the end to end
 transparency, it is possible to entirely neutralize the
 harmful effects, by let NAT boxes ask help end systems.
 
 An argument you and others have made many times boils down
  to but if we never had NAT, think how much better it
  would be!
 
 The reality is much better that NAT is not so harmful if NAT
 clients and gateways are designed properly to be able to
 reverse the harmful translations by NAT gateways.
 
 I have running code to make the reverse translations, with
 which protocols such as ftp with PORT commands are working.
 
   Masataka Ohta


No, I think you do not understand...

I have a NAT gateway with a single public address.

I have 15 FTP servers and 22 web servers behind it.

I want people to be able to go to ftp://hostname and/or http://hostname for 
each of them.

Please explain to me how your code solves this problem?

Yeah, thought so.

Owen

Re: Common operational misconceptions

2012-02-19 Thread Joe Greco

  I have running code to make the reverse translations, with
  which protocols such as ftp with PORT commands are working.
 
 No, I think you do not understand...
 
 I have a NAT gateway with a single public address.
 
 I have 15 FTP servers and 22 web servers behind it.
 
 I want people to be able to go to ftp://hostname and/or =
 http://hostname for each of them.

Owen,

Your suggestion here would set many security experts heads on fire.

Whatever will they do when NAT doesn't make such things virtually
impossible?

:-)

... JG
-- 
Joe Greco - sol.net Network Services - Milwaukee, WI - http://www.sol.net
We call it the 'one bite at the apple' rule. Give me one chance [and] then I
won't contact you again. - Direct Marketing Ass'n position on e-mail spam(CNN)
With 24 million small businesses in the US alone, that's way too many apples.

Re: Common operational misconceptions

2012-02-19 Thread Jimmy Hess

On Sun, Feb 19, 2012 at 6:24 PM, Owen DeLong o...@delong.com wrote:
 I have 15 FTP servers and 22 web servers behind it.
 I want people to be able to go to ftp://hostname and/or http://hostname 
 for each of them.

For HTTP;  You put a device on that one IP that will accept each TCP
connection,  await the SNI or Host  header from the client,   and then
make/forward  the connection to a proper server for that hostname.
The public IP address belongs to the FTP/HTTP  service  instead of
belonging to a server.


For FTP,  send to a desired FTP server based on the login username or
otherwise make a SRV record for the  _ftp  service for each hostname,
 and   set aside a TCP port for each FTP service's control connection.

The   ftp://user@hostname   approach  or the
ftp://user@basehostname/hostname/  is  probably more convenient
than ftp://hostname:1234, since many clients do not support SRV
records.

The problem is with the FTP protocol not supporting virtual hosting,
though;  this missing FTP feature is not a NAT problem per se.

The VHOST problem was solved with HTTP a long time ago.
It's just that FTP is a lot less popular / fell into some disuse,  so
the deficiency  (lack of virtual hosting support)   was  never
corrected -- and the protocol hasn't had a single update in a long
time.

So you'll have to have a workaround to do 15 FTP servers with one
global IP,  because your circumstance is so unusual,  that's just
life.

--
-JH

Re: Common operational misconceptions

2012-02-19 Thread Mark Andrews


In message 201202200107.q1k17w5l000...@aurora.sol.net, Joe Greco writes:
   I have running code to make the reverse translations, with
   which protocols such as ftp with PORT commands are working.
  
  No, I think you do not understand...
  
  I have a NAT gateway with a single public address.
  
  I have 15 FTP servers and 22 web servers behind it.
  
  I want people to be able to go to ftp://hostname and/or =
  http://hostname for each of them.
 
 Owen,
 
 Your suggestion here would set many security experts heads on fire.
 
 Whatever will they do when NAT doesn't make such things virtually
 impossible?
 
 :-)

Time to write How to use SRV with FTP.  CGN is going to push
the extension of a whole lot of protocols.

Mark
-- 
Mark Andrews, ISC
1 Seymour St., Dundas Valley, NSW 2117, Australia
PHONE: +61 2 9871 4742 INTERNET: ma...@isc.org

Re: Common operational misconceptions

2012-02-19 Thread Octavio Alvarez


On Wed, 15 Feb 2012 12:47:15 -0800, John Kristoff j...@cymru.com wrote:


I have a handful of common misconceptions that I'd put on a top 10 list,
but I'd like to solicit from this community what it considers to be the
most annoying and common operational misconceptions future operators
often come at you with.


The idea that the more access-points, the better our WiFi.

Oh, and the use of RJ45 for the unconfigured 8P8C, but this is not that
annoying.

Cheers.

Re: Common operational misconceptions

2012-02-19 Thread Karl Auer

On Sun, 2012-02-19 at 19:09 -0600, Jimmy Hess wrote:
 For HTTP;  You put a device on that one IP that will accept each TCP
 connection,  await the SNI or Host  header from the client,   and then
 make/forward  the connection to a proper server for that hostname.

So you need an extra device to work around NAT. Or you have to build
extra smarts into existing devices to work around NAT. There is a
pattern here...
 
 For FTP,  send to a desired FTP server based on the login username or
 otherwise make a SRV record for the  _ftp  service for each hostname,
  and   set aside a TCP port for each FTP service's control connection.

So NAT does indeed prevent the scenario Owen outlined.

It does not make sense to make that the application's fault. If you have
to build NAT-awareness (even indirectly, as in SRV-awareness) into every
application, then you've lost the game and it might be time to realise
that NAT is the problem, not all the applications.

 The problem is with the FTP protocol not supporting virtual hosting,
 though;  this missing FTP feature is not a NAT problem per se.

I'm not sure I agree with that, see above. And while virtual hosting may
be a Good Thing for various other reasons, it seems to me that if it is
required with NAT and is not required without NAT, then it is most
certainly the fault of NAT that it is required.

Regards, K.

-- 
~~~
Karl Auer (ka...@biplane.com.au)
http://www.biplane.com.au/kauer

GPG fingerprint: AE1D 4868 6420 AD9A A698 5251 1699 7B78 4EEE 6017
Old fingerprint: DA41 51B1 1481 16E1 F7E2 B2E9 3007 14ED 5736 F687


signature.asc
Description: This is a digitally signed message part

Re: Common operational misconceptions

2012-02-19 Thread Masataka Ohta

Owen DeLong wrote:

 I have running code to make the reverse translations, with
 which protocols such as ftp with PORT commands are working.

 No, I think you do not understand...

How can't I understand several minor issues with the running code.

 I have 15 FTP servers and 22 web servers behind it.
 I want people to be able to go to ftp://hostname  and/or http://hostname  
 for each of them.
 Please explain to me how your code solves this problem?

See

   draft-ohta-urlsrv-00.txt

   DNS SRV RRs of a domain implicitly specify servers and port numbers
   corresponding to the domain.

   By combining URLs and SRV RRs, no port numbers have to be specified
   explicitly in URLs, even if non-default port numbers are used, which
   makes URLs more concise for port based virtual and real hosting,
   where port based real hosting means that multiple servers sharing an
   IP address are distinguished by port numbers to give service for
   different URLs, which is the case for port forwarded servers behind
   NAT and servers with realm specific IP.

 Yeah, thought so.

The draft has been available since September 29, 2011.

Masataka Ohta

Re: Common operational misconceptions

2012-02-19 Thread Jimmy Hess

On Sun, Feb 19, 2012 at 10:09 PM, Andrew Jones a...@jonesy.com.au wrote:
 On Mon, 20 Feb 2012 11:17:32 +0900, Masataka Ohta
 It seems to me that this will create all sorts of headaches for firewall
 ALGs. Rather than just passing port 21/tcp traffic to the FTP ALG for
 example, the devices would need to inspect traffic on all ports and perform
[snip]

That doesn't work when the FTP control connection is encrypted using SSL.
Layer 4  Firewall devices should not be expecting to intercept FTP
traffic and make decisions based on the application layer contents of
the traffic.

I would suggest a requirement that FTP clients utilizing SRV records
to access FTP on an alternate port MUST utilize Firewall-Friendly FTP
as described by RFC1579.

Each FTP server can then be assigned its own port range, or the FTP
server can be configured to notify the Firewall device which ports to
forward using UpNP or a NAT traversal protocol such as STUN, and the
Firewall device can be configured  to forward the appropriate range of
ports  to the correct server.

--
-JH

Re: Common operational misconceptions

2012-02-19 Thread Masataka Ohta

George Bonser wrote:

 It is seemingly working well means there is not much PMTU changes,
 which means we had better assumes some PMTU (1280B, for example) and
 use it without PMTUD.

 It depends on the OS and the method being used.  If you set the
 option to 2 on Linux, it will do MTU probing constantly and
 react to MTU changes.

It actually does nothing.

Given the following statements in the RFC:

   An initial eff_pmtu of 1400 bytes might
   be a good compromise because it would be safe for nearly all tunnels
   over all common networking gear, and yet close to the optimal MTU for
   the majority of paths in the Internet today.

and

   Each Packetization Layer MUST determine when probing has converged,
   that is, when the probe size range is small enough that further
   probing is no longer worth its cost.  When probing has converged, a

the hosts are keep assuming PMTU of 1400B and if local MTU
is 1500B or less, no discovery is performed because the
probe size range is small enough.

 Also, the MTU for a given path only lives for 5 minutes anyway
 (by default) and is rediscovered with Linux.   (value in
 /proc/sys/net/ipv4/route/mtu_expires) but other operating
  systems may behave in other ways.

See above. Rediscovery with initial eff_pmtu of 1400B and
search_high of 1500B immediately terminates without any
probe packets sent.

Masataka Ohta

Re: Common operational misconceptions

2012-02-18 Thread Michael Painter


Paul Graydon wrote:

Give me someone who can already think and analyse over someone who
'knows' it all, any day.  You can be qualified to the hilt but
absolutely useless in the real world (I've watched CCNP and higher
struggling to figure out why they can't ping a 10.0.0.0/24 address at a
customers remote site, not even realising it's a private range, let
alone trying to trace the path of the ping,)


Hard to believe, but you're obviously serious.  What are their job titles?  
What were they hired to accomplish?
Also hard for me to understand that someone could study for CCNx and not get exposed to Private space and 1918...what am I 
missing?


--Michael

Re: Common operational misconceptions

2012-02-18 Thread Jeroen van Aart


Michael Sinatra wrote:

The words Internet and Web can be used interchangeably


I prefer the term intergophers myself.

--
Earthquake Magnitude: 4.9
Date: Friday, February 17, 2012 14:28:20 UTC
Location: Komandorskiye Ostrova, Russia region
Latitude: 54.5969; Longitude: 168.8863
Depth: 34.70 km

Re: Common operational misconceptions

2012-02-18 Thread Masataka Ohta


David Barak wrote:


From: Owen DeLong o...@delong.com



Sigh... NAT is a horrible hack that served us all too well in

 address conservation. Beyond that, it is merely a source of pain.


I understand why you say that - NAT did yeoman's work in address

 conservation. However, it also enabled (yes, really) lots of
 topologies and approaches which are *not* designed upon the
 end-to-end model. Some of these approaches have found their way
 into business proceses.

I'm afraid both of you don't try to understand why NAT was
harmful to destroy the end to end transparency nor the end
to end argument presented in the original paper by Saltezer
et. al:

  The function in question can completely and correctly be
  implemented only with the knowledge and help of the application
  standing at the end points of the communication system. Therefore,
  providing that questioned function as a feature of the
  communication system itself is not possible.

While plain NAT, which actively hide itself from end systems,
which means there can be no knowledge and help of the
application expected, is very harmful to the end to end
transparency, it is possible to entirely neutralize the
harmful effects, by let NAT boxes ask help end systems.


An argument you and others have made many times boils down

 to but if we never had NAT, think how much better it
 would be!

The reality is much better that NAT is not so harmful if NAT
clients and gateways are designed properly to be able to
reverse the harmful translations by NAT gateways.

I have running code to make the reverse translations, with
which protocols such as ftp with PORT commands are working.

Masataka Ohta

Re: Common operational misconceptions

2012-02-18 Thread Paul Graydon


On 2/17/2012 10:55 PM, Michael Painter wrote:

Paul Graydon wrote:

Give me someone who can already think and analyse over someone who
'knows' it all, any day.  You can be qualified to the hilt but
absolutely useless in the real world (I've watched CCNP and higher
struggling to figure out why they can't ping a 10.0.0.0/24 address at a
customers remote site, not even realising it's a private range, let
alone trying to trace the path of the ping,)


Hard to believe, but you're obviously serious.  What are their job 
titles?  What were they hired to accomplish?
Also hard for me to understand that someone could study for CCNx and 
not get exposed to Private space and 1918...what am I missing?


Yes I'm serious, they were CCNP qualified, hired as a NOC engineer for 
an ISP  Hosting company.  For the company the NOC team was the top tier 
of customer support (3rd line+), they looked after routers, switches, 
firewalls, servers, leased lines, and so on.
This individual was perfectly capable of regurgitating all the facts, 
figures and technical details you can imagine, probably pretty much the 
entire CCNP syllabus.  What they didn't seem that capable of was 
actually applying that to anything.  I'd bet good money that if I'd 
asked him at the time what the 1918 network ranges are he'd have been 
able to tell me.
This is exactly what we're teaching kids to do these days (makes me feel 
so old that I've already been saying this for several years and I'm only 
31) standardised tests aren't marked based on ability to apply 
knowledge, just the knowledge itself.  Hence my view, give me someone 
who knows how to think over someone who is qualified to the hilt.  These 
exam cram 'do a CCNP in a week' courses only serve to make it worse.


Paul

RE: Common operational misconceptions

2012-02-18 Thread George Bonser

 Yes I'm serious, they were CCNP qualified, hired as a NOC engineer for
 an ISP  Hosting company.  

There was a time a new hire with all the right holes punched in his ticket 
deleted an item in an access-list in a PIX that was running an older version of 
the software than he was familiar with.  The entire access-list disappeared and 
he was locked out, production stopped, like a train hitting a brick wall.  


 For the company the NOC team was the top
 tier of customer support (3rd line+), they looked after routers,
 switches, firewalls, servers, leased lines, and so on.
 This individual was perfectly capable of regurgitating all the facts,
 figures and technical details you can imagine, probably pretty much the
 entire CCNP syllabus.  What they didn't seem that capable of was
 actually applying that to anything.

You might be surprised at how common that is.  If you present them with ALL the 
diagrams and ALL of the configs on paper, they can figure it out.  In other 
words, if you recreate the same environment they had in their training class, 
they can do fine.  But what some can't seem to be able to do is visualize in 
their head how things are.  It is that layer of abstraction that separates them 
out.  They are fine for maintaining documentation or even for participating in 
a design review but you don't want them designing some new addition to the 
network or working on something live.  My first clue often comes from the 
quality of diagrams they produce. If the diagrams are accurate as far as what 
connects to what but do not reflect the actual flow of the network, that's a 
telltale sign.  Sort of like an electronic schematic.  If they sort of have 
random components/stages at random locations in the drawing that doesn't really 
reflect the functional flow through the device, that is my clue that I am 
likely dealing with a concrete thinker and not an abstract thinker.  Ditto if 
they demand that the symbol representing a particular piece of gear actually be 
a picture of that piece of gear.  If they get lost when gear is represented by 
a square box then they are probably part of the normal 85% of people who find 
it more difficult (actually have to try) to translate a square box on a diagram 
to a router in the rack in their head vs the 15% who do that naturally without 
any effort.

The access-list guy mentioned above would be great for looking at the config 
and producing a new one with the correct access control, but you wouldn't want 
him to be the one to apply it in production on a live network.  So even in that 
guy's case, there is a place where their skills can be quite useful and there 
are other places where their chance of making a costly mistake increase.  It is 
a matter of matching the person's role to their skills.

 I'd bet good money that if I'd
 asked him at the time what the 1918 network ranges are he'd have been
 able to tell me.

You'll be surprised how many people forget that 172.28.0.0 is rfc1918 space.  
They are so used to seeing 172.16 that they tend to forget 172.17-31. I've had 
to change null routes and access controls to include the entire /12.  They 
know that it is a /12 but seem to forget in practice when they see a second 
octet that isn't 16.


 This is exactly what we're teaching kids to do these days (makes me
 feel so old that I've already been saying this for several years and
 I'm only
 31) standardised tests aren't marked based on ability to apply
 knowledge, just the knowledge itself. 

Yes.  We teach them facts but now how to FIND facts.  Part of teaching is in 
teaching how to teach yourself.  It started with me when I was a kid.  When I 
had a question, my father would always say look it up and tell me even if he 
knew the answer perfectly well.  He had invested in an encyclopedia and the 
annual updates and was determined that I would use it.  It taught me how to 
research to find my own answers and it taught me to learn it well enough to 
explain it to someone else because Pop would always throw in a couple of 
questions for me after I explained it to him just to see if I actually got 
the underlying concept.  Besides, often in the course of researching one thing, 
I happened across a completely unrelated thing that caught my interest in that 
volume of the book and learned something I hadn't even been looking for.  
Forget the Internet, for people with kids at home, I would recommend a hard 
copy set of World Book with the Year Book and Science Year annual additions.  
That one in particular because the style in which they are written, they are 
actually pretty fun to read and have a lot of illustrations. 
http://www.worldbook.com/browse-all-products/item/953-advanced-research-package-2012
 (no affiliation at all with them, just a satisfied customer).  Soon, going 
to the books when a question arose became natural.

It is one thing to produce a teachable child, something quite different to 
produce the ability to learn independently and allow

Re: Common operational misconceptions

2012-02-18 Thread Justin M. Streiner


On Thu, 16 Feb 2012, Michael Sinatra wrote:

There was an old cruddy 1950s building on the UCB campus called Stanley Hall. 
(Now there's a new, nice, modern building on the UCB campus called Stanley 
Hall in place of the old one.)


It was great to take students on tours through this operational museum. 
(Well, the LBNL net was sort of operational.  It would just stop working for 
minutes on end and then come back mysteriously.)  You could basically see the 
first 10-15 years of the evolution of ethernet, and it was installed and 
working.


I have a few cruddy old 1950s (or earlier) buildings on campus where I can 
find thicknet and (dead for many years) vampire taps, along with thinnet, 
many different vintages of voice and data cabling, and many different 
vintages of fiber, including lots of OM1 multimode.  Makes for some very 
interesting pictures and what *is* that? conversations.


jms

Re: Common operational misconceptions

2012-02-18 Thread Michael Painter

Paul Graydon wrote:

Yes I'm serious, they were CCNP qualified, hired as a NOC engineer for

an ISP Hosting company. For the company the NOC team was the top tier
of customer support (3rd line+), they looked after routers, switches,
firewalls, servers, leased lines, and so on.
This individual was perfectly capable of regurgitating all the facts,
figures and technical details you can imagine, probably pretty much the
entire CCNP syllabus. What they didn't seem that capable of was
actually applying that to anything. I'd bet good money that if I'd
asked him at the time what the 1918 network ranges are he'd have been
able to tell me.
This is exactly what we're teaching kids to do these days (makes me feel
so old that I've already been saying this for several years and I'm only
31) standardised tests aren't marked based on ability to apply
knowledge, just the knowledge itself. Hence my view, give me someone
who knows how to think over someone who is qualified to the hilt. These
exam cram 'do a CCNP in a week' courses only serve to make it worse.

Paul

Ahh, I get you now...thanks.

Took me back to '64 and the battery of tests (all day!) I was given before getting hired by IBM for the 360 rollout. I
was amazed by the amount of questions of the if gear a turns ccw, what does lever b do? variety.
Later I was told that -all- the testing results were important, even the psychological ones, but what they really wanted
to find was the best analytical *mind*.

Best,

--Michael

Re: Common operational misconceptions

2012-02-17 Thread Leo Bicknell

In a message written on Thu, Feb 16, 2012 at 08:50:11PM -1000, Paul Graydon 
wrote:
 At the same time, it's shocking how many network people I come across 
 with no real grasp of even what OSI means by each layer, even if it's 
 only in theory.  Just having a grasp of that makes all the world of 
 difference when it comes to troubleshooting.  Start at layer 1 and work 
 upwards (unless you're able to make appropriate intuitive leaps.) Is it 
 physically connected? Are the link lights flashing? Can traffic route to 
 it, etc. etc.

I wouldn't call it a misconception, but I want to echo Paul's
comment.  I would venture over 90% of the engineers I work with
have no idea how to troubleshoot properly.  Thinking back to my own
education, I don't recall anyone in highschool or college attempting
to teach troubleshooting skills.  Most classes teach you how to
build things, not deal with them when they are broken.

The basic skills are probably obvious to someone who might design
course material if they sat down and thought about how to teach
troubleshooting.  However, there is one area that may not be obvious.
There's also a group management problem.  Many times troubleshooting
is done with multiple folks on the phone (say, customer, ISP and
vendor).  Not only do you have to know how to troubleshoot, but how
to get everyone on the same page so every possible cause isn't
tested 3 times.

I think all college level courses should include a break/fix
exercise/module after learning how to build something, and much of that
should be done in a group enviornment.

-- 
   Leo Bicknell - bickn...@ufp.org - CCIE 3440
PGP keys at http://www.ufp.org/~bicknell/


pgpqM2WcB0gd1.pgp
Description: PGP signature

Re: Common operational misconceptions

This list is awesome. Is anyone consolidating it? I'm still catching up 
on the thread


-Hammer-

I was a normal American nerd
-Jack Herer



On 2/17/2012 1:05 AM, Carsten Bormann wrote:

On Feb 17, 2012, at 07:50, Paul Graydon wrote:


what OSI means

Yet another common misconception popping up:

-- You can talk about the OSI model in the present tense

(That said -- yes, it is still useful as a set of simple terms for certain 
combinations of functions.
It is also still useful as a way to calibrate your gut feeling of what is going 
on in a network.
Just never expect OSI terms to have a precise meaning in today's networks.
1978 is now a third of a century ago...
If you need precision, you need to spell out what you mean in today's terms.)

Grüße, Carsten

RE: Common operational misconceptions

Actually I taught in a three year tech program for a while and although
trouble shooting was not in the curriculum, and as far as I know it
isn't anywhere, several of us adjunct faculty did teach it and got
reprimanded for it as part of our classes.  So much for the education
industry understanding the needs of business.  I taught basic PC
hardware and NT networking at the time.  We would actually have the
students assemble a PC and then in a subsequent class bring up a
network.  I got pretty good at nailing then with bugs while they were on
breaks. Heck, they had to go out to smoke. They would come back with a
network or PC that was no longer working.  I would then have them
explain what they saw, what they thought was wrong, justify it BEFORE
they could take any corrective action.  I also used some classroom
scenarios - they could ask me anything that they could physically learn
if they could tell me how they would check that.  I let them run bad
rabbit trails if it looked like it would cement the right way.  It
taught some step by step processes. 

BTW, the best trouble shooting course I have ever taken was the Kepner
Trego Problem Analysis/Decision Analysis class.  Caterpillar used it but
I have not seen it run anywhere in years.  It is hard-nosed and may not
be glitzy enough for today.  If you have a junior tech who isn't getting
there, I suggest - get their book and see if it helps.  NO, I do not
sell them or have stock in the company and NO, it will not do any good
unless he reads it.  I still use methodologies I learned in the class.


Ralph Brandt
Communications Engineer
HP Enterprise Services
Telephone +1 717.506.0802
FAX +1 717.506.4358
Email ralph.bra...@pateam.com
5095 Ritter Rd
Mechanicsburg PA 17055

-Original Message-
From: Leo Bicknell [mailto:bickn...@ufp.org] 
Sent: Friday, February 17, 2012 9:29 AM
To: nanog@nanog.org
Subject: Re: Common operational misconceptions

In a message written on Thu, Feb 16, 2012 at 08:50:11PM -1000, Paul
Graydon wrote:
 At the same time, it's shocking how many network people I come across 
 with no real grasp of even what OSI means by each layer, even if it's 
 only in theory.  Just having a grasp of that makes all the world of 
 difference when it comes to troubleshooting.  Start at layer 1 and
work 
 upwards (unless you're able to make appropriate intuitive leaps.) Is
it 
 physically connected? Are the link lights flashing? Can traffic route
to 
 it, etc. etc.

I wouldn't call it a misconception, but I want to echo Paul's
comment.  I would venture over 90% of the engineers I work with
have no idea how to troubleshoot properly.  Thinking back to my own
education, I don't recall anyone in highschool or college attempting
to teach troubleshooting skills.  Most classes teach you how to
build things, not deal with them when they are broken.

The basic skills are probably obvious to someone who might design
course material if they sat down and thought about how to teach
troubleshooting.  However, there is one area that may not be obvious.
There's also a group management problem.  Many times troubleshooting
is done with multiple folks on the phone (say, customer, ISP and
vendor).  Not only do you have to know how to troubleshoot, but how
to get everyone on the same page so every possible cause isn't
tested 3 times.

I think all college level courses should include a break/fix
exercise/module after learning how to build something, and much of that
should be done in a group enviornment.

-- 
   Leo Bicknell - bickn...@ufp.org - CCIE 3440
PGP keys at http://www.ufp.org/~bicknell/

Re: Common operational misconceptions


Let me simplify that. If you are over 35 you know how to troubleshoot.

Yes, I'm going to get flamed. Yes, there are exceptions in both directions.

-Hammer-

I was a normal American nerd
-Jack Herer



On 2/17/2012 8:29 AM, Leo Bicknell wrote:

In a message written on Thu, Feb 16, 2012 at 08:50:11PM -1000, Paul Graydon 
wrote:

At the same time, it's shocking how many network people I come across
with no real grasp of even what OSI means by each layer, even if it's
only in theory.  Just having a grasp of that makes all the world of
difference when it comes to troubleshooting.  Start at layer 1 and work
upwards (unless you're able to make appropriate intuitive leaps.) Is it
physically connected? Are the link lights flashing? Can traffic route to
it, etc. etc.

I wouldn't call it a misconception, but I want to echo Paul's
comment.  I would venture over 90% of the engineers I work with
have no idea how to troubleshoot properly.  Thinking back to my own
education, I don't recall anyone in highschool or college attempting
to teach troubleshooting skills.  Most classes teach you how to
build things, not deal with them when they are broken.

The basic skills are probably obvious to someone who might design
course material if they sat down and thought about how to teach
troubleshooting.  However, there is one area that may not be obvious.
There's also a group management problem.  Many times troubleshooting
is done with multiple folks on the phone (say, customer, ISP and
vendor).  Not only do you have to know how to troubleshoot, but how
to get everyone on the same page so every possible cause isn't
tested 3 times.

I think all college level courses should include a break/fix
exercise/module after learning how to build something, and much of that
should be done in a group enviornment.

RE: Common operational misconceptions

2012-02-17 Thread Mario Eirea

+1

Mario Eirea

From: Leo Bicknell [bickn...@ufp.org]
Sent: Friday, February 17, 2012 9:29 AM
To: nanog@nanog.org
Subject: Re: Common operational misconceptions

In a message written on Thu, Feb 16, 2012 at 08:50:11PM -1000, Paul Graydon 
wrote:
 At the same time, it's shocking how many network people I come across
 with no real grasp of even what OSI means by each layer, even if it's
 only in theory.  Just having a grasp of that makes all the world of
 difference when it comes to troubleshooting.  Start at layer 1 and work
 upwards (unless you're able to make appropriate intuitive leaps.) Is it
 physically connected? Are the link lights flashing? Can traffic route to
 it, etc. etc.

I wouldn't call it a misconception, but I want to echo Paul's
comment.  I would venture over 90% of the engineers I work with
have no idea how to troubleshoot properly.  Thinking back to my own
education, I don't recall anyone in highschool or college attempting
to teach troubleshooting skills.  Most classes teach you how to
build things, not deal with them when they are broken.

The basic skills are probably obvious to someone who might design
course material if they sat down and thought about how to teach
troubleshooting.  However, there is one area that may not be obvious.
There's also a group management problem.  Many times troubleshooting
is done with multiple folks on the phone (say, customer, ISP and
vendor).  Not only do you have to know how to troubleshoot, but how
to get everyone on the same page so every possible cause isn't
tested 3 times.

I think all college level courses should include a break/fix
exercise/module after learning how to build something, and much of that
should be done in a group enviornment.

--
   Leo Bicknell - bickn...@ufp.org - CCIE 3440
PGP keys at http://www.ufp.org/~bicknell/

RE: Common operational misconceptions

Hammer, you are at least 75% right.  You will get flamed and in most
cases, the 35 year age is close to right.  

But then in Programming where I spent most of my IT time since Feb 1963,
few current programmers have skills that they need to be really
successful.  Same thing.  

It is the fault of the educational system like one school district here
that teaches Alice, VB and then two days of C++ to High School Kids.
Heck, they will fiddle with Alice on their own.  They need some exposure
to one of the SQL's and how to build some tables, maybe a good script
language, some command line on SQL+ and unix or PostgresSQL and linux if
the school can't afford the unix licenses. 

The fun and games is more important than the substance and it goes into
the colleges in spades.

BTW, I am a school board member who votes 1:8 often on things But
let me give you a perspective, one of the board members called Golf an
Essential Life Skill.  Maybe, but how about balancing a checkbook...

Ralph Brandt
Communications Engineer
HP Enterprise Services
Telephone +1 717.506.0802
FAX +1 717.506.4358
Email ralph.bra...@pateam.com
5095 Ritter Rd
Mechanicsburg PA 17055


-Original Message-
From: -Hammer- [mailto:bhmc...@gmail.com] 
Sent: Friday, February 17, 2012 9:52 AM
To: nanog@nanog.org
Subject: Re: Common operational misconceptions

Let me simplify that. If you are over 35 you know how to troubleshoot.

Yes, I'm going to get flamed. Yes, there are exceptions in both
directions.

-Hammer-

I was a normal American nerd
-Jack Herer



On 2/17/2012 8:29 AM, Leo Bicknell wrote:
 In a message written on Thu, Feb 16, 2012 at 08:50:11PM -1000, Paul
Graydon wrote:
 At the same time, it's shocking how many network people I come across
 with no real grasp of even what OSI means by each layer, even if it's
 only in theory.  Just having a grasp of that makes all the world of
 difference when it comes to troubleshooting.  Start at layer 1 and
work
 upwards (unless you're able to make appropriate intuitive leaps.) Is
it
 physically connected? Are the link lights flashing? Can traffic route
to
 it, etc. etc.
 I wouldn't call it a misconception, but I want to echo Paul's
 comment.  I would venture over 90% of the engineers I work with
 have no idea how to troubleshoot properly.  Thinking back to my own
 education, I don't recall anyone in highschool or college attempting
 to teach troubleshooting skills.  Most classes teach you how to
 build things, not deal with them when they are broken.

 The basic skills are probably obvious to someone who might design
 course material if they sat down and thought about how to teach
 troubleshooting.  However, there is one area that may not be obvious.
 There's also a group management problem.  Many times troubleshooting
 is done with multiple folks on the phone (say, customer, ISP and
 vendor).  Not only do you have to know how to troubleshoot, but how
 to get everyone on the same page so every possible cause isn't
 tested 3 times.

 I think all college level courses should include a break/fix
 exercise/module after learning how to build something, and much of
that
 should be done in a group enviornment.

RE: Common operational misconceptions

2012-02-17 Thread Mario Eirea

Well, I will argue this. I think the important factor in any troubleshooting is 
having a real understanding of how the system works. That is, how different 
things interact with each others to achieve a specific goal. The biggest 
problem I see is that many people understand understand the individual parts 
but when it comes to understanding the system as a whole they fall miserably 
short. 

A short example, probably not the best but the one that comes to mind right now:

Someone replaces a device on the network with a new one. They give it the same 
IP address as the old device. They don't understand why the router cant 
communicate with it at first and then starts working. The people understand 
ARP, but cant correlate one event with another.

I guess if your 35 you have seen this at least once and can fix it. But what 
happens if you have never seen this problem or a related one? At this point 
your going to have to really troubleshoot, not just go on experience.

Mario Eirea

From: -Hammer- [bhmc...@gmail.com]
Sent: Friday, February 17, 2012 9:52 AM
To: nanog@nanog.org
Subject: Re: Common operational misconceptions

Let me simplify that. If you are over 35 you know how to troubleshoot.

Yes, I'm going to get flamed. Yes, there are exceptions in both directions.

-Hammer-

I was a normal American nerd
-Jack Herer



On 2/17/2012 8:29 AM, Leo Bicknell wrote:
 In a message written on Thu, Feb 16, 2012 at 08:50:11PM -1000, Paul Graydon 
 wrote:
 At the same time, it's shocking how many network people I come across
 with no real grasp of even what OSI means by each layer, even if it's
 only in theory.  Just having a grasp of that makes all the world of
 difference when it comes to troubleshooting.  Start at layer 1 and work
 upwards (unless you're able to make appropriate intuitive leaps.) Is it
 physically connected? Are the link lights flashing? Can traffic route to
 it, etc. etc.
 I wouldn't call it a misconception, but I want to echo Paul's
 comment.  I would venture over 90% of the engineers I work with
 have no idea how to troubleshoot properly.  Thinking back to my own
 education, I don't recall anyone in highschool or college attempting
 to teach troubleshooting skills.  Most classes teach you how to
 build things, not deal with them when they are broken.

 The basic skills are probably obvious to someone who might design
 course material if they sat down and thought about how to teach
 troubleshooting.  However, there is one area that may not be obvious.
 There's also a group management problem.  Many times troubleshooting
 is done with multiple folks on the phone (say, customer, ISP and
 vendor).  Not only do you have to know how to troubleshoot, but how
 to get everyone on the same page so every possible cause isn't
 tested 3 times.

 I think all college level courses should include a break/fix
 exercise/module after learning how to build something, and much of that
 should be done in a group enviornment.

Re: Common operational misconceptions

2012-02-17 Thread Jack Bates


On 2/17/2012 1:05 AM, Carsten Bormann wrote:

On Feb 17, 2012, at 07:50, Paul Graydon wrote:


what OSI means


Yet another common misconception popping up:

-- You can talk about the OSI model in the present tense

(That said -- yes, it is still useful as a set of simple terms for certain 
combinations of functions.
It is also still useful as a way to calibrate your gut feeling of what is going 
on in a network.
Just never expect OSI terms to have a precise meaning in today's networks.
1978 is now a third of a century ago...
If you need precision, you need to spell out what you mean in today's terms.)



Actually, I find it makes a perfect troubleshooting guideline in today's 
world; at least up to layer 4. I'm not saying it's a perfect match to 
troubleshooting a variety of MPLS problems, but it is a reminder that 
there are dependencies which must be checked.


In dealing with transport companies, the model is still a good 
representation of their service levels. It isn't uncommon to find their 
products defined as layer 2 services (ranging from tdm/sonet services to 
ethernet switching services), layer 3 services (often handled by their 
ISP department), and MPLS services (which can range from p2p transport 
to l3vpn).


Which brings up my final point. Until we quit naming things l2vpn or 
l3vpn, OSI still applies. :P



Jack

Re: Common operational misconceptions

2012-02-17 Thread Steve Clark


I agree with this 100%.

Having worked with many people over the last 40 years, the good trouble 
shooters understood how things
were suppose to work. This helps immeasurably in determining where to start 
looking.

On 02/17/2012 10:12 AM, Mario Eirea wrote:

Well, I will argue this. I think the important factor in any troubleshooting is 
having a real understanding of how the system works. That is, how different 
things interact with each others to achieve a specific goal. The biggest 
problem I see is that many people understand understand the individual parts 
but when it comes to understanding the system as a whole they fall miserably 
short.

A short example, probably not the best but the one that comes to mind right now:

Someone replaces a device on the network with a new one. They give it the same IP address 
as the old device. They don't understand why the router cant communicate with it at first 
and then starts working. The people understand ARP, but cant correlate one 
event with another.

I guess if your 35 you have seen this at least once and can fix it. But what 
happens if you have never seen this problem or a related one? At this point 
your going to have to really troubleshoot, not just go on experience.

Mario Eirea

From: -Hammer- [bhmc...@gmail.com]
Sent: Friday, February 17, 2012 9:52 AM
To: nanog@nanog.org
Subject: Re: Common operational misconceptions

Let me simplify that. If you are over 35 you know how to troubleshoot.

Yes, I'm going to get flamed. Yes, there are exceptions in both directions.

-Hammer-

I was a normal American nerd
-Jack Herer



On 2/17/2012 8:29 AM, Leo Bicknell wrote:

In a message written on Thu, Feb 16, 2012 at 08:50:11PM -1000, Paul Graydon 
wrote:

At the same time, it's shocking how many network people I come across
with no real grasp of even what OSI means by each layer, even if it's
only in theory.  Just having a grasp of that makes all the world of
difference when it comes to troubleshooting.  Start at layer 1 and work
upwards (unless you're able to make appropriate intuitive leaps.) Is it
physically connected? Are the link lights flashing? Can traffic route to
it, etc. etc.

I wouldn't call it a misconception, but I want to echo Paul's
comment.  I would venture over 90% of the engineers I work with
have no idea how to troubleshoot properly.  Thinking back to my own
education, I don't recall anyone in highschool or college attempting
to teach troubleshooting skills.  Most classes teach you how to
build things, not deal with them when they are broken.

The basic skills are probably obvious to someone who might design
course material if they sat down and thought about how to teach
troubleshooting.  However, there is one area that may not be obvious.
There's also a group management problem.  Many times troubleshooting
is done with multiple folks on the phone (say, customer, ISP and
vendor).  Not only do you have to know how to troubleshoot, but how
to get everyone on the same page so every possible cause isn't
tested 3 times.

I think all college level courses should include a break/fix
exercise/module after learning how to build something, and much of that
should be done in a group enviornment.







--
Stephen Clark
*NetWolves*
Director of Technology
Phone: 813-579-3200
Fax: 813-882-0209
Email: steve.cl...@netwolves.com
http://www.netwolves.com

Re: Common operational misconceptions

2012-02-17 Thread Jay Ashworth

- Original Message -
 From: Ridwan Sami rms2...@columbia.edu

 There is no legitimate reason for a user to use BitTorrent (someone
 will probably disagree with this).

Yeah, no.

You've clearly never tried to download a Linux installer DVD.

Cheers,
-- jr 'among dozens of other legitimate uses' a
-- 
Jay R. Ashworth  Baylink   j...@baylink.com
Designer The Things I Think   RFC 2100
Ashworth  Associates http://baylink.pitas.com 2000 Land Rover DII
St Petersburg FL USA  http://photo.imageinc.us +1 727 647 1274

Re: Common operational misconceptions

2012-02-17 Thread Jared Mauch


On Feb 17, 2012, at 9:29 AM, -Hammer- wrote:

 This list is awesome. Is anyone consolidating it? I'm still catching up on 
 the thread


I was thinking of making a checklist out of it.

- Jared

Re: Common operational misconceptions

2012-02-17 Thread Sven Olaf Kamphuis


There is no legitimate reason for a user to use BitTorrent (someone
will probably disagree with this).


There is no democratic basis -for- copyright, so far for legitimate.

Re: Common operational misconceptions


Well said. An American tragedy.

-Hammer-

I was a normal American nerd
-Jack Herer



On 2/17/2012 9:01 AM, Brandt, Ralph wrote:

Hammer, you are at least 75% right.  You will get flamed and in most
cases, the 35 year age is close to right.

But then in Programming where I spent most of my IT time since Feb 1963,
few current programmers have skills that they need to be really
successful.  Same thing.

It is the fault of the educational system like one school district here
that teaches Alice, VB and then two days of C++ to High School Kids.
Heck, they will fiddle with Alice on their own.  They need some exposure
to one of the SQL's and how to build some tables, maybe a good script
language, some command line on SQL+ and unix or PostgresSQL and linux if
the school can't afford the unix licenses.

The fun and games is more important than the substance and it goes into
the colleges in spades.

BTW, I am a school board member who votes 1:8 often on things But
let me give you a perspective, one of the board members called Golf an
Essential Life Skill.  Maybe, but how about balancing a checkbook...

Ralph Brandt
Communications Engineer
HP Enterprise Services
Telephone +1 717.506.0802
FAX +1 717.506.4358
Email ralph.bra...@pateam.com
5095 Ritter Rd
Mechanicsburg PA 17055


-Original Message-
From: -Hammer- [mailto:bhmc...@gmail.com]
Sent: Friday, February 17, 2012 9:52 AM
To: nanog@nanog.org
Subject: Re: Common operational misconceptions

Let me simplify that. If you are over 35 you know how to troubleshoot.

Yes, I'm going to get flamed. Yes, there are exceptions in both
directions.

-Hammer-

I was a normal American nerd
-Jack Herer



On 2/17/2012 8:29 AM, Leo Bicknell wrote:

In a message written on Thu, Feb 16, 2012 at 08:50:11PM -1000, Paul

Graydon wrote:

At the same time, it's shocking how many network people I come across
with no real grasp of even what OSI means by each layer, even if it's
only in theory.  Just having a grasp of that makes all the world of
difference when it comes to troubleshooting.  Start at layer 1 and

work

upwards (unless you're able to make appropriate intuitive leaps.) Is

it

physically connected? Are the link lights flashing? Can traffic route

to

it, etc. etc.

I wouldn't call it a misconception, but I want to echo Paul's
comment.  I would venture over 90% of the engineers I work with
have no idea how to troubleshoot properly.  Thinking back to my own
education, I don't recall anyone in highschool or college attempting
to teach troubleshooting skills.  Most classes teach you how to
build things, not deal with them when they are broken.

The basic skills are probably obvious to someone who might design
course material if they sat down and thought about how to teach
troubleshooting.  However, there is one area that may not be obvious.
There's also a group management problem.  Many times troubleshooting
is done with multiple folks on the phone (say, customer, ISP and
vendor).  Not only do you have to know how to troubleshoot, but how
to get everyone on the same page so every possible cause isn't
tested 3 times.

I think all college level courses should include a break/fix
exercise/module after learning how to build something, and much of

that

should be done in a group enviornment.

Re: Common operational misconceptions

2012-02-17 Thread Jack Bates




On 2/17/2012 9:18 AM, Steve Clark wrote:

Having worked with many people over the last 40 years, the good trouble
shooters understood how things
were suppose to work. This helps immeasurably in determining where to
start looking.



Ran into this not too long ago with a transport problem. The behavior I 
was seeing was indicative of the transport not stripping their outer 
tag. They put wireshark on a windows laptop and sent me the traffic 
captures. While I didn't know that M$ decided to do something silly like 
removing a single tag, all indicators were that the M$ stack fixed 
whatever was broken prior to wireshark. We took a capture from another 
device and proved the problem.


Which is a common transport problem I often see, Our configuration 
looks right, it must be on your end.



Jack

Re: Common operational misconceptions


Mario,
I was kinda having fun and kinda not. My point is that the 40-50 
year olds that were doing this 30 years ago grew up understanding things 
in order. Bits. Bytes. KiloBits. KiloBytes. (Some folks still get those 
confused). Hex. etc. Move on to the OSI model and it's the same thing. 
Voltage. Amplitude. Binary. etc. I think that this generation that I'm 
referring to is a great generation because we were at the beginning of 
the Internet blooming. There are folks on this forum that go back 
further. Into DARPA. Before DARPA was just chapter 1 one every single 
Cisco Press book. They have a unique understanding of the layers. I had 
that understanding in my 20s. The technology is so complicated these 
days that many folks miss those fundamentals and go right into VSS on 
the 6500s or MPLS over Juniper. In the end, it all comes in time.


-Hammer-

I was a normal American nerd
-Jack Herer



On 2/17/2012 9:12 AM, Mario Eirea wrote:

Well, I will argue this. I think the important factor in any troubleshooting is 
having a real understanding of how the system works. That is, how different 
things interact with each others to achieve a specific goal. The biggest 
problem I see is that many people understand understand the individual parts 
but when it comes to understanding the system as a whole they fall miserably 
short.

A short example, probably not the best but the one that comes to mind right now:

Someone replaces a device on the network with a new one. They give it the same IP address 
as the old device. They don't understand why the router cant communicate with it at first 
and then starts working. The people understand ARP, but cant correlate one 
event with another.

I guess if your 35 you have seen this at least once and can fix it. But what 
happens if you have never seen this problem or a related one? At this point 
your going to have to really troubleshoot, not just go on experience.

Mario Eirea

From: -Hammer- [bhmc...@gmail.com]
Sent: Friday, February 17, 2012 9:52 AM
To: nanog@nanog.org
Subject: Re: Common operational misconceptions

Let me simplify that. If you are over 35 you know how to troubleshoot.

Yes, I'm going to get flamed. Yes, there are exceptions in both directions.

-Hammer-

I was a normal American nerd
-Jack Herer



On 2/17/2012 8:29 AM, Leo Bicknell wrote:

In a message written on Thu, Feb 16, 2012 at 08:50:11PM -1000, Paul Graydon 
wrote:

At the same time, it's shocking how many network people I come across
with no real grasp of even what OSI means by each layer, even if it's
only in theory.  Just having a grasp of that makes all the world of
difference when it comes to troubleshooting.  Start at layer 1 and work
upwards (unless you're able to make appropriate intuitive leaps.) Is it
physically connected? Are the link lights flashing? Can traffic route to
it, etc. etc.

I wouldn't call it a misconception, but I want to echo Paul's
comment.  I would venture over 90% of the engineers I work with
have no idea how to troubleshoot properly.  Thinking back to my own
education, I don't recall anyone in highschool or college attempting
to teach troubleshooting skills.  Most classes teach you how to
build things, not deal with them when they are broken.

The basic skills are probably obvious to someone who might design
course material if they sat down and thought about how to teach
troubleshooting.  However, there is one area that may not be obvious.
There's also a group management problem.  Many times troubleshooting
is done with multiple folks on the phone (say, customer, ISP and
vendor).  Not only do you have to know how to troubleshoot, but how
to get everyone on the same page so every possible cause isn't
tested 3 times.

I think all college level courses should include a break/fix
exercise/module after learning how to build something, and much of that
should be done in a group enviornment.

Re: Common operational misconceptions


If you do, please share it. Thank you.

-Hammer-

I was a normal American nerd
-Jack Herer



On 2/17/2012 9:36 AM, Jared Mauch wrote:

On Feb 17, 2012, at 9:29 AM, -Hammer- wrote:


This list is awesome. Is anyone consolidating it? I'm still catching up on the 
thread


I was thinking of making a checklist out of it.

- Jared

Re: Common operational misconceptions

2012-02-17 Thread Charles Mills

Original poster who started thread said he would.

On Fri, Feb 17, 2012 at 10:51 AM, -Hammer- bhmc...@gmail.com wrote:

 If you do, please share it. Thank you.


 -Hammer-

 I was a normal American nerd
 -Jack Herer



 On 2/17/2012 9:36 AM, Jared Mauch wrote:

 On Feb 17, 2012, at 9:29 AM, -Hammer- wrote:

  This list is awesome. Is anyone consolidating it? I'm still catching up
 on the thread


 I was thinking of making a checklist out of it.

 - Jared

Re: Common operational misconceptions

2012-02-17 Thread John Kristoff

On Fri, 17 Feb 2012 08:29:42 -0600
-Hammer- bhmc...@gmail.com wrote:

 This list is awesome. Is anyone consolidating it? I'm still catching
 up on the thread

I'm collecting all responses, many of which have been sent to me off
list.  I was waiting for the thread to eventually end before
summarizing it.  Thanks everyone.

John

Re: Common operational misconceptions

2012-02-17 Thread Jack Bates


On 2/17/2012 10:04 AM, John Kristoff wrote:

I was waiting for the thread to eventually end


Greatest misconception of all.


Jack

RE: Common operational misconceptions

 
 It depends on how you define work well.
 
 As the RFC says:
 
indication of some significantly disruptive event in the network,
such as a router failure or a routing change to a path with a
 smaller
MTU.
 
 it can not react against PMTU changes very well.
 
 It is seemingly working well means there is not much PMTU changes,
 which means we had better assumes some PMTU (1280B, for example) and
 use it without PMTUD.
 
   Masataka Ohta

It depends on the OS and the method being used.  If you set the option to 2 
on Linux, it will do MTU probing constantly and react to MTU changes.  Also, 
the MTU for a given path only lives for 5 minutes anyway (by default) and is 
rediscovered with Linux.   (value in /proc/sys/net/ipv4/route/mtu_expires) 
but other operating systems may behave in other ways.

I agree that if one tries hard enough, they can ensure a broken path and there 
are always people who seem to devote a lot of energy to that end.  There's 
nothing that can be done about them but there is much the OS can try to do to 
defeat them at their task.

RE: Common operational misconceptions

2012-02-17 Thread Mario Eirea

I definitely understand and agree with what you saying. Actually, most my 
friends are over 50 years old... I do agree with you on the generational 
statement. My argument was that many people over 35 have no idea what they are 
doing, and some under 35 do know what they are doing. On thing is for sure, 
experience goes a long way. The importance is knowing the fundamentals and 
putting it all together (a skill that has been lost in recent times)

I have a lot to say about the current generation of people growing up in this 
country, but that's a whole other thread in a whole other list. :-)

Mario Eirea


From: -Hammer- [bhmc...@gmail.com]
Sent: Friday, February 17, 2012 10:51 AM
To: Mario Eirea
Cc: nanog@nanog.org
Subject: Re: Common operational misconceptions

Mario,
 I was kinda having fun and kinda not. My point is that the 40-50
year olds that were doing this 30 years ago grew up understanding things
in order. Bits. Bytes. KiloBits. KiloBytes. (Some folks still get those
confused). Hex. etc. Move on to the OSI model and it's the same thing.
Voltage. Amplitude. Binary. etc. I think that this generation that I'm
referring to is a great generation because we were at the beginning of
the Internet blooming. There are folks on this forum that go back
further. Into DARPA. Before DARPA was just chapter 1 one every single
Cisco Press book. They have a unique understanding of the layers. I had
that understanding in my 20s. The technology is so complicated these
days that many folks miss those fundamentals and go right into VSS on
the 6500s or MPLS over Juniper. In the end, it all comes in time.

-Hammer-

I was a normal American nerd
-Jack Herer



On 2/17/2012 9:12 AM, Mario Eirea wrote:
 Well, I will argue this. I think the important factor in any troubleshooting 
 is having a real understanding of how the system works. That is, how 
 different things interact with each others to achieve a specific goal. The 
 biggest problem I see is that many people understand understand the 
 individual parts but when it comes to understanding the system as a whole 
 they fall miserably short.

 A short example, probably not the best but the one that comes to mind right 
 now:

 Someone replaces a device on the network with a new one. They give it the 
 same IP address as the old device. They don't understand why the router cant 
 communicate with it at first and then starts working. The people understand 
 ARP, but cant correlate one event with another.

 I guess if your 35 you have seen this at least once and can fix it. But what 
 happens if you have never seen this problem or a related one? At this point 
 your going to have to really troubleshoot, not just go on experience.

 Mario Eirea
 
 From: -Hammer- [bhmc...@gmail.com]
 Sent: Friday, February 17, 2012 9:52 AM
 To: nanog@nanog.org
 Subject: Re: Common operational misconceptions

 Let me simplify that. If you are over 35 you know how to troubleshoot.

 Yes, I'm going to get flamed. Yes, there are exceptions in both directions.

 -Hammer-

 I was a normal American nerd
 -Jack Herer



 On 2/17/2012 8:29 AM, Leo Bicknell wrote:
 In a message written on Thu, Feb 16, 2012 at 08:50:11PM -1000, Paul Graydon 
 wrote:
 At the same time, it's shocking how many network people I come across
 with no real grasp of even what OSI means by each layer, even if it's
 only in theory.  Just having a grasp of that makes all the world of
 difference when it comes to troubleshooting.  Start at layer 1 and work
 upwards (unless you're able to make appropriate intuitive leaps.) Is it
 physically connected? Are the link lights flashing? Can traffic route to
 it, etc. etc.
 I wouldn't call it a misconception, but I want to echo Paul's
 comment.  I would venture over 90% of the engineers I work with
 have no idea how to troubleshoot properly.  Thinking back to my own
 education, I don't recall anyone in highschool or college attempting
 to teach troubleshooting skills.  Most classes teach you how to
 build things, not deal with them when they are broken.

 The basic skills are probably obvious to someone who might design
 course material if they sat down and thought about how to teach
 troubleshooting.  However, there is one area that may not be obvious.
 There's also a group management problem.  Many times troubleshooting
 is done with multiple folks on the phone (say, customer, ISP and
 vendor).  Not only do you have to know how to troubleshoot, but how
 to get everyone on the same page so every possible cause isn't
 tested 3 times.

 I think all college level courses should include a break/fix
 exercise/module after learning how to build something, and much of that
 should be done in a group enviornment.

Re: Common operational misconceptions

2012-02-17 Thread Scott Helms


On 2/17/2012 10:18 AM, Steve Clark wrote:

I agree with this 100%.

Having worked with many people over the last 40 years, the good 
trouble shooters understood how things
were suppose to work. This helps immeasurably in determining where to 
start looking.




This is dead on and why I always start classes with a refresher on the 
OSI model.  While the model isn't perfect it lets technicians and 
engineers construct a reasonable model of how things *ought* to be 
working.  While you certainly will run into devices that bend or even 
break the rules (sometimes for good reasons) its much easier to 
understand the exceptions if you know the normal operation for a 
repeater, bridge, or router.


--
Scott Helms
Vice President of Technology
ISP Alliance, Inc. DBA ZCorum
(678) 507-5000

http://twitter.com/kscotthelms

Re: Common operational misconceptions

2012-02-17 Thread Eugen Leitl

On Fri, Feb 17, 2012 at 10:33:12AM -0500, Jay Ashworth wrote:
 - Original Message -
  From: Ridwan Sami rms2...@columbia.edu
 
  There is no legitimate reason for a user to use BitTorrent (someone
  will probably disagree with this).
 
 Yeah, no.
 
 You've clearly never tried to download a Linux installer DVD.

Nevermind that Bram Cohen is preparing to tackle TV with a
BitTorrent-related protocol (no further details known yet).

Re: Common operational misconceptions

2012-02-17 Thread Mike Lyon

Sent from my iPhone

On Feb 17, 2012, at 7:48, Jack Bates jba...@brightok.net wrote:



 On 2/17/2012 9:18 AM, Steve Clark wrote:
 Having worked with many people over the last 40 years, the good trouble
 shooters understood how things
 were suppose to work. This helps immeasurably in determining where to
 start looking.


 Ran into this not too long ago with a transport problem. The behavior I was 
 seeing was indicative of the transport not stripping their outer tag. They 
 put wireshark on a windows laptop and sent me the traffic captures. While I 
 didn't know that M$ decided to do something silly like removing a single tag, 
 all indicators were that the M$ stack fixed whatever was broken prior to 
 wireshark. We took a capture from another device and proved the problem.

 Which is a common transport problem I often see, Our configuration looks 
 right, it must be on your end.


 Jack



If i had a dollar for everytime i've heard that from a telco, i'd be a
rich man...

Re: Common operational misconceptions

2012-02-17 Thread Gary Buhrmaster

On Fri, Feb 17, 2012 at 06:52, -Hammer- bhmc...@gmail.com wrote:
 Let me simplify that. If you are over 35 you know how to troubleshoot.

 Yes, I'm going to get flamed. Yes, there are exceptions in both directions.

Necessity is the mother of invention

Long before there was a Grainger (and Home Depot) in
every city, and you could get parts shipped overnight,
one had to make do, and making do meant being
able to figure things out to be able to git r done
with what you had on hand, or could figure out.

When working on my Grandfather's farm, I did not
look for work to do (actually, I looked for ways
not to do any work :-), but if the project required
pulling out the oxy-acetylene torch to cut and
weld something onto the tractor to get something
done, that is what you had to do, so you did it.
If the TV went on the blink (they all did then),
you opened up the back, looked for fried
components, and if one of the resistors was
smoking, you soldered in a replacement.  Or
you took the tubes down to the local drugstore
and tested them.  Even if you had no idea what
you were doing, you were willing (and expected)
to give it a shot, and try to fix it.  More often
than not you learned something along the way,
even if it took hours to figure it out (and had to
repair your repair a few times :-).  For those
without the capabilities, you took it to the shop,
where someone else did the troubleshooting
and repair.

Along the line, the costs of technicians to
do that type of work started to exceed the
cost of simply replacing the entire unit
(how many people remember when going
to the auto dealer that the cost of the parts
far exceeded the cost of the labor?  Now it
is the other way around).  Troubleshooting
became a lost art.  Swap 'til you drop
became the mantra.  It became the cost
effective way to do repairs.

There are advantages to the new way of
disposable devices, but almost no one
knows how they work anymore, and they
do not care to know.  The members of this
list are likely to be sufficiently self selected
to be in the minority of actually wanting to
know.

There is a (small) backlash of people who
are trying to get back into the world of
actually building things, and understanding
how they work (popularized by such things
as Make magazine, and Maker Faires).

Gary

Re: Common operational misconceptions

2012-02-17 Thread Mike Andrews

On Fri, Feb 17, 2012 at 08:46:02AM -0800, Mike Lyon wrote:
 Sent from my iPhone
 
 On Feb 17, 2012, at 7:48, Jack Bates jba...@brightok.net wrote:
 
 
 
  On 2/17/2012 9:18 AM, Steve Clark wrote:
  Having worked with many people over the last 40 years, the good trouble
  shooters understood how things
  were suppose to work. This helps immeasurably in determining where to
  start looking.
 
 
  Ran into this not too long ago with a transport problem. The behavior I was 
  seeing was indicative of the transport not stripping their outer tag. They 
  put wireshark on a windows laptop and sent me the traffic captures. While I 
  didn't know that M$ decided to do something silly like removing a single 
  tag, all indicators were that the M$ stack fixed whatever was broken 
  prior to wireshark. We took a capture from another device and proved the 
  problem.
 
  Which is a common transport problem I often see, Our configuration looks 
  right, it must be on your end.
 
 If i had a dollar for everytime i've heard that from a telco, i'd be a
 rich man...

That and I'm getting a good ping response here while I've got the cable
at my end unplugged from the equipment.

-- 
Mike Andrews, W5EGO
mi...@mikea.ath.cx
Tired old sysadmin

Re: Common operational misconceptions

2012-02-17 Thread Marcel Plug

I often struggle with the concept of teaching someone how to
troubleshoot.  We have young guys coming in all the time and it is
often an area in which they need to hone their skills.  I found this
article a while back and I keep it bookmarked, its the best article
I've seen that lays it all out pretty clearly.

http://packetlife.net/blog/2010/mar/10/the-science-of-network-troubleshooting/

-Marcel

I'm guessing, but I believe the author would fall into the under 35 category ;-)

On Fri, Feb 17, 2012 at 10:51 AM, -Hammer- bhmc...@gmail.com wrote:
 Mario,
    I was kinda having fun and kinda not. My point is that the 40-50 year
 olds that were doing this 30 years ago grew up understanding things in
 order. Bits. Bytes. KiloBits. KiloBytes. (Some folks still get those
 confused). Hex. etc. Move on to the OSI model and it's the same thing.
 Voltage. Amplitude. Binary. etc. I think that this generation that I'm
 referring to is a great generation because we were at the beginning of the
 Internet blooming. There are folks on this forum that go back further. Into
 DARPA. Before DARPA was just chapter 1 one every single Cisco Press book.
 They have a unique understanding of the layers. I had that understanding in
 my 20s. The technology is so complicated these days that many folks miss
 those fundamentals and go right into VSS on the 6500s or MPLS over Juniper.
 In the end, it all comes in time.


 -Hammer-

 I was a normal American nerd
 -Jack Herer



 On 2/17/2012 9:12 AM, Mario Eirea wrote:

 Well, I will argue this. I think the important factor in any
 troubleshooting is having a real understanding of how the system works. That
 is, how different things interact with each others to achieve a specific
 goal. The biggest problem I see is that many people understand understand
 the individual parts but when it comes to understanding the system as a
 whole they fall miserably short.

 A short example, probably not the best but the one that comes to mind
 right now:

 Someone replaces a device on the network with a new one. They give it the
 same IP address as the old device. They don't understand why the router cant
 communicate with it at first and then starts working. The people
 understand ARP, but cant correlate one event with another.

 I guess if your 35 you have seen this at least once and can fix it. But
 what happens if you have never seen this problem or a related one? At this
 point your going to have to really troubleshoot, not just go on experience.

 Mario Eirea
 
 From: -Hammer- [bhmc...@gmail.com]
 Sent: Friday, February 17, 2012 9:52 AM
 To: nanog@nanog.org
 Subject: Re: Common operational misconceptions

 Let me simplify that. If you are over 35 you know how to troubleshoot.

 Yes, I'm going to get flamed. Yes, there are exceptions in both
 directions.

 -Hammer-

 I was a normal American nerd
 -Jack Herer



 On 2/17/2012 8:29 AM, Leo Bicknell wrote:

 In a message written on Thu, Feb 16, 2012 at 08:50:11PM -1000, Paul
 Graydon wrote:

 At the same time, it's shocking how many network people I come across
 with no real grasp of even what OSI means by each layer, even if it's
 only in theory.  Just having a grasp of that makes all the world of
 difference when it comes to troubleshooting.  Start at layer 1 and work
 upwards (unless you're able to make appropriate intuitive leaps.) Is it
 physically connected? Are the link lights flashing? Can traffic route to
 it, etc. etc.

 I wouldn't call it a misconception, but I want to echo Paul's
 comment.  I would venture over 90% of the engineers I work with
 have no idea how to troubleshoot properly.  Thinking back to my own
 education, I don't recall anyone in highschool or college attempting
 to teach troubleshooting skills.  Most classes teach you how to
 build things, not deal with them when they are broken.

 The basic skills are probably obvious to someone who might design
 course material if they sat down and thought about how to teach
 troubleshooting.  However, there is one area that may not be obvious.
 There's also a group management problem.  Many times troubleshooting
 is done with multiple folks on the phone (say, customer, ISP and
 vendor).  Not only do you have to know how to troubleshoot, but how
 to get everyone on the same page so every possible cause isn't
 tested 3 times.

 I think all college level courses should include a break/fix
 exercise/module after learning how to build something, and much of that
 should be done in a group enviornment.

Re: Common operational misconceptions

2012-02-17 Thread Sven Olaf Kamphuis

wasn't tv already tackled by dvb-iptv + multicast (oh wait, multicast, 
that stuff that hardly ever globally works on ipv4 ;)


(yes, i'm that old that i even know what a tv was ;)

On Fri, 17 Feb 2012, Eugen Leitl wrote:


On Fri, Feb 17, 2012 at 10:33:12AM -0500, Jay Ashworth wrote:

- Original Message -

From: Ridwan Sami rms2...@columbia.edu



There is no legitimate reason for a user to use BitTorrent (someone
will probably disagree with this).


Yeah, no.

You've clearly never tried to download a Linux installer DVD.


Nevermind that Bram Cohen is preparing to tackle TV with a
BitTorrent-related protocol (no further details known yet).

RE: Common operational misconceptions

 
 I wouldn't call it a misconception, but I want to echo Paul's
 comment.  I would venture over 90% of the engineers I work with have no
 idea how to troubleshoot properly.  Thinking back to my own education,
 I don't recall anyone in highschool or college attempting to teach
 troubleshooting skills.  Most classes teach you how to build things,
 not deal with them when they are broken.

Look for people who grew up on a farm.  They are used to figuring out how to 
fix things they haven't seen before and generally attempt to gain knowledge of 
the fundamental principles of how things work so they can apply those 
principles in a similar situation.  For example, such a person may know enough 
about troubleshooting both gasoline and diesel engines and might have a better 
understanding of the underlying fundamentals of internal combustion engines to 
do a passable job troubleshooting something they have never seen before (air, 
fuel, timing).  There is a certain APPROACH to troubleshooting that transcends 
various fields.  Some naturally have a talent for it, others aren't so good at 
it.  Such people might be better in a multi-vendor network when there is a 
problem.  You can generally spot those people not by what they know, but by the 
quality of the questions they ask.  They generally know what they want to 
accomplish or what they are looking for, but they might want to know how that 
is done with this particular vendor's command set or how this particular vendor 
processes traffic.  

Some are natural designers, some are natural troubleshooters, some are natural 
documenters/support staff and they LIKE doing it.  It takes all of those skills.

One important thing to keep in mind, too, is that by identifying the skills and 
natural talents of your line staff, you yourself are being a value multiplier 
to your organization.  You are making best use of the resources that you have 
at your disposal and are improving the efficiency of the organization as an 
organic entity.  So this benefits everyone in the entire organization, 
including you.

Re: Common operational misconceptions

Mark Grigsby m...@pcinw.net writes:

 Speaking in the context of configuring an ipsec tunnel..

Once upon a time: 

Admin: We need Port 50 and Port 51 for the tunnel!
Me:You mean IP protocol 50 and 51?
Admin: It the same! You have no clue!

Jens
-- 
-
| Foelderichstr. 40   | 13595 Berlin, Germany| +49-151-18721264 |
| http://blog.quux.de | jabber: jensl...@guug.de | ---  | 
-

Re: Common operational misconceptions

2012-02-17 Thread John Osmon

On Fri, Feb 17, 2012 at 10:18:57AM -0500, Steve Clark wrote:
 I agree with this 100%.
 
 Having worked with many people over the last 40 years, the good trouble 
 shooters understood how things
 were suppose to work. This helps immeasurably in determining where to start 
 looking.

Don't forget that a lot of the best figure things out *while*
troubleshooting things they don't (yet) understand.

Give curious people good tools and interesting problems -- the rest will
sort itself out.

Re: Common operational misconceptions

Mathias Wolkert t...@netnod.se writes:

 Autoneg. The old timers that don't trust it after a few decades of
 decent code. Or those that lock one side and expect the other to adjust
 to that.

Autoneg is black magic. Doesn't work. You have manually configure duplex
and speed on one side 1!

SCNR

Jens
-- 
-
| Foelderichstr. 40   | 13595 Berlin, Germany| +49-151-18721264 |
| http://blog.quux.de | jabber: jensl...@guug.de | ---  | 
-

RE: Common operational misconceptions

 BTW, I am a school board member who votes 1:8 often on things But
 let me give you a perspective, one of the board members called Golf an
 Essential Life Skill.  Maybe, but how about balancing a checkbook...
 
 Ralph Brandt
 Communications Engineer
 HP Enterprise Services

One of the best courses I ever had was in 9th grade math class when Mr. Martin 
taught us how to do taxes.  And I mean even long forms with all the schedules 
and stuff for people who had investments and small sole proprietor businesses.  
It was a great practical math application, though it was mostly just 
arithmetic, some of the example cases were quite complex with estimated taxes, 
carrying forward losses from previous years, depreciation, etc.  It gave us 
some context in which we could understand why we might need to learn math in 
real life and made taxes less daunting when we got older.  Thanks, Mr. Martin!

Re: Common operational misconceptions

2012-02-17 Thread Sven Olaf Kamphuis



On Fri, 17 Feb 2012, Jens Link wrote:


Mathias Wolkert t...@netnod.se writes:


Autoneg. The old timers that don't trust it after a few decades of
decent code. Or those that lock one side and expect the other to adjust
to that.


you are referring to ehh *kuch* certain internet exchanges *kuch* ? :P

auto mdi/mii breaks teh internets! oeh noes! (not on any equipment we've 
owned for the past 15 years... funny how that works ;)




Autoneg is black magic. Doesn't work. You have manually configure duplex
and speed on one side 1!

SCNR

Jens
--
-
| Foelderichstr. 40   | 13595 Berlin, Germany| +49-151-18721264 |
| http://blog.quux.de | jabber: jensl...@guug.de | ---  |
-

Re: Common operational misconceptions

2012-02-17 Thread Jerry Jones


On Feb 17, 2012, at 11:33 AM, John Osmon wrote:

On Fri, Feb 17, 2012 at 10:18:57AM -0500, Steve Clark wrote:
 I agree with this 100%.
 
 Having worked with many people over the last 40 years, the good trouble 
 shooters understood how things
 were suppose to work. This helps immeasurably in determining where to start 
 looking.

Don't forget that a lot of the best figure things out *while*
troubleshooting things they don't (yet) understand.

Give curious people good tools and interesting problems -- the rest will
sort itself out.

Quote I have hear over the years


Problems are opportunities for learning - Just wish I was not doing so much 
learning some days

RE: Common operational misconceptions

 Long before there was a Grainger (and Home Depot) in every city, and
 you could get parts shipped overnight, one had to make do, and
 making do meant being able to figure things out to be able to git r
 done
 with what you had on hand, or could figure out.
 
 When working on my Grandfather's farm, I did not look for work to do
 (actually, I looked for ways not to do any work :-), but if the project
 required pulling out the oxy-acetylene torch to cut and weld something
 onto the tractor to get something done, that is what you had to do, so
 you did it.

Yep, when looking for troubleshooters, look for people that grew up/worked on a 
farm.

I absolutely agree.  They approach things from a completely different mindset.

Re: Common operational misconceptions

2012-02-17 Thread Jeff Kell

On 2/17/2012 12:00 PM, Gary Buhrmaster wrote:
 If the TV went on the blink (they all did then), you opened up the
 back, looked for fried components, and if one of the resistors was
 smoking, you soldered in a replacement. Or you took the tubes down to
 the local drugstore and tested them.

Wow...  would be handy if Radio Shack stocked router modules and blades,
and chassis to test your suspect ones?   :)

(Yes, remember the tube testers as well...)

Jeff

Re: Common operational misconceptions

2012-02-17 Thread Jared Mauch

I am grateful you have not used the hardware I have in the past 15 years.

I haven't seen anything recently not do it, but when interfacing with a 
customer who knows what old stuff they may be using. 

Jared Mauch

On Feb 17, 2012, at 12:41 PM, Sven Olaf Kamphuis s...@cb3rob.net wrote:

 auto mdi/mii breaks teh internets! oeh noes! (not on any equipment we've 
 owned for the past 15 years... funny how that works ;)

RE: Common operational misconceptions

 
 Wow...  would be handy if Radio Shack stocked router modules and
 blades,
 and chassis to test your suspect ones?   :)
 
 (Yes, remember the tube testers as well...)
 
 Jeff

Heh, that's been a notion I have had for a while.  Opening an all-night shop 
somewhere in Silicon Valley that sold patch cords, memory sticks, disk drives, 
maybe even common router blades, optics modules, fans, etc.  Sell it for a bit 
more margin than the going rate for the day shops and ONLY be open at 
night/early morning, say 7pm to 11am.  Maybe do some remote hands work, too.

Fry's wanted $55 for a 1 meter LC-LC multi-mode patch cord yesterday at the 
store on Arques in Sunnyvale.

Re: Common operational misconceptions

2012-02-17 Thread Leo Bicknell

In a message written on Fri, Feb 17, 2012 at 06:06:52PM +, George Bonser 
wrote:
 Heh, that's been a notion I have had for a while.  Opening an all-night shop 
 somewhere in Silicon Valley that sold patch cords, memory sticks, disk 
 drives, maybe even common router blades, optics modules, fans, etc.  Sell it 
 for a bit more margin than the going rate for the day shops and ONLY be 
 open at night/early morning, say 7pm to 11am.  Maybe do some remote hands 
 work, too.

I've repeatedly asked $BIG_COLO_PROVIDERS to offer a vending machine
in the lobby next to the one with sodas that sold Cat 5, Fiber,
SFP's, USB sticks, and so on.  Even at a moderate margin I suspect it
would be quite profitable to them, and quite welcomed by customers who
show up in the middle of the night when nothing is open and need parts.

-- 
   Leo Bicknell - bickn...@ufp.org - CCIE 3440
PGP keys at http://www.ufp.org/~bicknell/


pgpTvd8Rspgq8.pgp
Description: PGP signature

Re: Common operational misconceptions

2012-02-17 Thread Jay Ashworth

- Original Message -
 From: Mike Andrews mi...@mikea.ath.cx

   Which is a common transport problem I often see, Our
   configuration looks right, it must be on your end.
 
  If i had a dollar for everytime i've heard that from a telco, i'd be
  a rich man...
 
 That and I'm getting a good ping response here while I've got the
 cable at my end unplugged from the equipment.

The problem's leaving here fine!

Cheers,
-- jra
-- 
Jay R. Ashworth  Baylink   j...@baylink.com
Designer The Things I Think   RFC 2100
Ashworth  Associates http://baylink.pitas.com 2000 Land Rover DII
St Petersburg FL USA  http://photo.imageinc.us +1 727 647 1274

RE: Common operational misconceptions

To find counterfeit they teach you what good money looks like.  When you
are looking at a sniffer trace you are generally looking for something
that is not right. 



Ralph Brandt
Communications Engineer
HP Enterprise Services
Telephone +1 717.506.0802
FAX +1 717.506.4358
Email ralph.bra...@pateam.com
5095 Ritter Rd
Mechanicsburg PA 17055

-Original Message-
From: Scott Helms [mailto:khe...@ispalliance.net] 
Sent: Friday, February 17, 2012 11:24 AM
To: nanog@nanog.org
Subject: Re: Common operational misconceptions

On 2/17/2012 10:18 AM, Steve Clark wrote:
 I agree with this 100%.

 Having worked with many people over the last 40 years, the good 
 trouble shooters understood how things
 were suppose to work. This helps immeasurably in determining where to 
 start looking.


This is dead on and why I always start classes with a refresher on the 
OSI model.  While the model isn't perfect it lets technicians and 
engineers construct a reasonable model of how things *ought* to be 
working.  While you certainly will run into devices that bend or even 
break the rules (sometimes for good reasons) its much easier to 
understand the exceptions if you know the normal operation for a 
repeater, bridge, or router.

-- 
Scott Helms
Vice President of Technology
ISP Alliance, Inc. DBA ZCorum
(678) 507-5000

http://twitter.com/kscotthelms

Re: Common operational misconceptions

Leo Bicknell bickn...@ufp.org writes:

 I've repeatedly asked $BIG_COLO_PROVIDERS to offer a vending machine
 in the lobby next to the one with sodas that sold Cat 5, Fiber,
 SFP's, USB sticks, and so on.  

Hmm.

http://gearomat.com/

Jens
-- 
-
| Foelderichstr. 40   | 13595 Berlin, Germany| +49-151-18721264 |
| http://blog.quux.de | jabber: jensl...@guug.de | ---  | 
-

Re: Common operational misconceptions

2012-02-17 Thread Paul Graydon


On 02/17/2012 04:29 AM, Leo Bicknell wrote:

In a message written on Thu, Feb 16, 2012 at 08:50:11PM -1000, Paul Graydon 
wrote:

At the same time, it's shocking how many network people I come across
with no real grasp of even what OSI means by each layer, even if it's
only in theory.  Just having a grasp of that makes all the world of
difference when it comes to troubleshooting.  Start at layer 1 and work
upwards (unless you're able to make appropriate intuitive leaps.) Is it
physically connected? Are the link lights flashing? Can traffic route to
it, etc. etc.

I wouldn't call it a misconception, but I want to echo Paul's
comment.  I would venture over 90% of the engineers I work with
have no idea how to troubleshoot properly.  Thinking back to my own
education, I don't recall anyone in highschool or college attempting
to teach troubleshooting skills.  Most classes teach you how to
build things, not deal with them when they are broken.
The Cisco CCNA syllabus used to emphasise the layer 1-7 approach to 
troubleshooting.  Not sure if they still do, or if trainers even bother 
to mention it (mine did back when I did it several years ago)



The basic skills are probably obvious to someone who might design
course material if they sat down and thought about how to teach
troubleshooting.  However, there is one area that may not be obvious.
There's also a group management problem.  Many times troubleshooting
is done with multiple folks on the phone (say, customer, ISP and
vendor).  Not only do you have to know how to troubleshoot, but how
to get everyone on the same page so every possible cause isn't
tested 3 times.
Never trust what you can't prove yourself, that includes vendors and 
customers.  Every now and then I forget this and find hours later that 
I've wasted a whole bunch of time because I trusted when someone said 
something that it actually was the case.  It's really often better to 
test something a third time even if Vendor and Customer tell you 
something is a particular way.




I think all college level courses should include a break/fix
exercise/module after learning how to build something, and much of that
should be done in a group enviornment.

Definitely.  I've learnt more in my time from breaking things than I've 
ever learnt setting them up; however the education system is focused on 
breadth of knowledge, not depth.  Students are expected to be able to 
regurgitate ridiculous amounts of facts and figures, so that they pass 
standardised tests, not understand how to actually use them.


Paul

Re: Common operational misconceptions


On Feb 17, 2012, at 6:52 AM, -Hammer- wrote:

 Let me simplify that. If you are over 35 you know how to troubleshoot.
 

Is this a statement or something to be added to the list of misconceptions that 
are commonplace out there?

Not trying to be flip... Honestly asking the question. I can see it going 
either way, frankly. ;-)

Owen

RE: Common operational misconceptions

2012-02-17 Thread Kenneth M. Chipps Ph.D.

Exactly right. They have some much information floating around in their
heads many of them cannot fit it together. But once they get on the job, all
of those little synapses rapidly connect, and then the light comes on.

Higher education is just like drivers education. You did not learn to drive
in drivers education. You learned how to drive by driving. Higher education
gives you the foundation on which to learn.

-Original Message-
From: Paul Graydon [mailto:p...@paulgraydon.co.uk] 
Sent: Friday, February 17, 2012 12:33 PM
To: nanog@nanog.org
Subject: Re: Common operational misconceptions

On 02/17/2012 04:29 AM, Leo Bicknell wrote:
 In a message written on Thu, Feb 16, 2012 at 08:50:11PM -1000, Paul
Graydon wrote:
 At the same time, it's shocking how many network people I come across 
 with no real grasp of even what OSI means by each layer, even if it's 
 only in theory.  Just having a grasp of that makes all the world of 
 difference when it comes to troubleshooting.  Start at layer 1 and 
 work upwards (unless you're able to make appropriate intuitive 
 leaps.) Is it physically connected? Are the link lights flashing? Can 
 traffic route to it, etc. etc.
 I wouldn't call it a misconception, but I want to echo Paul's 
 comment.  I would venture over 90% of the engineers I work with have 
 no idea how to troubleshoot properly.  Thinking back to my own 
 education, I don't recall anyone in highschool or college attempting 
 to teach troubleshooting skills.  Most classes teach you how to build 
 things, not deal with them when they are broken.
The Cisco CCNA syllabus used to emphasise the layer 1-7 approach to
troubleshooting.  Not sure if they still do, or if trainers even bother to
mention it (mine did back when I did it several years ago)

 The basic skills are probably obvious to someone who might design 
 course material if they sat down and thought about how to teach 
 troubleshooting.  However, there is one area that may not be obvious.
 There's also a group management problem.  Many times troubleshooting 
 is done with multiple folks on the phone (say, customer, ISP and 
 vendor).  Not only do you have to know how to troubleshoot, but how to 
 get everyone on the same page so every possible cause isn't tested 3 
 times.
Never trust what you can't prove yourself, that includes vendors and
customers.  Every now and then I forget this and find hours later that I've
wasted a whole bunch of time because I trusted when someone said something
that it actually was the case.  It's really often better to test something a
third time even if Vendor and Customer tell you something is a particular
way.


 I think all college level courses should include a break/fix
 exercise/module after learning how to build something, and much of 
 that should be done in a group enviornment.

Definitely.  I've learnt more in my time from breaking things than I've ever
learnt setting them up; however the education system is focused on breadth
of knowledge, not depth.  Students are expected to be able to regurgitate
ridiculous amounts of facts and figures, so that they pass standardised
tests, not understand how to actually use them.

Paul

Re: Common operational misconceptions


On Feb 17, 2012, at 7:20 AM, David Barak wrote:

 From: Owen DeLong o...@delong.com
 
 Sigh... NAT is a horrible hack that served us all too well in address 
 conservation. Beyond that, it is merely a source of pain.
 
 I understand why you say that - NAT did yeoman's work in address 
 conservation.  However, it also enabled (yes, really) lots of topologies and 
 approaches which are *not* designed upon the end-to-end model.  Some of these 
 approaches have found their way into business proceses.  
 
Yes... An unfortunate and very damaging side effect to be sure.

 An argument you and others have made many times boils down to but if we 
 never had NAT, think how much better it would be!  
 
 To this, the response so what? is not unreasonable - organizations which 
 have built up processes and products around the non-end-to-end model may or 
 may not see a benefit in changing their ways.  Asserting that there is 
 something wrong with existing, succesful business practices is not, by 
 itself, compelling.  
 

People who make money selling weapons don't necessarily see a benefit to peace. 
People who avoid the high costs of toxic waste disposal by putting it into 
ground water don't necessarily see a benefit to having an EPA or laws that 
prohibit doing so.

If you're not party to the war and you're not one of the people whose water 
supply is affected by the toxins flowing into it, then, the response so what? 
may seem reasonable from your perspective.

NAT is much the same way. It is a form of toxic pollutant that has damaging 
effects outside of the business that has chosen to deploy NAT. At best, it 
started out as a necessary evil for address conservation. Dependence on it 
beyond that seems to me to be akin to a form of drug addiction.

 While you and I may find this type of packet header manipulation distasteful, 
 there's lots of organizations for which it's normal operations.  The more NAT 
 for v6 gets fought, the more folks will fight to preserve it.  Time could be 
 better spent demonstrating why NAT isn't needed in X or Y use case, and 
 providing configuration snippets / assistance for non-NAT-based solutions to 
 those various groups.
 

And I do exactly that when presented with actual use cases where people believe 
NAT is needed.

You can find several instances of my having done that in previous NANOG threads.

Owen

Re: Common operational misconceptions

Now, come on... If you're in the 40-50 range, you should have put octal before 
hex. :p

Owen
(Who grew up on a PDP-11 in his high-school and still remembers 1777300 and its 
significance to anyone who has used a larger PDP system)

On Feb 17, 2012, at 7:51 AM, -Hammer- wrote:

 Mario,
I was kinda having fun and kinda not. My point is that the 40-50 year olds 
 that were doing this 30 years ago grew up understanding things in order. 
 Bits. Bytes. KiloBits. KiloBytes. (Some folks still get those confused). Hex. 
 etc. Move on to the OSI model and it's the same thing. Voltage. Amplitude. 
 Binary. etc. I think that this generation that I'm referring to is a great 
 generation because we were at the beginning of the Internet blooming. There 
 are folks on this forum that go back further. Into DARPA. Before DARPA was 
 just chapter 1 one every single Cisco Press book. They have a unique 
 understanding of the layers. I had that understanding in my 20s. The 
 technology is so complicated these days that many folks miss those 
 fundamentals and go right into VSS on the 6500s or MPLS over Juniper. In the 
 end, it all comes in time.
 
 -Hammer-
 
 I was a normal American nerd
 -Jack Herer
 
 
 
 On 2/17/2012 9:12 AM, Mario Eirea wrote:
 Well, I will argue this. I think the important factor in any troubleshooting 
 is having a real understanding of how the system works. That is, how 
 different things interact with each others to achieve a specific goal. The 
 biggest problem I see is that many people understand understand the 
 individual parts but when it comes to understanding the system as a whole 
 they fall miserably short.
 
 A short example, probably not the best but the one that comes to mind right 
 now:
 
 Someone replaces a device on the network with a new one. They give it the 
 same IP address as the old device. They don't understand why the router cant 
 communicate with it at first and then starts working. The people 
 understand ARP, but cant correlate one event with another.
 
 I guess if your 35 you have seen this at least once and can fix it. But what 
 happens if you have never seen this problem or a related one? At this point 
 your going to have to really troubleshoot, not just go on experience.
 
 Mario Eirea
 
 From: -Hammer- [bhmc...@gmail.com]
 Sent: Friday, February 17, 2012 9:52 AM
 To: nanog@nanog.org
 Subject: Re: Common operational misconceptions
 
 Let me simplify that. If you are over 35 you know how to troubleshoot.
 
 Yes, I'm going to get flamed. Yes, there are exceptions in both directions.
 
 -Hammer-
 
 I was a normal American nerd
 -Jack Herer
 
 
 
 On 2/17/2012 8:29 AM, Leo Bicknell wrote:
 In a message written on Thu, Feb 16, 2012 at 08:50:11PM -1000, Paul Graydon 
 wrote:
 At the same time, it's shocking how many network people I come across
 with no real grasp of even what OSI means by each layer, even if it's
 only in theory.  Just having a grasp of that makes all the world of
 difference when it comes to troubleshooting.  Start at layer 1 and work
 upwards (unless you're able to make appropriate intuitive leaps.) Is it
 physically connected? Are the link lights flashing? Can traffic route to
 it, etc. etc.
 I wouldn't call it a misconception, but I want to echo Paul's
 comment.  I would venture over 90% of the engineers I work with
 have no idea how to troubleshoot properly.  Thinking back to my own
 education, I don't recall anyone in highschool or college attempting
 to teach troubleshooting skills.  Most classes teach you how to
 build things, not deal with them when they are broken.
 
 The basic skills are probably obvious to someone who might design
 course material if they sat down and thought about how to teach
 troubleshooting.  However, there is one area that may not be obvious.
 There's also a group management problem.  Many times troubleshooting
 is done with multiple folks on the phone (say, customer, ISP and
 vendor).  Not only do you have to know how to troubleshoot, but how
 to get everyone on the same page so every possible cause isn't
 tested 3 times.
 
 I think all college level courses should include a break/fix
 exercise/module after learning how to build something, and much of that
 should be done in a group enviornment.

Re: Common operational misconceptions


This reminds me of what I think is the biggest root misconception of the 20th 
and 21st centuries:

Rapid step-by-step training can replace conceptual education on the 
fundamentals.

In other words, we have moved from the old-school of teaching people why things 
work and how they work to a newer school of teaching people how to complete 
specific tasks. This has had the following negative effects, IMHO:

1.  When the only tool you have is a hammer, you try to mold every problem 
into a nail.
2.  When you only know a procedure for doing something and don't understand 
the fundamentals
of why X is supposed to occur at step Y, then when you get result A 
instead of X, your only options
are to either continue to step Z and hope everything turns out OK, or, 
go back to an earlier step
and hope everything works this time.
3.  Troubleshooting skills are limited to knowing the number of the 
vendor's help desk.

I once worked with a director of QA that epitomized this. It was a small 
company, so, as director, he was directly responsible for most of the tasks in 
the QA lab. He was meticulous in following directions which was a good thing. 
However, when he reached a step where he did not get the expected result, he 
was limited to telling the engineers that the test failed at step X and would 
not make any effort to identify or resolve the problem and would literally 
block the entire QA process waiting for engineering to resolve the issue before 
he would continue testing. Worse, he would not test independent pieces of the 
system in parallel, so, when he blocked on one system failing, he wouldn't test 
the others, either. Further investigation revealed that this was because he 
didn't actually know which systems were or were not dependent on each other. He 
was so completely immersed in the procedural school of thought that he was 
literally unwilling to accept conceptual knowledge or develop an understanding 
of the theory and principles of operation of any of the systems.

Owen

On Feb 17, 2012, at 8:13 AM, Mario Eirea wrote:

 I definitely understand and agree with what you saying. Actually, most my 
 friends are over 50 years old... I do agree with you on the generational 
 statement. My argument was that many people over 35 have no idea what they 
 are doing, and some under 35 do know what they are doing. On thing is for 
 sure, experience goes a long way. The importance is knowing the fundamentals 
 and putting it all together (a skill that has been lost in recent times)
 
 I have a lot to say about the current generation of people growing up in this 
 country, but that's a whole other thread in a whole other list. :-)
 
 Mario Eirea

Re: Common operational misconceptions

2012-02-17 Thread Valdis . Kletnieks

On Fri, 17 Feb 2012 10:49:13 PST, Owen DeLong said:
 Now, come on... If you're in the 40-50 range, you should have put octal
 before hex. :p

IBM S/360 definitely preferred hex.  And EBCDIC.



pgpJXJPC98gau.pgp
Description: PGP signature

Re: Common operational misconceptions

2012-02-17 Thread Robert Bonomi

 From nanog-bounces+bonomi=mail.r-bonomi@nanog.org  Fri Feb 17 13:11:28 
 2012
 To: Owen DeLong o...@delong.com
 Subject: Re: Common operational misconceptions
 From: valdis.kletni...@vt.edu
 Date: Fri, 17 Feb 2012 14:04:45 -0500
 Cc: nanog@nanog.org nanog@nanog.org

 On Fri, 17 Feb 2012 10:49:13 PST, Owen DeLong said:
  Now, come on... If you're in the 40-50 range, you should have put octal
  before hex. :p

 IBM S/360 definitely preferred hex.  And EBCDIC.

And the  _real_ number crunchers used ones-complement arithmetic.  Which led to
to mahines that couldn't add -- they did addition by 'complement and subtract'.

Re: Common operational misconceptions

I give I give. I should have. But I left a bunch of stuff out and the 
folks that I'm referring to know it. One of the fun things I do with 
network guys is ask them about canonical bit ordering across routers. 
Always causes the room to go quiet. Them someone of the appropriate age 
group will speak up after he's done laughing.


The best thing I have ever figured out to tell less experienced (I 
didn't say younger) folks is to start at the bottom of the stack and 
work your way up. That's the way many of us troubleshoot. Is the cable 
on the floor? That's bad. If not, is the ARP entry incomplete? That's 
bad. If not, is the route missing? That's bad. If not, can you reach the 
route? Try this radical command that was invented by Steve Jobs while 
working on his first IPhone (They won't know who Vint Cerf or anyone 
else is and by using Steves name they will trust you)(I run Android):

telnet 1.2.3.4 1433
What? It answered? So the SQL service is running? Then it ain't the 
network dude
So many times people don't pick up on that. But when they do, it's like 
a light bulb went off and they see the world differently. Like 
subnetting


-Hammer-

I was a normal American nerd
-Jack Herer



On 2/17/2012 12:49 PM, Owen DeLong wrote:

Now, come on... If you're in the 40-50 range, you should have put octal before 
hex. :p

Owen
(Who grew up on a PDP-11 in his high-school and still remembers 1777300 and its 
significance to anyone who has used a larger PDP system)

On Feb 17, 2012, at 7:51 AM, -Hammer- wrote:


Mario,
I was kinda having fun and kinda not. My point is that the 40-50 year olds 
that were doing this 30 years ago grew up understanding things in order. Bits. 
Bytes. KiloBits. KiloBytes. (Some folks still get those confused). Hex. etc. 
Move on to the OSI model and it's the same thing. Voltage. Amplitude. Binary. 
etc. I think that this generation that I'm referring to is a great generation 
because we were at the beginning of the Internet blooming. There are folks on 
this forum that go back further. Into DARPA. Before DARPA was just chapter 1 
one every single Cisco Press book. They have a unique understanding of the 
layers. I had that understanding in my 20s. The technology is so complicated 
these days that many folks miss those fundamentals and go right into VSS on the 
6500s or MPLS over Juniper. In the end, it all comes in time.

-Hammer-

I was a normal American nerd
-Jack Herer



On 2/17/2012 9:12 AM, Mario Eirea wrote:

Well, I will argue this. I think the important factor in any troubleshooting is 
having a real understanding of how the system works. That is, how different 
things interact with each others to achieve a specific goal. The biggest 
problem I see is that many people understand understand the individual parts 
but when it comes to understanding the system as a whole they fall miserably 
short.

A short example, probably not the best but the one that comes to mind right now:

Someone replaces a device on the network with a new one. They give it the same IP address 
as the old device. They don't understand why the router cant communicate with it at first 
and then starts working. The people understand ARP, but cant correlate one 
event with another.

I guess if your 35 you have seen this at least once and can fix it. But what 
happens if you have never seen this problem or a related one? At this point 
your going to have to really troubleshoot, not just go on experience.

Mario Eirea

From: -Hammer- [bhmc...@gmail.com]
Sent: Friday, February 17, 2012 9:52 AM
To: nanog@nanog.org
Subject: Re: Common operational misconceptions

Let me simplify that. If you are over 35 you know how to troubleshoot.

Yes, I'm going to get flamed. Yes, there are exceptions in both directions.

-Hammer-

I was a normal American nerd
-Jack Herer



On 2/17/2012 8:29 AM, Leo Bicknell wrote:

In a message written on Thu, Feb 16, 2012 at 08:50:11PM -1000, Paul Graydon 
wrote:

At the same time, it's shocking how many network people I come across
with no real grasp of even what OSI means by each layer, even if it's
only in theory.  Just having a grasp of that makes all the world of
difference when it comes to troubleshooting.  Start at layer 1 and work
upwards (unless you're able to make appropriate intuitive leaps.) Is it
physically connected? Are the link lights flashing? Can traffic route to
it, etc. etc.

I wouldn't call it a misconception, but I want to echo Paul's
comment.  I would venture over 90% of the engineers I work with
have no idea how to troubleshoot properly.  Thinking back to my own
education, I don't recall anyone in highschool or college attempting
to teach troubleshooting skills.  Most classes teach you how to
build things, not deal with them when they are broken.

The basic skills are probably obvious to someone who might design
course material if they sat down and thought about how to teach
troubleshooting.  However, there is one area

Re: Common operational misconceptions


Well put and great example Owen.

-Hammer-

I was a normal American nerd
-Jack Herer



On 2/17/2012 12:59 PM, Owen DeLong wrote:

This reminds me of what I think is the biggest root misconception of the 20th 
and 21st centuries:

Rapid step-by-step training can replace conceptual education on the 
fundamentals.

In other words, we have moved from the old-school of teaching people why things 
work and how they work to a newer school of teaching people how to complete 
specific tasks. This has had the following negative effects, IMHO:

1.  When the only tool you have is a hammer, you try to mold every problem 
into a nail.
2.  When you only know a procedure for doing something and don't understand 
the fundamentals
of why X is supposed to occur at step Y, then when you get result A 
instead of X, your only options
are to either continue to step Z and hope everything turns out OK, or, 
go back to an earlier step
and hope everything works this time.
3.  Troubleshooting skills are limited to knowing the number of the 
vendor's help desk.

I once worked with a director of QA that epitomized this. It was a small 
company, so, as director, he was directly responsible for most of the tasks in 
the QA lab. He was meticulous in following directions which was a good thing. 
However, when he reached a step where he did not get the expected result, he 
was limited to telling the engineers that the test failed at step X and would 
not make any effort to identify or resolve the problem and would literally 
block the entire QA process waiting for engineering to resolve the issue before 
he would continue testing. Worse, he would not test independent pieces of the 
system in parallel, so, when he blocked on one system failing, he wouldn't test 
the others, either. Further investigation revealed that this was because he 
didn't actually know which systems were or were not dependent on each other. He 
was so completely immersed in the procedural school of thought that he was 
literally unwilling to accept conceptual knowledge or develop an understanding 
of the theory and principles of operation of any of the systems.

Owen

On Feb 17, 2012, at 8:13 AM, Mario Eirea wrote:


I definitely understand and agree with what you saying. Actually, most my 
friends are over 50 years old... I do agree with you on the generational 
statement. My argument was that many people over 35 have no idea what they are 
doing, and some under 35 do know what they are doing. On thing is for sure, 
experience goes a long way. The importance is knowing the fundamentals and 
putting it all together (a skill that has been lost in recent times)

I have a lot to say about the current generation of people growing up in this 
country, but that's a whole other thread in a whole other list. :-)

Mario Eirea

Re: Common operational misconceptions

2012-02-17 Thread Scott Weeks




I find a lot of new folks have a hard time with the difference in 
port numbers and protocol numbers.  

I just went through this with a CCsomething-more-than NA, but with 
virtually no hands-on experience a few minutes ago.  Very disturbing 
when a person can take the higher level tests, but still can't 
understand the basics.  All they do is use the certification testing 
programs and memorize everything they can.  :-(


grrr
scott

Re: Common operational misconceptions

Owen DeLong o...@delong.com writes:

 1.When the only tool you have is a hammer, you try to mold every problem 
 into a nail.

Ack.

 2.When you only know a procedure for doing something and don't understand 
 the fundamentals
   of why X is supposed to occur at step Y, then when you get result A 
 instead of X, your only options
   are to either continue to step Z and hope everything turns out OK, or, 
 go back to an earlier step
   and hope everything works this time.

But procedures are important. How else can you get enough exper^Widiots
working for little money. Big Macs vs. The Naked Chef is great:

http://www.joelonsoftware.com/articles/fog24.html


 3.Troubleshooting skills are limited to knowing the number of the 
 vendor's help desk.

There are no problems! Can't be. And if there are they hire external
experts. BTDT. Those are well paid jobs.

Jens
-- 
-
| Foelderichstr. 40   | 13595 Berlin, Germany| +49-151-18721264 |
| http://blog.quux.de | jabber: jensl...@guug.de | ---  | 
-

Re: Common operational misconceptions

2012-02-17 Thread Tom Perrine

-BEGIN PGP SIGNED MESSAGE-
Hash: RIPEMD160

On 2/17/12 11:04 AM, valdis.kletni...@vt.edu wrote:
 On Fri, 17 Feb 2012 10:49:13 PST, Owen DeLong said:
 Now, come on... If you're in the 40-50 range, you should have put octal 
 before hex. :p
 
 IBM S/360 definitely preferred hex.  And EBCDIC.
 

GCOS - 36 bits and Octal and BCD (ASCII added later)
DEC 10 and 20 - 36 bits and Octal
PDP-8 - Octal

- --tep

-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.10 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEAREDAAYFAk8+qEEACgkQPu53fhcIEuBr9gCfU46kCDPbmgSVQGAw5nnOsrNO
hJ4AnRpr4Ig46x5JZlcK+kL43JGFcbCS
=cSWb
-END PGP SIGNATURE-

RE: Common operational misconceptions

  3.  Troubleshooting skills are limited to knowing the number of the
 vendor's help desk.
 
 There are no problems! Can't be. And if there are they hire external
 experts. BTDT. Those are well paid jobs.

I see that a lot and there is often an organizational reason for it.  If a tech 
says I have a ticket open with the vendor and provides the ticket and status 
updates on a regular basis, he's covered as far as the people higher up in the 
organization are concerned.  If the C(X)O wants to know what's going on, the 
manager can shift the focus to the vendor and say we are waiting for a fix from 
them.

A tech trying to troubleshoot it and fix it themselves is going to be hounded 
every five minutes for status updates and won't be able to get any work done 
because every five minutes (I kid you not, I have worked where that is a 
requirement) he has to pull his head out of what he is doing and answer a bunch 
of questions from the PHBs.  And you always get how long is it going to be 
and you want to say 10 minutes longer than it would have been if you hadn't 
interrupted me but you bite your tongue.

RE: Common operational misconceptions

 A tech trying to troubleshoot it and fix it themselves is going to be
 hounded every five minutes for status updates and won't be able to get
 any work done because every five minutes (I kid you not, I have worked
 where that is a requirement) he has to pull his head out of what he is
 doing and answer a bunch of questions from the PHBs.  And you always
 get how long is it going to be and you want to say 10 minutes longer
 than it would have been if you hadn't interrupted me but you bite your
 tongue.
 

Though the flip side of that is that if someone has been neck deep in a problem 
for hours, you should force them to take a break, go get a drink of water, step 
outside for fresh air or a smoke if they do, or just talk to a colleague for a 
moment and review the problem.  In my case, the stepping away for a few minutes 
has sometimes allowed the answer to the problem to suddenly snap into focus or 
in the process of describing it to someone else the forming of the thoughts to 
describe it often allows a new aspect of the problem to become visible that you 
hadn't noticed before.

Re: Common operational misconceptions

2012-02-17 Thread Ray Soucy

As someone who was born in 1984 I respectfully disagree. ;-)




On Fri, Feb 17, 2012 at 9:52 AM, -Hammer- bhmc...@gmail.com wrote:
 Let me simplify that. If you are over 35 you know how to troubleshoot.

 Yes, I'm going to get flamed. Yes, there are exceptions in both directions.


 -Hammer-

 I was a normal American nerd
 -Jack Herer



 On 2/17/2012 8:29 AM, Leo Bicknell wrote:

 In a message written on Thu, Feb 16, 2012 at 08:50:11PM -1000, Paul
 Graydon wrote:

 At the same time, it's shocking how many network people I come across
 with no real grasp of even what OSI means by each layer, even if it's
 only in theory.  Just having a grasp of that makes all the world of
 difference when it comes to troubleshooting.  Start at layer 1 and work
 upwards (unless you're able to make appropriate intuitive leaps.) Is it
 physically connected? Are the link lights flashing? Can traffic route to
 it, etc. etc.

 I wouldn't call it a misconception, but I want to echo Paul's
 comment.  I would venture over 90% of the engineers I work with
 have no idea how to troubleshoot properly.  Thinking back to my own
 education, I don't recall anyone in highschool or college attempting
 to teach troubleshooting skills.  Most classes teach you how to
 build things, not deal with them when they are broken.

 The basic skills are probably obvious to someone who might design
 course material if they sat down and thought about how to teach
 troubleshooting.  However, there is one area that may not be obvious.
 There's also a group management problem.  Many times troubleshooting
 is done with multiple folks on the phone (say, customer, ISP and
 vendor).  Not only do you have to know how to troubleshoot, but how
 to get everyone on the same page so every possible cause isn't
 tested 3 times.

 I think all college level courses should include a break/fix
 exercise/module after learning how to build something, and much of that
 should be done in a group enviornment.





-- 
Ray Soucy

Epic Communications Specialist

Phone: +1 (207) 561-3526

Networkmaine, a Unit of the University of Maine System
http://www.networkmaine.net/

Re: Common operational misconceptions

2012-02-17 Thread Randy Bush

 GCOS - 36 bits and Octal and BCD (ASCII added later)
 DEC 10 and 20 - 36 bits and Octal
 PDP-8 - Octal

704 - was i think 36-bit but the mind fades
704x/709x - 36 bit
1401 - variable word length with BCD+zone-bit encoding per char

randy

Re: Common operational misconceptions

I have found that the best solution to persistent hounding goes about like this:

Sir, I'm doing everything I can to resolve the problem as quickly as possible. 
However, I can focus on giving you status updates every 5 minutes, or, I can 
focus on resolving the problem. I cannot do both. which would you prefer?

As to the internal vs. external question, most organizations I've worked for 
have just wanted the problem solved. They didn't so much care whether I was 
taking a lot of time to solve it or the vendor was taking a lot of time to 
respond to me, they wanted the problem fixed and I was the one they could fire.

As such, it was in my interest to usually learn most of the systems better than 
the tech support folk at the other end of the phone.

YMMV

Owen

On Feb 17, 2012, at 11:35 AM, George Bonser wrote:

 A tech trying to troubleshoot it and fix it themselves is going to be
 hounded every five minutes for status updates and won't be able to get
 any work done because every five minutes (I kid you not, I have worked
 where that is a requirement) he has to pull his head out of what he is
 doing and answer a bunch of questions from the PHBs.  And you always
 get how long is it going to be and you want to say 10 minutes longer
 than it would have been if you hadn't interrupted me but you bite your
 tongue.
 
 
 Though the flip side of that is that if someone has been neck deep in a 
 problem for hours, you should force them to take a break, go get a drink of 
 water, step outside for fresh air or a smoke if they do, or just talk to a 
 colleague for a moment and review the problem.  In my case, the stepping away 
 for a few minutes has sometimes allowed the answer to the problem to suddenly 
 snap into focus or in the process of describing it to someone else the 
 forming of the thoughts to describe it often allows a new aspect of the 
 problem to become visible that you hadn't noticed before.

Re: Common operational misconceptions

2012-02-17 Thread Todd Snyder

as a 33 year old, I'm looking forward to hitting 35 so I can finally
understand what you guys are talking about!  Will I get some sort of glow
or achievement?

think I'll get a raise when I can add 'troubleshooting' to my resume? :)

On Fri, Feb 17, 2012 at 2:42 PM, Ray Soucy r...@maine.edu wrote:

 As someone who was born in 1984 I respectfully disagree. ;-)




 On Fri, Feb 17, 2012 at 9:52 AM, -Hammer- bhmc...@gmail.com wrote:
  Let me simplify that. If you are over 35 you know how to troubleshoot.
 
  Yes, I'm going to get flamed. Yes, there are exceptions in both
 directions.
 
 
  -Hammer-
 
  I was a normal American nerd
  -Jack Herer
 
 
 
  On 2/17/2012 8:29 AM, Leo Bicknell wrote:
 
  In a message written on Thu, Feb 16, 2012 at 08:50:11PM -1000, Paul
  Graydon wrote:
 
  At the same time, it's shocking how many network people I come across
  with no real grasp of even what OSI means by each layer, even if it's
  only in theory.  Just having a grasp of that makes all the world of
  difference when it comes to troubleshooting.  Start at layer 1 and work
  upwards (unless you're able to make appropriate intuitive leaps.) Is it
  physically connected? Are the link lights flashing? Can traffic route
 to
  it, etc. etc.
 
  I wouldn't call it a misconception, but I want to echo Paul's
  comment.  I would venture over 90% of the engineers I work with
  have no idea how to troubleshoot properly.  Thinking back to my own
  education, I don't recall anyone in highschool or college attempting
  to teach troubleshooting skills.  Most classes teach you how to
  build things, not deal with them when they are broken.
 
  The basic skills are probably obvious to someone who might design
  course material if they sat down and thought about how to teach
  troubleshooting.  However, there is one area that may not be obvious.
  There's also a group management problem.  Many times troubleshooting
  is done with multiple folks on the phone (say, customer, ISP and
  vendor).  Not only do you have to know how to troubleshoot, but how
  to get everyone on the same page so every possible cause isn't
  tested 3 times.
 
  I think all college level courses should include a break/fix
  exercise/module after learning how to build something, and much of that
  should be done in a group enviornment.
 
 



 --
 Ray Soucy

 Epic Communications Specialist

 Phone: +1 (207) 561-3526

 Networkmaine, a Unit of the University of Maine System
 http://www.networkmaine.net/

RE: Common operational misconceptions