[c-nsp] SegV exception On 7206 LNS

[Dominic]

Hi Everyone:

My Cisco 7206VXR with NPE-G2 runs as an LNS terminating PPPOE sessions. It also 
terminates a DS3 used for data T1s. About once a week or so,  a SegV exception 
happens, and the router resets itself. I have no idear why. There seems to be 
no pattern to it, and I can't figure out for the life of me why this is 
happening.

Does anyone have an idea about what I should be looking at? Usage is still 
pretty decent, averaging at abotu 30%. 


show ver
...
System returned to ROM by error - a SegV exception, PC 0x73FE74 at 14:34:13 edt 
Tue Sep 21 2010
System restarted at 14:36:08 edt Tue Sep 21 2010
System image file is "disk2:c7200p-spservicesk9-mz.150-1.M.bin"
...

show stacks
...
 14:34:06 edt Tue Sep 21 2010: Unexpected exception to CPU: vector 300, PC = 
0x73FE74  , LR = 0x73FDA4
-Traceback= 0x73FE74z 0x73FDA4z 0x196B7C8z 0x11E74z 0x6883D0z 0x68BDB0z 
0x6837B0z 0x66E140z 0x66BC08z 0x4D9z 0x66E784z 0x66BCC4z 0x1939F4Cz 
0x65BCE50z 0x1939B58z 0x1939F38z
..



Dominic
___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: [c-nsp] Operational impact of switching from ingress to egress replication mode

[Peter Rathlev]

On Tue, 2010-09-21 at 22:12 +0100, Phil Mayers wrote:
> 2. Use CoPP for everything else; DO NOT use the glean or cef receive 
> limiter

I'm confused here: Why not use the glean limiter? As I understand it you
can simplify the CoPP maps a lot. (Non-IP traffic like IS-IS still being
a special case of course.)

What am I missing? Is it because you want to catch special cases of
gleaning earlier in the CoPP and differentiate rates?

-- 
Peter


___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: [c-nsp] Operational impact of switching from ingress to egress replication mode

[Benjamin Lovell]

John,

If you are a first hop mcast router then connected could matter to you but as a 
transit box the two you would care about the most are fib-miss(i.e. no hardware 
entry found so punt) and non-RPF(you came in on the wrong interface). Setting 
these two to a low number (100's of pps) should provide the CPU some protection 
for the short period needed to reprogram hardware when you change the 
replication mode. 

-Ben


On Sep 21, 2010, at 5:19 PM, John Neiberger wrote:

> I see the "mls rate-limit multicast ipv4 connected" command, for
> directly connected sources, but is there a command that would apply to
> traffic going through box?
> 
> We have very little unicast traffic but a whole bunch of multicast
> traffic, some of which is directly connected but much of it is simply
> passing through.
> 
> Thanks again for all your help, I appreciate your time.
> 
> On Tue, Sep 21, 2010 at 3:12 PM, Phil Mayers  wrote:
>> On 09/21/2010 09:58 PM, John Neiberger wrote:
>>> 
>>> Sorry.  I also meant to say it's a Sup 720-3BXL. Based on what I can
>>> see on CCO, that thing can forward 400 Mpps of ipv4 traffic. Does that
>>> mean that I can set a rate limit of, say, 300 Mpps and somewhat guard
>>> the CPU from meltdown for a few moments?
>> 
>> I wish! 300Mpps hitting the CPU of a sup720 will kill it stone dead. It has
>> (two) 600MHz CPUs, and they will not survive such a load.
>> 
>> There's lots of info in this in the archives, but in brief: the sup720/pfc3
>> architecture forwards most packets in hardware. Some packets are however
>> punted to CPU; these include:
>> 
>>  1. Packets which need ARP resolution ("glean")
>> 
>>  2. Multicast packets, which are trickled to the CPU so the CPU can see them
>> and build (and refresh) hardware forwarding state
>> 
>>  3. ACL and uRPF denies, which are trickled to the CPU so it can maintain
>> counters
>> 
>>  4. Various other traffic like TTL failures, needing ICMP
>> 
>>  5. Obviously, packets address to the CPU (routing traffic, layer 2 PDUs)
>> 
>> Because the CPUs on these boxes are very, very puny, you want to limit what
>> hits the CPU. There are two methods available:
>> 
>>  1. The "mls rate-limit" commands; these will place a simple numeric rate
>> cap on certain types of traffic, and is done in hardware. There's no
>> prioritisation, but for certain types of traffic (e.g. TTL failures) you can
>> and should IMHO set low-ish limits. You SHOULD NOT use the "general" CEF
>> limiter; because you should use...
>> 
>>  2. CoPP - basically QoS on traffic punted to CPU. This is superior because
>> you can write ACLs defining what is most important to you, with very
>> granular control over policy. It suffers a couple of problems -
>> broadcast/multicast and non-IP traffic are done in software, and it can't
>> distinguish between glean and receive traffic, making a default-deny policy
>> tricky.
>> 
>> 
>> In short, common advice seems to be:
>> 
>>  1. Set low limits on the mls limiters for TTL & MTU failure, and optionally
>> ACL-drop ICMP unreach:
>> 
>> mls rate-limit unicast ip icmp unreachable acl-drop 0
>> mls rate-limit all ttl-failure 100 10
>> mls rate-limit all mtu-failure 100 10
>> 
>>  2. Use CoPP for everything else; DO NOT use the glean or cef receive
>> limiter
>> 
>> Search the archives and the Cisco docs for more info; it's not something I
>> can summarise in 5 minutes I'm afraid.
>> 
>> In your case, if you are going to perform a task which will potentially punt
>> a lot of multicast traffic to the CPU, I was suggesting that there are MLS
>> limiters which will reduce this; see my earlier email; though we run with
>> the defaults, which are (quite) high PPS values!
>> 
>> sh mls rate-limit  | inc MC
>> MCAST NON RPF   Off  -   - -
>>MCAST DFLT ADJ   On  10 100  Not sharing
>>  MCAST DIRECT CON   Off  -   - -
>>  MCAST PARTIAL SC   On  10 100  Not sharing
>> 


___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: [c-nsp] Weird Traceroute Issue to Specific Destination

[Paul Stewart]

No, just OSPF only - we thought about extending BGP to measure the effects
(just a BGP default route though as Sup2).  I just don't see where that
would help though ;)

Paul


-Original Message-
From: Heath Jones [mailto:hj1...@gmail.com] 
Sent: September-21-10 4:34 PM
To: Paul Stewart
Cc: cisco-nsp@puck.nether.net
Subject: Re: [c-nsp] Weird Traceroute Issue to Specific Destination

Is BGP information passed in to this 6500 (that connects to VXR), or
are you just using a default route?


___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: [c-nsp] Operational impact of switching from ingress to egress replication mode

[John Neiberger]

I see the "mls rate-limit multicast ipv4 connected" command, for
directly connected sources, but is there a command that would apply to
traffic going through box?

We have very little unicast traffic but a whole bunch of multicast
traffic, some of which is directly connected but much of it is simply
passing through.

Thanks again for all your help, I appreciate your time.

On Tue, Sep 21, 2010 at 3:12 PM, Phil Mayers  wrote:
> On 09/21/2010 09:58 PM, John Neiberger wrote:
>>
>> Sorry.  I also meant to say it's a Sup 720-3BXL. Based on what I can
>> see on CCO, that thing can forward 400 Mpps of ipv4 traffic. Does that
>> mean that I can set a rate limit of, say, 300 Mpps and somewhat guard
>> the CPU from meltdown for a few moments?
>
> I wish! 300Mpps hitting the CPU of a sup720 will kill it stone dead. It has
> (two) 600MHz CPUs, and they will not survive such a load.
>
> There's lots of info in this in the archives, but in brief: the sup720/pfc3
> architecture forwards most packets in hardware. Some packets are however
> punted to CPU; these include:
>
>  1. Packets which need ARP resolution ("glean")
>
>  2. Multicast packets, which are trickled to the CPU so the CPU can see them
> and build (and refresh) hardware forwarding state
>
>  3. ACL and uRPF denies, which are trickled to the CPU so it can maintain
> counters
>
>  4. Various other traffic like TTL failures, needing ICMP
>
>  5. Obviously, packets address to the CPU (routing traffic, layer 2 PDUs)
>
> Because the CPUs on these boxes are very, very puny, you want to limit what
> hits the CPU. There are two methods available:
>
>  1. The "mls rate-limit" commands; these will place a simple numeric rate
> cap on certain types of traffic, and is done in hardware. There's no
> prioritisation, but for certain types of traffic (e.g. TTL failures) you can
> and should IMHO set low-ish limits. You SHOULD NOT use the "general" CEF
> limiter; because you should use...
>
>  2. CoPP - basically QoS on traffic punted to CPU. This is superior because
> you can write ACLs defining what is most important to you, with very
> granular control over policy. It suffers a couple of problems -
> broadcast/multicast and non-IP traffic are done in software, and it can't
> distinguish between glean and receive traffic, making a default-deny policy
> tricky.
>
>
> In short, common advice seems to be:
>
>  1. Set low limits on the mls limiters for TTL & MTU failure, and optionally
> ACL-drop ICMP unreach:
>
> mls rate-limit unicast ip icmp unreachable acl-drop 0
> mls rate-limit all ttl-failure 100 10
> mls rate-limit all mtu-failure 100 10
>
>  2. Use CoPP for everything else; DO NOT use the glean or cef receive
> limiter
>
> Search the archives and the Cisco docs for more info; it's not something I
> can summarise in 5 minutes I'm afraid.
>
> In your case, if you are going to perform a task which will potentially punt
> a lot of multicast traffic to the CPU, I was suggesting that there are MLS
> limiters which will reduce this; see my earlier email; though we run with
> the defaults, which are (quite) high PPS values!
>
> sh mls rate-limit  | inc MC
>         MCAST NON RPF   Off                  -       -     -
>        MCAST DFLT ADJ   On              10     100  Not sharing
>      MCAST DIRECT CON   Off                  -       -     -
>      MCAST PARTIAL SC   On              10     100  Not sharing
>

___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: [c-nsp] Operational impact of switching from ingress to egress replication mode

[Phil Mayers]

On 09/21/2010 09:58 PM, John Neiberger wrote:

Sorry.  I also meant to say it's a Sup 720-3BXL. Based on what I can
see on CCO, that thing can forward 400 Mpps of ipv4 traffic. Does that
mean that I can set a rate limit of, say, 300 Mpps and somewhat guard
the CPU from meltdown for a few moments?


I wish! 300Mpps hitting the CPU of a sup720 will kill it stone dead. It 
has (two) 600MHz CPUs, and they will not survive such a load.


There's lots of info in this in the archives, but in brief: the 
sup720/pfc3 architecture forwards most packets in hardware. Some packets 
are however punted to CPU; these include:


 1. Packets which need ARP resolution ("glean")

 2. Multicast packets, which are trickled to the CPU so the CPU can see 
them and build (and refresh) hardware forwarding state


 3. ACL and uRPF denies, which are trickled to the CPU so it can 
maintain counters


 4. Various other traffic like TTL failures, needing ICMP

 5. Obviously, packets address to the CPU (routing traffic, layer 2 PDUs)

Because the CPUs on these boxes are very, very puny, you want to limit 
what hits the CPU. There are two methods available:


 1. The "mls rate-limit" commands; these will place a simple numeric 
rate cap on certain types of traffic, and is done in hardware. There's 
no prioritisation, but for certain types of traffic (e.g. TTL failures) 
you can and should IMHO set low-ish limits. You SHOULD NOT use the 
"general" CEF limiter; because you should use...


 2. CoPP - basically QoS on traffic punted to CPU. This is superior 
because you can write ACLs defining what is most important to you, with 
very granular control over policy. It suffers a couple of problems - 
broadcast/multicast and non-IP traffic are done in software, and it 
can't distinguish between glean and receive traffic, making a 
default-deny policy tricky.



In short, common advice seems to be:

 1. Set low limits on the mls limiters for TTL & MTU failure, and 
optionally ACL-drop ICMP unreach:


mls rate-limit unicast ip icmp unreachable acl-drop 0
mls rate-limit all ttl-failure 100 10
mls rate-limit all mtu-failure 100 10

 2. Use CoPP for everything else; DO NOT use the glean or cef receive 
limiter


Search the archives and the Cisco docs for more info; it's not something 
I can summarise in 5 minutes I'm afraid.


In your case, if you are going to perform a task which will potentially 
punt a lot of multicast traffic to the CPU, I was suggesting that there 
are MLS limiters which will reduce this; see my earlier email; though we 
run with the defaults, which are (quite) high PPS values!


sh mls rate-limit  | inc MC
 MCAST NON RPF   Off  -   - -
MCAST DFLT ADJ   On  10 100  Not sharing
  MCAST DIRECT CON   Off  -   - -
  MCAST PARTIAL SC   On  10 100  Not sharing
___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: [c-nsp] Operational impact of switching from ingress to egress replication mode

[John Neiberger]

Sorry.  I also meant to say it's a Sup 720-3BXL. Based on what I can
see on CCO, that thing can forward 400 Mpps of ipv4 traffic. Does that
mean that I can set a rate limit of, say, 300 Mpps and somewhat guard
the CPU from meltdown for a few moments?


On Tue, Sep 21, 2010 at 2:52 PM, John Neiberger  wrote:
> Are you all referring to "mls ip cef rate-limit"? If so, what do you
> think would be a good value to use on a Sup 720? We'd like to set it
> so that the CPU isn't overloaded so much that routing protocols drop
> and we don't lose our SSH sessions. That way we can monitor it and
> watch to see when the CPU drops back down to normal.
>
> Thanks!
> John
>
> On Tue, Sep 21, 2010 at 2:16 PM, John Neiberger  wrote:
>> Thanks. I've never used the MLS limiters before, so I'll look into how
>> they're configured in case we decide to use them. But we also have the
>> option of moving most of our production traffic away from these boxes
>> temporarily, so we may be able to just deal with the temporary chaos.
>>
>> John
>>
>> On Tue, Sep 21, 2010 at 2:04 PM, Benjamin Lovell  wrote:
>>> Excellent point and suggestion. This should prevent punts from smashing 
>>> your control plane and causing a cascading effect like the one I described.
>>>
>>> -Ben
>>>
>>>
>>> On Sep 21, 2010, at 3:47 PM, Phil Mayers wrote:
>>>
 On 09/21/2010 08:27 PM, Benjamin Lovell wrote:
> The primary thing to worry about here is the mcast packet rate not
> number of mroutes. Replication change will cause all mcast packets to
> be punted to CPU for a short period(few 100 msec or so).

 Remember you can rate-limit this with the MLS limiters. Whilst they 
 defaults are (very) high, lowering it for the duration of this change 
 could ease the problems.
 ___
 cisco-nsp mailing list  cisco-...@puck.nether.net
 https://puck.nether.net/mailman/listinfo/cisco-nsp
 archive at http://puck.nether.net/pipermail/cisco-nsp/
>>>
>>>
>>> ___
>>> cisco-nsp mailing list  cisco-...@puck.nether.net
>>> https://puck.nether.net/mailman/listinfo/cisco-nsp
>>> archive at http://puck.nether.net/pipermail/cisco-nsp/
>>>
>>
>

___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: [c-nsp] Operational impact of switching from ingress to egress replication mode

[Phil Mayers]

On 09/21/2010 09:52 PM, John Neiberger wrote:

Are you all referring to "mls ip cef rate-limit"? If so, what do you


I was referring to the multicast punt rate limiters:

mls rate-limit multicast ipv4 ?
  connected   Rate limiting of multicast packets from directly 
connected source

  fib-missRate limiting of fib-missed multicast packets
  igmpRate limiting of the IGMP protocol packets
  ip-options  rate limiting of multicast packets with ip options
  non-rpf Rate limiting of non-rpf multicast packets
  partial rate limiting of multicast packets during partial-SC state

I don't know what "mls ip cef rate-limit" does; my 6500/SXI box doesn't 
have it.



think would be a good value to use on a Sup 720? We'd like to set it
so that the CPU isn't overloaded so much that routing protocols drop
and we don't lose our SSH sessions. That way we can monitor it and
watch to see when the CPU drops back down to normal.


If you want to rate-limit unicast traffic hitting the CPU I would 
investigate CoPP. There's a lot of info on this in the list archives, 
and there are many reasons to prefer it to mls rate limiters for unicast 
traffic.

___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: [c-nsp] Operational impact of switching from ingress to egress replication mode

[John Neiberger]

Are you all referring to "mls ip cef rate-limit"? If so, what do you
think would be a good value to use on a Sup 720? We'd like to set it
so that the CPU isn't overloaded so much that routing protocols drop
and we don't lose our SSH sessions. That way we can monitor it and
watch to see when the CPU drops back down to normal.

Thanks!
John

On Tue, Sep 21, 2010 at 2:16 PM, John Neiberger  wrote:
> Thanks. I've never used the MLS limiters before, so I'll look into how
> they're configured in case we decide to use them. But we also have the
> option of moving most of our production traffic away from these boxes
> temporarily, so we may be able to just deal with the temporary chaos.
>
> John
>
> On Tue, Sep 21, 2010 at 2:04 PM, Benjamin Lovell  wrote:
>> Excellent point and suggestion. This should prevent punts from smashing your 
>> control plane and causing a cascading effect like the one I described.
>>
>> -Ben
>>
>>
>> On Sep 21, 2010, at 3:47 PM, Phil Mayers wrote:
>>
>>> On 09/21/2010 08:27 PM, Benjamin Lovell wrote:
 The primary thing to worry about here is the mcast packet rate not
 number of mroutes. Replication change will cause all mcast packets to
 be punted to CPU for a short period(few 100 msec or so).
>>>
>>> Remember you can rate-limit this with the MLS limiters. Whilst they 
>>> defaults are (very) high, lowering it for the duration of this change could 
>>> ease the problems.
>>> ___
>>> cisco-nsp mailing list  cisco-...@puck.nether.net
>>> https://puck.nether.net/mailman/listinfo/cisco-nsp
>>> archive at http://puck.nether.net/pipermail/cisco-nsp/
>>
>>
>> ___
>> cisco-nsp mailing list  cisco-...@puck.nether.net
>> https://puck.nether.net/mailman/listinfo/cisco-nsp
>> archive at http://puck.nether.net/pipermail/cisco-nsp/
>>
>

___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: [c-nsp] Weird Traceroute Issue to Specific Destination

[Heath Jones]

Is BGP information passed in to this 6500 (that connects to VXR), or
are you just using a default route?
___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: [c-nsp] Weird Traceroute Issue to Specific Destination

[Heath Jones]

> I put a static route at our Internet edge - we redistribute static into OSPF
> so now this /32 destination is able to be seen in the routing table (other
> than the default originated route).

That really does suggest that the routing information is incorrect. I
mentioned previously about routing networks behind some customers dsl
links (can't remember the attribute - framed-route or something)..
This could explain the intermittent connectivity, combined with some
policy routing or vrf seperation would explain the traceroute results.

Others have suggested policy routing also. I'm thinking that you don't
have 1x routing table - are you using vrf's or doing some vlan
trickery?

Its either:
- The output previously for cef has led us astray.
- The routing for this customer isnt the same as what we were looking
at with the cef output.
- Bug (unlikely, rebooted already, fixed with static).

Either way I'm labbing this up to see if I can actually create a
similar problem..

> I pulled the static out and same problem re-occurs... any thoughts?
Head explodes now :)
___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: [c-nsp] netflow tools

[Nairolf Relsserp]

On Mon, Sep 20, 2010 at 1:01 AM, Sharlon R. Carty  wrote:

> Anyone know of any netflow collector tools that can filter the data based on
> ASN? The majority tools I have tried filter based on IP address, IP group,
> domain name etc.

I recommend IsarFlow: http://isarflow.com/

This product is capable of a broad but meaningful variety of
netflow-driven analysis.

Two of the analysis also take the AS-numbers into account:

* analysis considering the source and destination AS of the flows:
This analysis is useful to find out which AS you communicate with
mostly (based on flows or transmitted bytes). For example, if you find
out that you communicate a lot with an AS you are not peering with
directly, this information might help to decide which AS to peer with
next (if possible)! :) It might be also a good base of information to
generally build your BGP routing-policy upon.

* analysis considering the peering-AS: If you have several AS you peer
with directly, this helps you to find out how your traffic distributes
to those. This may be helpful to check if your BGP routing-policy is
working as desired.

Feel free to contact me if you need more information about this solution.

Kind Regards,
Florian Pressler

-- 
DI(FH) Florian Pressler  |   Senior Systems Engineer| Security
CCIE #21049

phone: +43-1-23060-3280  |   mobile: +43-664-88445030   |  e-fax:
+43-1-23060-3289
email: f...@ong.at | skype: fpressler |
http://www.xing.com/profile/Florian_Pressler

Open Networks GmbH | Mooslackengasse 17 | 1190 Wien |
http://www.opennetworks.at/
___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: [c-nsp] Operational impact of switching from ingress to egress replication mode

[John Neiberger]

Thanks. I've never used the MLS limiters before, so I'll look into how
they're configured in case we decide to use them. But we also have the
option of moving most of our production traffic away from these boxes
temporarily, so we may be able to just deal with the temporary chaos.

John

On Tue, Sep 21, 2010 at 2:04 PM, Benjamin Lovell  wrote:
> Excellent point and suggestion. This should prevent punts from smashing your 
> control plane and causing a cascading effect like the one I described.
>
> -Ben
>
>
> On Sep 21, 2010, at 3:47 PM, Phil Mayers wrote:
>
>> On 09/21/2010 08:27 PM, Benjamin Lovell wrote:
>>> The primary thing to worry about here is the mcast packet rate not
>>> number of mroutes. Replication change will cause all mcast packets to
>>> be punted to CPU for a short period(few 100 msec or so).
>>
>> Remember you can rate-limit this with the MLS limiters. Whilst they defaults 
>> are (very) high, lowering it for the duration of this change could ease the 
>> problems.
>> ___
>> cisco-nsp mailing list  cisco-...@puck.nether.net
>> https://puck.nether.net/mailman/listinfo/cisco-nsp
>> archive at http://puck.nether.net/pipermail/cisco-nsp/
>
>
> ___
> cisco-nsp mailing list  cisco-...@puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-nsp
> archive at http://puck.nether.net/pipermail/cisco-nsp/
>

___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: [c-nsp] Operational impact of switching from ingress to egress replication mode

[Benjamin Lovell]

Excellent point and suggestion. This should prevent punts from smashing your 
control plane and causing a cascading effect like the one I described. 

-Ben


On Sep 21, 2010, at 3:47 PM, Phil Mayers wrote:

> On 09/21/2010 08:27 PM, Benjamin Lovell wrote:
>> The primary thing to worry about here is the mcast packet rate not
>> number of mroutes. Replication change will cause all mcast packets to
>> be punted to CPU for a short period(few 100 msec or so).
> 
> Remember you can rate-limit this with the MLS limiters. Whilst they defaults 
> are (very) high, lowering it for the duration of this change could ease the 
> problems.
> ___
> cisco-nsp mailing list  cisco-nsp@puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-nsp
> archive at http://puck.nether.net/pipermail/cisco-nsp/


___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: [c-nsp] Weird Traceroute Issue to Specific Destination

[Paul Stewart]

Ok... so here's the latest.

I put a static route at our Internet edge - we redistribute static into OSPF
so now this /32 destination is able to be seen in the routing table (other
than the default originated route).  This solves the issue if I statically
assign it the next hop (which is their ISP that we peer directly with via
LINX exchange point).  I don't like this solution though because if there
was ever an issue with the static next-hop it would never fail over ... plus
I don't see *why* I should have to do this in order to find the problem ;)

I pulled the static out and same problem re-occurs... any thoughts?

Thanks,

Paul


-Original Message-
From: cisco-nsp-boun...@puck.nether.net
[mailto:cisco-nsp-boun...@puck.nether.net] On Behalf Of chip
Sent: September-21-10 3:21 PM
To: cisco-nsp@puck.nether.net
Subject: Re: [c-nsp] Weird Traceroute Issue to Specific Destination

Just for fun, put in a static route for the /32 and then remove it on the
box that can't reach.

--chip


-- 
Just my $.02, your mileage may vary,  batteries not included, etc
___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/


___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: [c-nsp] Operational impact of switching from ingress to egress replication mode

[Phil Mayers]

On 09/21/2010 08:27 PM, Benjamin Lovell wrote:

The primary thing to worry about here is the mcast packet rate not
number of mroutes. Replication change will cause all mcast packets to
be punted to CPU for a short period(few 100 msec or so).


Remember you can rate-limit this with the MLS limiters. Whilst they 
defaults are (very) high, lowering it for the duration of this change 
could ease the problems.

___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: [c-nsp] Weird Traceroute Issue to Specific Destination

[Gert Doering]

Hi,

On Tue, Sep 21, 2010 at 11:47:45AM -0400, Paul Stewart wrote:
> Hehe. yeah, I hear ya.. At first I thought this is just one of those "hey,
> dummy look at the routing table"..;)

Mmmmh.  Any sort of policy routing or traffic engineering involved?

gert
-- 
USENET is *not* the non-clickable part of WWW!
   //www.muc.de/~gert/
Gert Doering - Munich, Germany g...@greenie.muc.de
fax: +49-89-35655025g...@net.informatik.tu-muenchen.de


pgpTzvwTTuQTp.pgp
Description: PGP signature
___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: [c-nsp] Operational impact of switching from ingress to egress replication mode

[John Neiberger]

That's good to know. We actually have several Gbps of multicast video
running over these boxes. Based on that, I suspect that we'll do that
change some other night unless I have plenty of time tonight to deal
with it.

Thanks for everyone's input. That helps me out a lot!
John

On Tue, Sep 21, 2010 at 1:27 PM, Benjamin Lovell  wrote:
> The primary thing to worry about here is the mcast packet rate not number of 
> mroutes. Replication change will cause all mcast packets to be punted to CPU 
> for a short period(few 100 msec or so). I have seen this go badly under the 
> following conditions.
>
> 1) High rate of mcast traffic. few Gigs per sec.
> 2) BFD will tight timers in the 750msec range.
>
> When mcast started getting punted the CPU spiked to the moon. Input queues 
> got smashed. BFD and therefor OSPF dropped. RPF interfaces changed and it 
> took almost 30min for the box to calm back down.
>
> If you have a moderate to high mcast packet rate I would not count on this 
> being a hitless change.
>
> -Ben
>
>
> On Sep 21, 2010, at 3:01 PM, John Neiberger wrote:
>
>> If I recall correctly, we have over 500 mroutes. I was just speaking
>> to a Cisco engineer that works with us about this. I think I'm going
>> to save this change until last. We have a lot of etherchannels and we
>> want to convert those to routed links with ECMP first, then we'll
>> switch over to egress replication. It sounds like we shouldn't have
>> more than a couple of seconds of impact.
>>
>> Thanks!
>> John
>>
>> On Tue, Sep 21, 2010 at 12:46 PM, Tim Stevenson  wrote:
>>> Hi John,
>>> Switching replication modes basically purges the hardware of all mroutes and
>>> those will be reprogrammed based on the current software state. It will be
>>> potentially disruptive for all mroutes, but the exact amount of traffic
>>> loss/blackholing would depend on the rate of each stream at the time, and
>>> the overall amount of time it takes to reprogram. For a few 100 mroutes, I
>>> would not expect much impact.
>>>
>>> Hope that helps,
>>> Tim
>>>
>>> At 11:30 AM 9/21/2010, John Neiberger averred:
>>>
 We're running 12.2(33)SRC2, I believe. It's actually experimental code
 and the exact version is overwritten with another code.

 On Tue, Sep 21, 2010 at 12:04 PM, Jeffrey Pazahanick 
 wrote:
> John,
> Having switched back and forth a few times, I never noticed more than
> a 1-2 second outage.
> What version of code are you on?
>
> On Tue, Sep 21, 2010 at 11:59 AM, John Neiberger 
> wrote:
>> We're going to be doing a whole bunch of maintenance tonight during a
>> maintenance window. One of the many things on our plate is to switch
>> from ingress replication mode to egress on some 7600s that have a few
>> hundred multicast routes on them. We know there is going to be at
>> least a minor blip while things settle down after making the change,
>> but I wanted to see if anyone on the list has done this and what the
>> operational impact was. I've heard there will be slight interruption
>> in traffic, but what sort of interruption are we talking about? Are we
>> speaking about a second or two?
>>
>> I'm asking because we're trying to decide if we want to split this out
>> to another night. If the disruption is minor and the risk is low then
>> we'll do it tonight. Otherwise, we might choose to do it on a separate
>> night.
>>
>> Any thoughts?
>> ___
>> cisco-nsp mailing list  cisco-...@puck.nether.net
>>
>> https://puck.nether.net/mailman/listinfo/cisco-nsp
>> archive at
>> http://puck.nether.net/pipermail/cisco-nsp/
>>
>

 ___
 cisco-nsp mailing list  cisco-...@puck.nether.net

 https://puck.nether.net/mailman/listinfo/cisco-nsp
 archive at
 http://puck.nether.net/pipermail/cisco-nsp/
>>>
>>>
>>>
>>>
>>> Tim Stevenson, tstev...@cisco.com
>>> Routing & Switching CCIE #5561
>>> Distinguished Technical Marketing Engineer, Cisco Nexus 7000
>>> Cisco - http://www.cisco.com
>>> IP Phone: 408-526-6759
>>> 
>>> The contents of this message may be *Cisco Confidential*
>>> and are intended for the specified recipients only.
>>>
>>>
>>>
>>
>> ___
>> cisco-nsp mailing list  cisco-...@puck.nether.net
>> https://puck.nether.net/mailman/listinfo/cisco-nsp
>> archive at http://puck.nether.net/pipermail/cisco-nsp/
>
>

___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: [c-nsp] FIB aggregation (was: Conditional advertise-map)

[Heath Jones]

> As I understand the problem, your interpretaion is correct. Only the
> worst case de-aggregation of /16 into 256 /24s would cause every
> even/odd /24 to find a new next-hop. Most cases would be somewhere
> between x0 (same next-hop) through x8 (your example) and up to x256
> (worst case).

Yes it is definately more of a concern with different next-hops for
prefixes within some aggregated space.
My gut feeling is that cam implementation is going to be an issue, as
I think ordering of aggregated prefixes in the cam and how to handle
non-matching prefixes in an aggregate is a problem for existing
implementations.

> Maybe someone could look at historic instability (from BGPmon?) the past
> couple of years and model how FIB aggregation would behave?

I've just finished the code for parsing mrt indextablev2 dumps. I have
to come up with some algorithm to choose the most affected ASs based
on prefixes and neighbor ASs. Any ideas appreciated!!
Once that is done and I have a few ASs to concentrate on, I was
planning to pump in real bgp update information (have to 'guesstimate'
how it will be seen by those ASs) from routeviews.
This should give a pretty good real-world answer to the question of
aggregation in tcam, for the worst case scenarios at the moment.
___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: [c-nsp] Operational impact of switching from ingress to egress replication mode

[Benjamin Lovell]

The primary thing to worry about here is the mcast packet rate not number of 
mroutes. Replication change will cause all mcast packets to be punted to CPU 
for a short period(few 100 msec or so). I have seen this go badly under the 
following conditions. 

1) High rate of mcast traffic. few Gigs per sec.
2) BFD will tight timers in the 750msec range. 

When mcast started getting punted the CPU spiked to the moon. Input queues got 
smashed. BFD and therefor OSPF dropped. RPF interfaces changed and it took 
almost 30min for the box to calm back down. 

If you have a moderate to high mcast packet rate I would not count on this 
being a hitless change. 

-Ben


On Sep 21, 2010, at 3:01 PM, John Neiberger wrote:

> If I recall correctly, we have over 500 mroutes. I was just speaking
> to a Cisco engineer that works with us about this. I think I'm going
> to save this change until last. We have a lot of etherchannels and we
> want to convert those to routed links with ECMP first, then we'll
> switch over to egress replication. It sounds like we shouldn't have
> more than a couple of seconds of impact.
> 
> Thanks!
> John
> 
> On Tue, Sep 21, 2010 at 12:46 PM, Tim Stevenson  wrote:
>> Hi John,
>> Switching replication modes basically purges the hardware of all mroutes and
>> those will be reprogrammed based on the current software state. It will be
>> potentially disruptive for all mroutes, but the exact amount of traffic
>> loss/blackholing would depend on the rate of each stream at the time, and
>> the overall amount of time it takes to reprogram. For a few 100 mroutes, I
>> would not expect much impact.
>> 
>> Hope that helps,
>> Tim
>> 
>> At 11:30 AM 9/21/2010, John Neiberger averred:
>> 
>>> We're running 12.2(33)SRC2, I believe. It's actually experimental code
>>> and the exact version is overwritten with another code.
>>> 
>>> On Tue, Sep 21, 2010 at 12:04 PM, Jeffrey Pazahanick 
>>> wrote:
 John,
 Having switched back and forth a few times, I never noticed more than
 a 1-2 second outage.
 What version of code are you on?
 
 On Tue, Sep 21, 2010 at 11:59 AM, John Neiberger 
 wrote:
> We're going to be doing a whole bunch of maintenance tonight during a
> maintenance window. One of the many things on our plate is to switch
> from ingress replication mode to egress on some 7600s that have a few
> hundred multicast routes on them. We know there is going to be at
> least a minor blip while things settle down after making the change,
> but I wanted to see if anyone on the list has done this and what the
> operational impact was. I've heard there will be slight interruption
> in traffic, but what sort of interruption are we talking about? Are we
> speaking about a second or two?
> 
> I'm asking because we're trying to decide if we want to split this out
> to another night. If the disruption is minor and the risk is low then
> we'll do it tonight. Otherwise, we might choose to do it on a separate
> night.
> 
> Any thoughts?
> ___
> cisco-nsp mailing list  cisco-nsp@puck.nether.net
> 
> https://puck.nether.net/mailman/listinfo/cisco-nsp
> archive at
> http://puck.nether.net/pipermail/cisco-nsp/
> 
 
>>> 
>>> ___
>>> cisco-nsp mailing list  cisco-nsp@puck.nether.net
>>> 
>>> https://puck.nether.net/mailman/listinfo/cisco-nsp
>>> archive at
>>> http://puck.nether.net/pipermail/cisco-nsp/
>> 
>> 
>> 
>> 
>> Tim Stevenson, tstev...@cisco.com
>> Routing & Switching CCIE #5561
>> Distinguished Technical Marketing Engineer, Cisco Nexus 7000
>> Cisco - http://www.cisco.com
>> IP Phone: 408-526-6759
>> 
>> The contents of this message may be *Cisco Confidential*
>> and are intended for the specified recipients only.
>> 
>> 
>> 
> 
> ___
> cisco-nsp mailing list  cisco-nsp@puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-nsp
> archive at http://puck.nether.net/pipermail/cisco-nsp/


___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: [c-nsp] Operational impact of switching from ingress to egress replication mode

[Jeffrey Pazahanick]

FWIW, I had some issues with SRC2 and egress replication..  Might be
specific to the ES20+ cards.

On Tue, Sep 21, 2010 at 2:01 PM, John Neiberger  wrote:
> If I recall correctly, we have over 500 mroutes. I was just speaking
> to a Cisco engineer that works with us about this. I think I'm going
> to save this change until last. We have a lot of etherchannels and we
> want to convert those to routed links with ECMP first, then we'll
> switch over to egress replication. It sounds like we shouldn't have
> more than a couple of seconds of impact.
>
> Thanks!
> John
>
> On Tue, Sep 21, 2010 at 12:46 PM, Tim Stevenson  wrote:
>> Hi John,
>> Switching replication modes basically purges the hardware of all mroutes and
>> those will be reprogrammed based on the current software state. It will be
>> potentially disruptive for all mroutes, but the exact amount of traffic
>> loss/blackholing would depend on the rate of each stream at the time, and
>> the overall amount of time it takes to reprogram. For a few 100 mroutes, I
>> would not expect much impact.
>>
>> Hope that helps,
>> Tim
>>
>> At 11:30 AM 9/21/2010, John Neiberger averred:
>>
>>> We're running 12.2(33)SRC2, I believe. It's actually experimental code
>>> and the exact version is overwritten with another code.
>>>
>>> On Tue, Sep 21, 2010 at 12:04 PM, Jeffrey Pazahanick 
>>> wrote:
>>> > John,
>>> > Having switched back and forth a few times, I never noticed more than
>>> > a 1-2 second outage.
>>> > What version of code are you on?
>>> >
>>> > On Tue, Sep 21, 2010 at 11:59 AM, John Neiberger 
>>> > wrote:
>>> >> We're going to be doing a whole bunch of maintenance tonight during a
>>> >> maintenance window. One of the many things on our plate is to switch
>>> >> from ingress replication mode to egress on some 7600s that have a few
>>> >> hundred multicast routes on them. We know there is going to be at
>>> >> least a minor blip while things settle down after making the change,
>>> >> but I wanted to see if anyone on the list has done this and what the
>>> >> operational impact was. I've heard there will be slight interruption
>>> >> in traffic, but what sort of interruption are we talking about? Are we
>>> >> speaking about a second or two?
>>> >>
>>> >> I'm asking because we're trying to decide if we want to split this out
>>> >> to another night. If the disruption is minor and the risk is low then
>>> >> we'll do it tonight. Otherwise, we might choose to do it on a separate
>>> >> night.
>>> >>
>>> >> Any thoughts?
>>> >> ___
>>> >> cisco-nsp mailing list  cisco-...@puck.nether.net
>>> >>
>>> >> https://puck.nether.net/mailman/listinfo/cisco-nsp
>>> >> archive at
>>> >> http://puck.nether.net/pipermail/cisco-nsp/
>>> >>
>>> >
>>>
>>> ___
>>> cisco-nsp mailing list  cisco-...@puck.nether.net
>>>
>>> https://puck.nether.net/mailman/listinfo/cisco-nsp
>>> archive at
>>> http://puck.nether.net/pipermail/cisco-nsp/
>>
>>
>>
>>
>> Tim Stevenson, tstev...@cisco.com
>> Routing & Switching CCIE #5561
>> Distinguished Technical Marketing Engineer, Cisco Nexus 7000
>> Cisco - http://www.cisco.com
>> IP Phone: 408-526-6759
>> 
>> The contents of this message may be *Cisco Confidential*
>> and are intended for the specified recipients only.
>>
>>
>>
>

___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: [c-nsp] Weird Traceroute Issue to Specific Destination

[Roger Wiklund]

Strange indeed.

I have seen a similar problem with the default route + CEF bug. But
that was on C10K.

You could try to add a static /32 route to the BADIP on the
xx.xx.120.25 box, just to exclude some default route issue. Also to
create a specific CEF entry.

Have you done some ip packet debugging when tracerouting etc?


On Tue, Sep 21, 2010 at 9:02 PM, Heath Jones  wrote:
> What happens when you try BADIP+1 or something close to it?
> Also if you happened to have assigned this BADIP to a dsl customer (or
> in a routed network via radius attribute behind it), and had the
> config on the lns cause the next hop to be the 6500 (policy routing,
> vrf etc)..
>
> I noticed the cef version was pretty high also. Was that before the
> reboot, or because of a lot of users coming and going?
>
> I have a headache because of this problem now ;)
>
>
>
> On 21 September 2010 19:39, Paul Stewart  wrote:
>> Thanks to everyone... yeah, this is a very strange issue.  We've tested
>> about 150 destinations through that path so far and only one of the
>> destination IP's has given us the weird timeouts in the traceroute (which
>> results in the traffic not passing specific to that destination).
>>
>> Last night, we had the opportunity to do a maintenance window and rebooted
>> the 6500 and 7206VXR closest to the customer - no change.  They had been up
>> for about a year...
>>
>> No, there isn't any security related devices sitting along there - we have
>> them, just not in that part of the network.  They are not inline neither...
>>
>> We'll keep poking away - appreciate it..
>>
>> Paul
>
> ___
> cisco-nsp mailing list  cisco-...@puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-nsp
> archive at http://puck.nether.net/pipermail/cisco-nsp/
>

___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: [c-nsp] Weird Traceroute Issue to Specific Destination

[chip]

Just for fun, put in a static route for the /32 and then remove it on the
box that can't reach.

--chip


-- 
Just my $.02, your mileage may vary,  batteries not included, etc
___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: [c-nsp] Weird Traceroute Issue to Specific Destination

[Paul Stewart]

Thanks - very interesting

So I tried some hostnames below and above (server#.domainname.co.uk) and
they all went through... then I seen your suggestion about IP (why I didn't
think of that I have no idea) and interestingly enough one IP below OR one
IP above works perfectly fine.  So on the surface it appears that only a
single /32 is causing us all this grief.

The CEF version was taken before the reboot - so it's pretty low again ... 

Paul


-Original Message-
From: Heath Jones [mailto:hj1...@gmail.com] 
Sent: September-21-10 3:03 PM
To: Paul Stewart
Cc: cisco-nsp@puck.nether.net
Subject: Re: [c-nsp] Weird Traceroute Issue to Specific Destination

What happens when you try BADIP+1 or something close to it?
Also if you happened to have assigned this BADIP to a dsl customer (or
in a routed network via radius attribute behind it), and had the
config on the lns cause the next hop to be the 6500 (policy routing,
vrf etc)..

I noticed the cef version was pretty high also. Was that before the
reboot, or because of a lot of users coming and going?

I have a headache because of this problem now ;)



On 21 September 2010 19:39, Paul Stewart  wrote:
> Thanks to everyone... yeah, this is a very strange issue.  We've tested
> about 150 destinations through that path so far and only one of the
> destination IP's has given us the weird timeouts in the traceroute (which
> results in the traffic not passing specific to that destination).
>
> Last night, we had the opportunity to do a maintenance window and rebooted
> the 6500 and 7206VXR closest to the customer - no change.  They had been
up
> for about a year...
>
> No, there isn't any security related devices sitting along there - we have
> them, just not in that part of the network.  They are not inline
neither...
>
> We'll keep poking away - appreciate it..
>
> Paul



___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

[c-nsp] FIB aggregation (was: Conditional advertise-map)

[Peter Rathlev]

On Tue, 2010-09-21 at 18:46 +0100, Heath Jones wrote: 
> I've read in a lot of places now that if you have say 192.168.0.0/16
> and then a /24 drops out, that you will end up with 255 /24's in the
> table. I'm trying to understand why that is.. (classful implementation
> almost?)
> My thinking suggests you could have for example one each: /17 + /18 +
> /19 + /20 + /21 + /22 + /23 + /24.

As I understand the problem, your interpretaion is correct. Only the
worst case de-aggregation of /16 into 256 /24s would cause every
even/odd /24 to find a new next-hop. Most cases would be somewhere
between x0 (same next-hop) through x8 (your example) and up to x256
(worst case).

It would of course take the CPU some time to calculate the new
aggregation, and in that period you would potentially have an invalid
FIB for more than just the prefix that disappeared.

If one of your upstreams (or one of their upstreams) have issues you
could see a lot of aggregated prefixes split up like this. And you can't
easily know for sure how many in advance.

Maybe someone could look at historic instability (from BGPmon?) the past
couple of years and model how FIB aggregation would behave?

-- 
Peter


___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: [c-nsp] Weird Traceroute Issue to Specific Destination

[Heath Jones]

What happens when you try BADIP+1 or something close to it?
Also if you happened to have assigned this BADIP to a dsl customer (or
in a routed network via radius attribute behind it), and had the
config on the lns cause the next hop to be the 6500 (policy routing,
vrf etc)..

I noticed the cef version was pretty high also. Was that before the
reboot, or because of a lot of users coming and going?

I have a headache because of this problem now ;)



On 21 September 2010 19:39, Paul Stewart  wrote:
> Thanks to everyone... yeah, this is a very strange issue.  We've tested
> about 150 destinations through that path so far and only one of the
> destination IP's has given us the weird timeouts in the traceroute (which
> results in the traffic not passing specific to that destination).
>
> Last night, we had the opportunity to do a maintenance window and rebooted
> the 6500 and 7206VXR closest to the customer - no change.  They had been up
> for about a year...
>
> No, there isn't any security related devices sitting along there - we have
> them, just not in that part of the network.  They are not inline neither...
>
> We'll keep poking away - appreciate it..
>
> Paul

___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: [c-nsp] Operational impact of switching from ingress to egress replication mode

[John Neiberger]

If I recall correctly, we have over 500 mroutes. I was just speaking
to a Cisco engineer that works with us about this. I think I'm going
to save this change until last. We have a lot of etherchannels and we
want to convert those to routed links with ECMP first, then we'll
switch over to egress replication. It sounds like we shouldn't have
more than a couple of seconds of impact.

Thanks!
John

On Tue, Sep 21, 2010 at 12:46 PM, Tim Stevenson  wrote:
> Hi John,
> Switching replication modes basically purges the hardware of all mroutes and
> those will be reprogrammed based on the current software state. It will be
> potentially disruptive for all mroutes, but the exact amount of traffic
> loss/blackholing would depend on the rate of each stream at the time, and
> the overall amount of time it takes to reprogram. For a few 100 mroutes, I
> would not expect much impact.
>
> Hope that helps,
> Tim
>
> At 11:30 AM 9/21/2010, John Neiberger averred:
>
>> We're running 12.2(33)SRC2, I believe. It's actually experimental code
>> and the exact version is overwritten with another code.
>>
>> On Tue, Sep 21, 2010 at 12:04 PM, Jeffrey Pazahanick 
>> wrote:
>> > John,
>> > Having switched back and forth a few times, I never noticed more than
>> > a 1-2 second outage.
>> > What version of code are you on?
>> >
>> > On Tue, Sep 21, 2010 at 11:59 AM, John Neiberger 
>> > wrote:
>> >> We're going to be doing a whole bunch of maintenance tonight during a
>> >> maintenance window. One of the many things on our plate is to switch
>> >> from ingress replication mode to egress on some 7600s that have a few
>> >> hundred multicast routes on them. We know there is going to be at
>> >> least a minor blip while things settle down after making the change,
>> >> but I wanted to see if anyone on the list has done this and what the
>> >> operational impact was. I've heard there will be slight interruption
>> >> in traffic, but what sort of interruption are we talking about? Are we
>> >> speaking about a second or two?
>> >>
>> >> I'm asking because we're trying to decide if we want to split this out
>> >> to another night. If the disruption is minor and the risk is low then
>> >> we'll do it tonight. Otherwise, we might choose to do it on a separate
>> >> night.
>> >>
>> >> Any thoughts?
>> >> ___
>> >> cisco-nsp mailing list  cisco-...@puck.nether.net
>> >>
>> >> https://puck.nether.net/mailman/listinfo/cisco-nsp
>> >> archive at
>> >> http://puck.nether.net/pipermail/cisco-nsp/
>> >>
>> >
>>
>> ___
>> cisco-nsp mailing list  cisco-...@puck.nether.net
>>
>> https://puck.nether.net/mailman/listinfo/cisco-nsp
>> archive at
>> http://puck.nether.net/pipermail/cisco-nsp/
>
>
>
>
> Tim Stevenson, tstev...@cisco.com
> Routing & Switching CCIE #5561
> Distinguished Technical Marketing Engineer, Cisco Nexus 7000
> Cisco - http://www.cisco.com
> IP Phone: 408-526-6759
> 
> The contents of this message may be *Cisco Confidential*
> and are intended for the specified recipients only.
>
>
>

___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: [c-nsp] Operational impact of switching from ingress to egress replication mode

[Tim Stevenson]

Hi John,
Switching replication modes basically purges the hardware of all 
mroutes and those will be reprogrammed based on the current software 
state. It will be potentially disruptive for all mroutes, but the 
exact amount of traffic loss/blackholing would depend on the rate of 
each stream at the time, and the overall amount of time it takes to 
reprogram. For a few 100 mroutes, I would not expect much impact.


Hope that helps,
Tim

At 11:30 AM 9/21/2010, John Neiberger averred:


We're running 12.2(33)SRC2, I believe. It's actually experimental code
and the exact version is overwritten with another code.

On Tue, Sep 21, 2010 at 12:04 PM, Jeffrey Pazahanick 
 wrote:

> John,
> Having switched back and forth a few times, I never noticed more than
> a 1-2 second outage.
> What version of code are you on?
>
> On Tue, Sep 21, 2010 at 11:59 AM, John Neiberger 
 wrote:

>> We're going to be doing a whole bunch of maintenance tonight during a
>> maintenance window. One of the many things on our plate is to switch
>> from ingress replication mode to egress on some 7600s that have a few
>> hundred multicast routes on them. We know there is going to be at
>> least a minor blip while things settle down after making the change,
>> but I wanted to see if anyone on the list has done this and what the
>> operational impact was. I've heard there will be slight interruption
>> in traffic, but what sort of interruption are we talking about? Are we
>> speaking about a second or two?
>>
>> I'm asking because we're trying to decide if we want to split this out
>> to another night. If the disruption is minor and the risk is low then
>> we'll do it tonight. Otherwise, we might choose to do it on a separate
>> night.
>>
>> Any thoughts?
>> ___
>> cisco-nsp mailing list  cisco-nsp@puck.nether.net
>> 
https://puck.nether.net/mailman/listinfo/cisco-nsp
>> archive at 
http://puck.nether.net/pipermail/cisco-nsp/

>>
>

___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at 
http://puck.nether.net/pipermail/cisco-nsp/





Tim Stevenson, tstev...@cisco.com
Routing & Switching CCIE #5561
Distinguished Technical Marketing Engineer, Cisco Nexus 7000
Cisco - http://www.cisco.com
IP Phone: 408-526-6759

The contents of this message may be *Cisco Confidential*
and are intended for the specified recipients only.


___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: [c-nsp] Weird Traceroute Issue to Specific Destination

[Paul Stewart]

Thanks to everyone... yeah, this is a very strange issue.  We've tested
about 150 destinations through that path so far and only one of the
destination IP's has given us the weird timeouts in the traceroute (which
results in the traffic not passing specific to that destination).

Last night, we had the opportunity to do a maintenance window and rebooted
the 6500 and 7206VXR closest to the customer - no change.  They had been up
for about a year...

No, there isn't any security related devices sitting along there - we have
them, just not in that part of the network.  They are not inline neither...

We'll keep poking away - appreciate it..

Paul


-Original Message-
From: Heath Jones [mailto:hj1...@gmail.com] 
Sent: September-21-10 12:59 PM
To: Paul Stewart
Cc: Gert Doering; cisco-nsp@puck.nether.net
Subject: Re: [c-nsp] Weird Traceroute Issue to Specific Destination

So far, it's got me stumped! Put some port mirroring and capture from
that 6500 perhaps..
My mind is hovering somewhere around ttl / forward path being ok, but
return broken for some reason.. but I don't think it will have any
success.
Its worth checking (as Brian said) 'show ip cef exact-route' (it's not
mpls specific). Based on the info so far, the forward path seems ok.

Some people have 'passive' security related devices sitting in the
middle that they don't tell anyone about - is that a possibility in
your network?

I'll let you know if I have further thoughts, but good luck with it!!



> No, pure co-incidence ... the first couple of octets are significantly
> difference but good catch ;)
>
> If nobody can find anything obvious here (appreciate all the input and
> troubleshooting ideas) then we'll keep poking away and take this back
> offlist on my side


___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: [c-nsp] Operational impact of switching from ingress to egress replication mode

[John Neiberger]

We're running 12.2(33)SRC2, I believe. It's actually experimental code
and the exact version is overwritten with another code.

On Tue, Sep 21, 2010 at 12:04 PM, Jeffrey Pazahanick  wrote:
> John,
> Having switched back and forth a few times, I never noticed more than
> a 1-2 second outage.
> What version of code are you on?
>
> On Tue, Sep 21, 2010 at 11:59 AM, John Neiberger  wrote:
>> We're going to be doing a whole bunch of maintenance tonight during a
>> maintenance window. One of the many things on our plate is to switch
>> from ingress replication mode to egress on some 7600s that have a few
>> hundred multicast routes on them. We know there is going to be at
>> least a minor blip while things settle down after making the change,
>> but I wanted to see if anyone on the list has done this and what the
>> operational impact was. I've heard there will be slight interruption
>> in traffic, but what sort of interruption are we talking about? Are we
>> speaking about a second or two?
>>
>> I'm asking because we're trying to decide if we want to split this out
>> to another night. If the disruption is minor and the risk is low then
>> we'll do it tonight. Otherwise, we might choose to do it on a separate
>> night.
>>
>> Any thoughts?
>> ___
>> cisco-nsp mailing list  cisco-...@puck.nether.net
>> https://puck.nether.net/mailman/listinfo/cisco-nsp
>> archive at http://puck.nether.net/pipermail/cisco-nsp/
>>
>

___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: [c-nsp] Operational impact of switching from ingress to egress replication mode

[Jeffrey Pazahanick]

John,
Having switched back and forth a few times, I never noticed more than
a 1-2 second outage.
What version of code are you on?

On Tue, Sep 21, 2010 at 11:59 AM, John Neiberger  wrote:
> We're going to be doing a whole bunch of maintenance tonight during a
> maintenance window. One of the many things on our plate is to switch
> from ingress replication mode to egress on some 7600s that have a few
> hundred multicast routes on them. We know there is going to be at
> least a minor blip while things settle down after making the change,
> but I wanted to see if anyone on the list has done this and what the
> operational impact was. I've heard there will be slight interruption
> in traffic, but what sort of interruption are we talking about? Are we
> speaking about a second or two?
>
> I'm asking because we're trying to decide if we want to split this out
> to another night. If the disruption is minor and the risk is low then
> we'll do it tonight. Otherwise, we might choose to do it on a separate
> night.
>
> Any thoughts?
> ___
> cisco-nsp mailing list  cisco-...@puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-nsp
> archive at http://puck.nether.net/pipermail/cisco-nsp/
>

___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: [c-nsp] Conditional advertise-map

[Heath Jones]

Hi Gert & All,

Just a quick one.. I've been working on getting this modelled up by
using mrt dumps from routeviews.
I'm processing the tables to try and find out which AS has the most
omni-directional routing information. I mean, trying to figure out who
would get hammered the most if they were doing 'tcam aggregation'.
An AS that has a lot of non-contiguous prefixes going out of a lot of
different directions is key..

I've read in a lot of places now that if you have say 192.168.0.0/16
and then a /24 drops out, that you will end up with 255 /24's in the
table. I'm trying to understand why that is.. (classful implementation
almost?)
My thinking suggests you could have for example one each: /17 + /18 +
/19 + /20 + /21 + /22 + /23 + /24.


Any thoughts appreciated!

Heath






> Well, today I can look at my BGP sessions and say "I am receiving
> 294.712 prefixes, and so I need a TCAM that is at least that big".
___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: [c-nsp] Weird Traceroute Issue to Specific Destination

[Brian Turnbow]

Hi all

Please see comments in line
 

> -Original Message-
> From: cisco-nsp-boun...@puck.nether.net 
> [mailto:cisco-nsp-boun...@puck.nether.net] On Behalf Of Paul Stewart
> Sent: martedì 21 settembre 2010 17.48
> To: 'Heath Jones'
> Cc: cisco-nsp@puck.nether.net
> Subject: Re: [c-nsp] Weird Traceroute Issue to Specific Destination
> 
> Hehe. yeah, I hear ya.. At first I thought this is just one 
> of those "hey,
> dummy look at the routing table"..;)
> 
>  
> 
> What's killing me is that every hop from the 7200 right to 
> our Internet edge
> shows the 0.0.0.0/0 OSPF route as preferred which is what's expected.
> 
>  
> 
> dis2-rtr-mb#show ip route xx.xxx.2.226
> 
> % Network not in table
> 
> dis2-rtr-mb#show ip cef xx.xxx.2.226
> 
> 0.0.0.0/0, version 8684984, epoch 1, cached adjacency xx.xxx.0.226
> 
> 0 packets, 0 bytes
> 
>   via xx.xxx.0.226, Vlan4, 0 dependencies
> 
> next hop xx.xxx.0.226, Vlan4
> 
> valid cached adjacency
> 

You may want to try 
sh ip cef exact-route with source and destination to see if it changes, 
as well as the sh mls cef flavours on the 6500/7600s
and don't forget to check labels if you have mpls.

Brian




> 
>  
> 
> Paul
> 
>  
> 
>  
> 
> From: Heath Jones [mailto:hj1...@gmail.com] 
> Sent: Tuesday, September 21, 2010 11:38 AM
> To: Paul Stewart
> Cc: cisco-nsp@puck.nether.net
> Subject: Re: [c-nsp] Weird Traceroute Issue to Specific Destination
> 
>  
> 
> I need a coffee or 2, I am misreading absolutely everything today!!
> 
>  
> 
> Ok so that IP is not the customer IP - it's the destination 
> on the other
> side of the net somewhere..
> 
> Gert is correct, the routing and forwarding tables will show 
> you what is
> different about that ip.
> 
>  
> 
>  
> 
> 
> 
>  
> 
> On 21 September 2010 16:23, Heath Jones  wrote:
> 
> If my understanding is correct here, then the DSL user is 
> probably blocking
> inbound icmp so you would expect the traceroutes you see.. 
> (just constant
> timeouts).
> 
> Lets take a step back here... What problem is the customer reporting?
> 
>  
> 
>  
> 
> 
> 
>  
> 
> On 21 September 2010 16:04, Paul Stewart  wrote:
> 
> Yes, loopback is in place and the source . yes, loopback in 
> routing table
> (redistributed via OSPF).  This 7206VXR has been in 
> production for over 4
> years and we have no issues reaching any other websites.
> 
>  
> 
> I'm confident that if the remote IP was blocking us or 
> something of that
> nature that the traceroute would at least transverse our igp 
> properly . 
> 
>  
> 
> Thanks,
> 
>  
> 
> Paul
> 
>  
> 
>  
> 
> From: Heath Jones [mailto:hj1...@gmail.com] 
> Sent: Tuesday, September 21, 2010 11:00 AM 
> 
> 
> To: Paul Stewart
> Cc: cisco-nsp@puck.nether.net
> Subject: Re: [c-nsp] Weird Traceroute Issue to Specific Destination
> 
>  
> 
> If it's not a firewall, its probably routing.. Is the 7206VXR using a
> loopback for the source of the icmp request packets, and do 
> you have a route
> back to this ip in your igp?
> 
> 
> 
>  
> 
> On 21 September 2010 15:17, Paul Stewart  wrote:
> 
> Thank you - good thinking but I checked and there's nothing 
> in there to
> limit ICMP at all..;)
> 
>  
> 
> Paul
> 
>  
> 
>  
> 
> From: Heath Jones [mailto:hj1...@gmail.com] 
> Sent: Tuesday, September 21, 2010 10:05 AM
> To: Paul Stewart
> Cc: cisco-nsp@puck.nether.net
> Subject: Re: [c-nsp] Weird Traceroute Issue to Specific Destination
> 
>  
> 
> Hi Paul - perhaps you have a firewall filter preventing the 
> ingress icmp
> replies (to the 7206VXR)..?
> 
> 
> 
>  
> 
> On 21 September 2010 14:54, Paul Stewart  wrote:
> 
> Hi folks..
> 
> 
> 
> We have a customer who is connected over DSL who is having 
> issues getting to
> a certain remote site more often than not.  Sometime they can 
> reach this
> site, but most of the time they cannot.
> 
> 
> 
> They connect to a 7206VXR, which then connects to a 6509 which then
> connections to 6509, 6509, then 7606 out to Internet.  Long 
> story short,
> there is no reported issues along this connectivity at all 
> and we can only
> replicate this complaint to one remote IP address.  
> Logically, we would push
> this back and say "not our problem" which we're confident 
> it's not *but*
> there's one strange thing that is bugging me and I can't put 
> logic around
> this (I also have a terrible head cold and not thinking straight).
> 
> 
> 
> When logged into the 7206VXR where the customer connects via DSL, a
> traceroute to the Internet loops normally like this:
> 
> 
> 
> acs1-con-bb#traceroute www.cnn.com  
> 
> Translating "www.cnn.com  "...domain server
> (208.67.222.222) [OK]
> 
> 
> 
> Type escape sequence to abort.
> 
> Tracing the route to www.cnn.com   
> (157.166.224.26)
> 
> 
> 
>  1 xx.xxx.7.65 0 msec 0 msec 0 msec
> 
>  2 xx.xx.120.25 8 msec 8 msec 8 msec
> 
>  3 core2-rtr-to-ge4-12-vl4.nexicom.net
>   (98.124.0.226) 
> 20 msec

Re: [c-nsp] Embeded Packet Capture for DHCP Troubleshooting

[Heath Jones]

If the problem was portfast (which is a very good suggestion as it
causes problems with a whole bunch of things when listening at the
start), the following would not work..

>  And from the switch side - no errors or flaps. And the trouble is the 
> clients in question don't ever fully complete the DHCP transaction upon power 
> up to assign an ip - so they can't connect -but if I assign an IP manually on 
> the client - then they can connect right up to Citrix servers.

Is it the case that the hosts get them sometime after the first 30
seconds, stop making requests, and that manual assignment is not
required?

___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: [c-nsp] Embeded Packet Capture for DHCP Troubleshooting

[Jeff Wojciechowski]

Yes - portfast on on all interface - makes sense that it could confuse DHCP.

Thanks,

-Jeff


-Original Message-
From: cisco-nsp-boun...@puck.nether.net 
[mailto:cisco-nsp-boun...@puck.nether.net] On Behalf Of Raymond Burkholder
Sent: Tuesday, September 21, 2010 10:41 AM
To: cisco-nsp@puck.nether.net
Subject: Re: [c-nsp] Embeded Packet Capture for DHCP Troubleshooting

>
> 2801 is doing dhcp - and I wanted to get a packet capture to see why 2
thin
> clients won't get dhcp leases successfully from the 2801 upon boot up
> (so before the client has an ip from dhcp - isn't the dhcp transaction
> all
arp?)
>

Check your switch.   Do you have portfast turned on?  I've seen port
negotiation get in the way of dhcp negotiation.


--
This message has been scanned for viruses and dangerous content by MailScanner, 
and is believed to be clean.

___
cisco-nsp mailing list  cisco-nsp@puck.nether.net 
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

This electronic mail (including any attachments) may contain information that 
is privileged, confidential, or otherwise protected from disclosure to anyone 
other than its intended recipient(s). Any dissemination or use of this 
electronic mail or its contents (including any attachments) by persons other 
than the intended recipient(s) is strictly prohibited. If you have received 
this message in error, please delete the original message in its entirety 
(including any attachments) and notify us immediately by reply email so that we 
may correct our internal records.  Midland Paper Company accepts no 
responsibility for any loss or damage from use of this electronic mail, 
including any damage resulting from a computer virus.

___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

[c-nsp] Operational impact of switching from ingress to egress replication mode

[John Neiberger]

We're going to be doing a whole bunch of maintenance tonight during a
maintenance window. One of the many things on our plate is to switch
from ingress replication mode to egress on some 7600s that have a few
hundred multicast routes on them. We know there is going to be at
least a minor blip while things settle down after making the change,
but I wanted to see if anyone on the list has done this and what the
operational impact was. I've heard there will be slight interruption
in traffic, but what sort of interruption are we talking about? Are we
speaking about a second or two?

I'm asking because we're trying to decide if we want to split this out
to another night. If the disruption is minor and the risk is low then
we'll do it tonight. Otherwise, we might choose to do it on a separate
night.

Any thoughts?
___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: [c-nsp] Weird Traceroute Issue to Specific Destination

[Heath Jones]

So far, it's got me stumped! Put some port mirroring and capture from
that 6500 perhaps..
My mind is hovering somewhere around ttl / forward path being ok, but
return broken for some reason.. but I don't think it will have any
success.
Its worth checking (as Brian said) 'show ip cef exact-route' (it's not
mpls specific). Based on the info so far, the forward path seems ok.

Some people have 'passive' security related devices sitting in the
middle that they don't tell anyone about - is that a possibility in
your network?

I'll let you know if I have further thoughts, but good luck with it!!



> No, pure co-incidence ... the first couple of octets are significantly
> difference but good catch ;)
>
> If nobody can find anything obvious here (appreciate all the input and
> troubleshooting ideas) then we'll keep poking away and take this back
> offlist on my side
___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: [c-nsp] Embeded Packet Capture for DHCP Troubleshooting

[Jeff Wojciechowski]

Makes sense - thanks again everyone.

Thanks,

-Jeff


-Original Message-
From: cisco-nsp-boun...@puck.nether.net 
[mailto:cisco-nsp-boun...@puck.nether.net] On Behalf Of Phil Mayers
Sent: Tuesday, September 21, 2010 11:16 AM
To: cisco-nsp@puck.nether.net
Subject: Re: [c-nsp] Embeded Packet Capture for DHCP Troubleshooting

On 21/09/10 16:23, Jeff Wojciechowski wrote:

> boot up (so before the client has an ip from dhcp - isn't the dhcp
> transaction all arp?)

No. It's UDP packets on ports 67 and 68.

It uses source/dest addresses of 0.0.0.0 and 255.255.255.255, but it's still 
plain IP packets, so embedded packet capture should work.
___
cisco-nsp mailing list  cisco-nsp@puck.nether.net 
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

This electronic mail (including any attachments) may contain information that 
is privileged, confidential, or otherwise protected from disclosure to anyone 
other than its intended recipient(s). Any dissemination or use of this 
electronic mail or its contents (including any attachments) by persons other 
than the intended recipient(s) is strictly prohibited. If you have received 
this message in error, please delete the original message in its entirety 
(including any attachments) and notify us immediately by reply email so that we 
may correct our internal records.  Midland Paper Company accepts no 
responsibility for any loss or damage from use of this electronic mail, 
including any damage resulting from a computer virus.

___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: [c-nsp] Weird Traceroute Issue to Specific Destination

[Paul Stewart]

No, just pure routing ;)  Thanks though for the thought...

Paul


-Original Message-
From: Heath Jones [mailto:hj1...@gmail.com] 
Sent: September-21-10 12:35 PM
To: Paul Stewart
Cc: cisco-nsp@puck.nether.net
Subject: Re: [c-nsp] Weird Traceroute Issue to Specific Destination

Just a random thought.. have you got any ipsec tunnels going? I just
checked and they don't appear in cef output (cryptomap on ingress).


___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: [c-nsp] Weird Traceroute Issue to Specific Destination

[Heath Jones]

Just a random thought.. have you got any ipsec tunnels going? I just
checked and they don't appear in cef output (cryptomap on ingress).
___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: [c-nsp] Weird Traceroute Issue to Specific Destination

[Paul Stewart]

>> dis2-rtr-mb#show ip route xx.xxx.2.226
>> % Network not in table
>> dis2-rtr-mb#show ip cef xx.xxx.2.226
>> 0.0.0.0/0, version 8684984, epoch 1, cached adjacency xx.xxx.0.226
>> 0 packets, 0 bytes
>>  via xx.xxx.0.226, Vlan4, 0 dependencies
>>next hop xx.xxx.0.226, Vlan4
>>valid cached adjacency

> Am I reading into it too much when I see a similarity between
> xx.xxx.2.226 and xx.xxx.0.226 ?

No, pure co-incidence ... the first couple of octets are significantly
difference but good catch ;)

If nobody can find anything obvious here (appreciate all the input and
troubleshooting ideas) then we'll keep poking away and take this back
offlist on my side

Thanks,

Paul



___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: [c-nsp] Embeded Packet Capture for DHCP Troubleshooting

[Raymond Burkholder]

> 
> 2801 is doing dhcp - and I wanted to get a packet capture to see why 2
thin
> clients won't get dhcp leases successfully from the 2801 upon boot up (so
> before the client has an ip from dhcp - isn't the dhcp transaction all
arp?)
> 

Check your switch.   Do you have portfast turned on?  I've seen port
negotiation get in the way of dhcp negotiation.


-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.

___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: [c-nsp] Weird Traceroute Issue to Specific Destination

[Heath Jones]

> dis2-rtr-mb#show ip route xx.xxx.2.226
> % Network not in table
> dis2-rtr-mb#show ip cef xx.xxx.2.226
> 0.0.0.0/0, version 8684984, epoch 1, cached adjacency xx.xxx.0.226
> 0 packets, 0 bytes
>  via xx.xxx.0.226, Vlan4, 0 dependencies
>next hop xx.xxx.0.226, Vlan4
>valid cached adjacency

Am I reading into it too much when I see a similarity between
xx.xxx.2.226 and xx.xxx.0.226 ?


> Try "show ip cef".  Sometimes cef gets confused about things (like "oh,
> I've heard an ARP reply for this address, must be locally connected").
>
> (This is a fairly old CEF bug, and I'm sure it got fixed long since, but
> might be something along that lines)

I was thinking along these lines when I noticed the Vlan4 (ie.
switched environment with potential for multiple hosts per segment),
but didn't realise there was actually a bug historically.

> gert

> PS: this thread shows a serious lack of reasonable quoting.

Fixed :)

___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: [c-nsp] Embeded Packet Capture for DHCP Troubleshooting

[Phil Mayers]

On 21/09/10 16:23, Jeff Wojciechowski wrote:


boot up (so before the client has an ip from dhcp - isn't the dhcp
transaction all arp?)


No. It's UDP packets on ports 67 and 68.

It uses source/dest addresses of 0.0.0.0 and 255.255.255.255, but it's 
still plain IP packets, so embedded packet capture should work.

___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: [c-nsp] Embeded Packet Capture for DHCP Troubleshooting

[Jeff Wojciechowski]

I'll give that a whirl - thanks so much!!

-Jeff


-Original Message-
From: Heath Jones [mailto:hj1...@gmail.com]
Sent: Tuesday, September 21, 2010 11:12 AM
To: Jeff Wojciechowski
Cc: cisco-nsp@puck.nether.net
Subject: Re: [c-nsp] Embeded Packet Capture for DHCP Troubleshooting

Well I think your on the right track with the packet capture then.
It looks as if the client either never sees the dhcp proposal, or the proposal 
itself is unacceptable to the client. (there are more potential issues, but 
only when there is more than 1 dhcp server)

so for the access list:
access-list 100 permit udp any any eq bootps access-list 100 permit udp any any 
eq bootpc

as a good comparison, you can use working dhcp packet captures on the net 
(search for wireshark sample captures).

cheers




On 21 September 2010 16:58, Jeff Wojciechowski 
 wrote:
> Nope - got a cat3560 doing nothing at all really - just one vlan - no qos - 
> nothing crazy at all.
>
> Clients aren't locked down either (even had the terminals set to
> factory defaults with DHCP)
>
>  And from the switch side - no errors or flaps. And the trouble is the 
> clients in question don't ever fully complete the DHCP transaction upon power 
> up to assign an ip - so they can't connect -but if I assign an IP manually on 
> the client - then they can connect right up to Citrix servers.
>
> That's why I figured get a packet capture to look at.
>
> Thanks,
>
> -Jeff
>
>
> -Original Message-
> From: Heath Jones [mailto:hj1...@gmail.com]
> Sent: Tuesday, September 21, 2010 10:52 AM
> To: Jeff Wojciechowski
> Cc: cisco-nsp@puck.nether.net
> Subject: Re: [c-nsp] Embeded Packet Capture for DHCP Troubleshooting
>
> Is the switch in the middle doing anything 'funky', or just pretty bog 
> standard?
> Are the clients configured to accept any dhcp response, or are they locked 
> down in any way?
>
> If the ethernet port is flapping according to the client, it may make new 
> requests when the link comes back up each time..
> If a client sees a gratuitous arp or packet from another host with the same 
> IP address, it may drop the lease and request a new one..
>
> It sounds like it works though, they get the IP, but just keep asking when 
> they don't need to (before dhcp lease expiry) ??
>
>
>
> This electronic mail (including any attachments) may contain information that 
> is privileged, confidential, or otherwise protected from disclosure to anyone 
> other than its intended recipient(s). Any dissemination or use of this 
> electronic mail or its contents (including any attachments) by persons other 
> than the intended recipient(s) is strictly prohibited. If you have received 
> this message in error, please delete the original message in its entirety 
> (including any attachments) and notify us immediately by reply email so that 
> we may correct our internal records.  Midland Paper Company accepts no 
> responsibility for any loss or damage from use of this electronic mail, 
> including any damage resulting from a computer virus.
>

This electronic mail (including any attachments) may contain information that 
is privileged, confidential, or otherwise protected from disclosure to anyone 
other than its intended recipient(s). Any dissemination or use of this 
electronic mail or its contents (including any attachments) by persons other 
than the intended recipient(s) is strictly prohibited. If you have received 
this message in error, please delete the original message in its entirety 
(including any attachments) and notify us immediately by reply email so that we 
may correct our internal records.  Midland Paper Company accepts no 
responsibility for any loss or damage from use of this electronic mail, 
including any damage resulting from a computer virus.

___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: [c-nsp] Embeded Packet Capture for DHCP Troubleshooting

[Heath Jones]

Well I think your on the right track with the packet capture then.
It looks as if the client either never sees the dhcp proposal, or the
proposal itself is unacceptable to the client. (there are more
potential issues, but only when there is more than 1 dhcp server)

so for the access list:
access-list 100 permit udp any any eq bootps
access-list 100 permit udp any any eq bootpc

as a good comparison, you can use working dhcp packet captures on the
net (search for wireshark sample captures).

cheers




On 21 September 2010 16:58, Jeff Wojciechowski
 wrote:
> Nope - got a cat3560 doing nothing at all really - just one vlan - no qos - 
> nothing crazy at all.
>
> Clients aren't locked down either (even had the terminals set to factory 
> defaults with DHCP)
>
>  And from the switch side - no errors or flaps. And the trouble is the 
> clients in question don't ever fully complete the DHCP transaction upon power 
> up to assign an ip - so they can't connect -but if I assign an IP manually on 
> the client - then they can connect right up to Citrix servers.
>
> That's why I figured get a packet capture to look at.
>
> Thanks,
>
> -Jeff
>
>
> -Original Message-
> From: Heath Jones [mailto:hj1...@gmail.com]
> Sent: Tuesday, September 21, 2010 10:52 AM
> To: Jeff Wojciechowski
> Cc: cisco-nsp@puck.nether.net
> Subject: Re: [c-nsp] Embeded Packet Capture for DHCP Troubleshooting
>
> Is the switch in the middle doing anything 'funky', or just pretty bog 
> standard?
> Are the clients configured to accept any dhcp response, or are they locked 
> down in any way?
>
> If the ethernet port is flapping according to the client, it may make new 
> requests when the link comes back up each time..
> If a client sees a gratuitous arp or packet from another host with the same 
> IP address, it may drop the lease and request a new one..
>
> It sounds like it works though, they get the IP, but just keep asking when 
> they don't need to (before dhcp lease expiry) ??
>
>
>
> This electronic mail (including any attachments) may contain information that 
> is privileged, confidential, or otherwise protected from disclosure to anyone 
> other than its intended recipient(s). Any dissemination or use of this 
> electronic mail or its contents (including any attachments) by persons other 
> than the intended recipient(s) is strictly prohibited. If you have received 
> this message in error, please delete the original message in its entirety 
> (including any attachments) and notify us immediately by reply email so that 
> we may correct our internal records.  Midland Paper Company accepts no 
> responsibility for any loss or damage from use of this electronic mail, 
> including any damage resulting from a computer virus.
>

___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: [c-nsp] Weird Traceroute Issue to Specific Destination

[Gert Doering]

Hi,

On Tue, Sep 21, 2010 at 11:37:41AM -0400, Paul Stewart wrote:
> With that in mind, I'm puzzled . maybe it's simply a matter of "sit back and
> take a good long look" as Gert just suggested ;)

Try "show ip cef".  Sometimes cef gets confused about things (like "oh,
I've heard an ARP reply for this address, must be locally connected").

(This is a fairly old CEF bug, and I'm sure it got fixed long since, but
might be something along that lines)

gert

PS: this thread shows a serious lack of reasonable quoting.
-- 
USENET is *not* the non-clickable part of WWW!
   //www.muc.de/~gert/
Gert Doering - Munich, Germany g...@greenie.muc.de
fax: +49-89-35655025g...@net.informatik.tu-muenchen.de


pgpMqivWAWQCp.pgp
Description: PGP signature
___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: [c-nsp] Embeded Packet Capture for DHCP Troubleshooting

[Jeff Wojciechowski]

Nope - got a cat3560 doing nothing at all really - just one vlan - no qos - 
nothing crazy at all.

Clients aren't locked down either (even had the terminals set to factory 
defaults with DHCP)

 And from the switch side - no errors or flaps. And the trouble is the clients 
in question don't ever fully complete the DHCP transaction upon power up to 
assign an ip - so they can't connect -but if I assign an IP manually on the 
client - then they can connect right up to Citrix servers.

That's why I figured get a packet capture to look at.

Thanks,

-Jeff


-Original Message-
From: Heath Jones [mailto:hj1...@gmail.com]
Sent: Tuesday, September 21, 2010 10:52 AM
To: Jeff Wojciechowski
Cc: cisco-nsp@puck.nether.net
Subject: Re: [c-nsp] Embeded Packet Capture for DHCP Troubleshooting

Is the switch in the middle doing anything 'funky', or just pretty bog standard?
Are the clients configured to accept any dhcp response, or are they locked down 
in any way?

If the ethernet port is flapping according to the client, it may make new 
requests when the link comes back up each time..
If a client sees a gratuitous arp or packet from another host with the same IP 
address, it may drop the lease and request a new one..

It sounds like it works though, they get the IP, but just keep asking when they 
don't need to (before dhcp lease expiry) ??



This electronic mail (including any attachments) may contain information that 
is privileged, confidential, or otherwise protected from disclosure to anyone 
other than its intended recipient(s). Any dissemination or use of this 
electronic mail or its contents (including any attachments) by persons other 
than the intended recipient(s) is strictly prohibited. If you have received 
this message in error, please delete the original message in its entirety 
(including any attachments) and notify us immediately by reply email so that we 
may correct our internal records.  Midland Paper Company accepts no 
responsibility for any loss or damage from use of this electronic mail, 
including any damage resulting from a computer virus.

___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: [c-nsp] Embeded Packet Capture for DHCP Troubleshooting

[Heath Jones]

Is the switch in the middle doing anything 'funky', or just pretty bog standard?
Are the clients configured to accept any dhcp response, or are they
locked down in any way?

If the ethernet port is flapping according to the client, it may make
new requests when the link comes back up each time..
If a client sees a gratuitous arp or packet from another host with the
same IP address, it may drop the lease and request a new one..

It sounds like it works though, they get the IP, but just keep asking
when they don't need to (before dhcp lease expiry) ??



On 21 September 2010 16:31, Jeff Wojciechowski
 wrote:
> Yessir – IPV4.
>
>
>
> Basically the debugging on the router kept showing the IPV4 address being
> offered to the particular client over and over again – that’s why I wanted
> the packet capture to dig into the issue a little deeper.
>
>
>
> Thanks,
>
>
>
> -Jeff
>
>
>
> From: Heath Jones [mailto:hj1...@gmail.com]
> Sent: Tuesday, September 21, 2010 10:28 AM
>
>
>
> It's a L2 & L3 broadcast for requests, then unicast for replies. The clients
> will never have IP addresses before they get a lease from the DHCP server -
> that is what a DHCP lease is (on v4). Are you talking about IPv4?
>
> So your saying that at some point in time the clients are issuing dhcp
> requests, but not seeing replies?
>
>
>
>
>
> On 21 September 2010 16:23, Jeff Wojciechowski
>  wrote:
>
> 2801 is doing dhcp - and I wanted to get a packet capture to see why 2 thin
> clients won't get dhcp leases successfully from the 2801 upon boot up (so
> before the client has an ip from dhcp - isn't the dhcp transaction all arp?)
>
>
> Sent from Midland Paper Company's Blackberry Server
>
>
> From: Heath Jones [mailto:hj1...@gmail.com]
> Sent: Tuesday, September 21, 2010 10:12 AM
> To: Jeff Wojciechowski
> Cc: cisco-nsp@puck.nether.net 
> Subject: Re: [c-nsp] Embeded Packet Capture for DHCP Troubleshooting
>
>
> I just re-read and saw it was the 2601 as the dhcp server.
>
>
>
> What problem are you trying to solve?
>
> On 21 September 2010 16:10, Heath Jones  wrote:
>
> Hi Jeff - which device is performing the dhcp server function and what
> problem are you actually having?
>
>
>
> On 21 September 2010 15:46, Jeff Wojciechowski
>  wrote:
>
> All:
>
> I would like to use Embedded Packet Capture at a remote site to troubleshoot
> IOS DHCP at a remote site.
>
> (referencing
> http://www.cisco.com/en/US/docs/ios/netmgmt/configuration/guide/nm_packet_capture_ps6441_TSD_Products_Configuration_Guide_Chapter.html#wp1060070)
>
> The concept looks pretty straightforward but since DHCP negotiation isn't IP
> - I am having trouble with building the access list to determine what
> traffic to capture.
>
> The device I would like to perform the capture on is a 2801 and that is the
> device serving DHCP to clients at that site.
>
> Utilization on that router is pretty low so if I *had* to capture all
> traffic on F0/0 and dump it via TFTP to the fileserver at the site - I think
> that might be a viable option? But the fileserver is connected to the switch
> that connected to F0/0 that I want to perform the capture on.
>
> Regular DHCP debugs only show that 2 devices are continually offered IPs but
> then apparently keep re-requesting them. (So maybe I am barking up the wrong
> tree here.)
>
> Diagram:
>
> WANSerial-2801-F0/0Switch--DHCP Clients and Fileserver
>
> Any and all hints would be greatly appreciated.
>
> Thanks much,
>
> Jeff Wojciechowski
> LAN, WAN and Telephony Administrator
> Midland Paper Company
> 101 E Palatine Rd
> Wheeling, IL 60090
> * tel: 847.777.2829
> * fax: 847.403.6829
> e-mail:
> jeff.wojciechow...@midlandpaper.com
> http://www.midlandpaper.com
>
>
>
>
>  
> This electronic mail (including any attachments) may contain information
> that is privileged, confidential, or otherwise protected from disclosure to
> anyone other than its intended recipient(s). Any dissemination or use of
> this electronic mail or its contents (including any attachments) by persons
> other than the intended recipient(s) is strictly prohibited. If you have
> received this message in error, please delete the original message in its
> entirety (including any attachments) and notify us immediately by reply
> email so that we may correct our internal records. Midland Paper Company
> accepts no responsibility for any loss or damage from use of this electronic
> mail, including any damage resulting from a computer virus.
> ___
> cisco-nsp mailing list  cisco-...@puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-nsp
> archive at http://puck.nether.net/pipermail/cisco-nsp/
>
>
>
>
>
> 
>
> This electronic mail (including any attachments) may contain information
> that is privileged, confidential, or otherwise protected from disclosure to
> anyone other than its intended recipient(s). Any 

Re: [c-nsp] Weird Traceroute Issue to Specific Destination

[Paul Stewart]

Hehe. yeah, I hear ya.. At first I thought this is just one of those "hey,
dummy look at the routing table"..;)

 

What's killing me is that every hop from the 7200 right to our Internet edge
shows the 0.0.0.0/0 OSPF route as preferred which is what's expected.

 

dis2-rtr-mb#show ip route xx.xxx.2.226

% Network not in table

dis2-rtr-mb#show ip cef xx.xxx.2.226

0.0.0.0/0, version 8684984, epoch 1, cached adjacency xx.xxx.0.226

0 packets, 0 bytes

  via xx.xxx.0.226, Vlan4, 0 dependencies

next hop xx.xxx.0.226, Vlan4

valid cached adjacency

 

 

Paul

 

 

From: Heath Jones [mailto:hj1...@gmail.com] 
Sent: Tuesday, September 21, 2010 11:38 AM
To: Paul Stewart
Cc: cisco-nsp@puck.nether.net
Subject: Re: [c-nsp] Weird Traceroute Issue to Specific Destination

 

I need a coffee or 2, I am misreading absolutely everything today!!

 

Ok so that IP is not the customer IP - it's the destination on the other
side of the net somewhere..

Gert is correct, the routing and forwarding tables will show you what is
different about that ip.

 

 



 

On 21 September 2010 16:23, Heath Jones  wrote:

If my understanding is correct here, then the DSL user is probably blocking
inbound icmp so you would expect the traceroutes you see.. (just constant
timeouts).

Lets take a step back here... What problem is the customer reporting?

 

 



 

On 21 September 2010 16:04, Paul Stewart  wrote:

Yes, loopback is in place and the source . yes, loopback in routing table
(redistributed via OSPF).  This 7206VXR has been in production for over 4
years and we have no issues reaching any other websites.

 

I'm confident that if the remote IP was blocking us or something of that
nature that the traceroute would at least transverse our igp properly . 

 

Thanks,

 

Paul

 

 

From: Heath Jones [mailto:hj1...@gmail.com] 
Sent: Tuesday, September 21, 2010 11:00 AM 


To: Paul Stewart
Cc: cisco-nsp@puck.nether.net
Subject: Re: [c-nsp] Weird Traceroute Issue to Specific Destination

 

If it's not a firewall, its probably routing.. Is the 7206VXR using a
loopback for the source of the icmp request packets, and do you have a route
back to this ip in your igp?



 

On 21 September 2010 15:17, Paul Stewart  wrote:

Thank you - good thinking but I checked and there's nothing in there to
limit ICMP at all..;)

 

Paul

 

 

From: Heath Jones [mailto:hj1...@gmail.com] 
Sent: Tuesday, September 21, 2010 10:05 AM
To: Paul Stewart
Cc: cisco-nsp@puck.nether.net
Subject: Re: [c-nsp] Weird Traceroute Issue to Specific Destination

 

Hi Paul - perhaps you have a firewall filter preventing the ingress icmp
replies (to the 7206VXR)..?



 

On 21 September 2010 14:54, Paul Stewart  wrote:

Hi folks..



We have a customer who is connected over DSL who is having issues getting to
a certain remote site more often than not.  Sometime they can reach this
site, but most of the time they cannot.



They connect to a 7206VXR, which then connects to a 6509 which then
connections to 6509, 6509, then 7606 out to Internet.  Long story short,
there is no reported issues along this connectivity at all and we can only
replicate this complaint to one remote IP address.  Logically, we would push
this back and say "not our problem" which we're confident it's not *but*
there's one strange thing that is bugging me and I can't put logic around
this (I also have a terrible head cold and not thinking straight).



When logged into the 7206VXR where the customer connects via DSL, a
traceroute to the Internet loops normally like this:



acs1-con-bb#traceroute www.cnn.com  

Translating "www.cnn.com  "...domain server
(208.67.222.222) [OK]



Type escape sequence to abort.

Tracing the route to www.cnn.com   (157.166.224.26)



 1 xx.xxx.7.65 0 msec 0 msec 0 msec

 2 xx.xx.120.25 8 msec 8 msec 8 msec

 3 core2-rtr-to-ge4-12-vl4.nexicom.net
  (98.124.0.226) 20 msec 16
msec 88
msec

 4 ge4-0-0.core1.toronto1.nexicom.net
  (98.124.59.17) 16 msec 20 msec
16
msec

 5 ge-6-1-113.car1.Toronto2.Level3.net
  (4.59.180.9) 156 msec 176
msec 88
msec

 6 ae-8-8.ebr1.Chicago1.Level3.net 
(4.69.140.245) 32 msec 32 msec 32 msec

 7 ae-1-100.ebr2.Chicago1.Level3.net
  (4.69.132.42) 32 msec 28 msec
32 msec

 8 ae-3-3.ebr2.Atlanta2.Level3.net 
(4.69.132.74) 52 msec 52 msec 72 msec

 9 ae-2-52.edge4.Atlanta2.Level3.net
  (4.68.103.46) 52 msec 56 msec

   ae-1-51.edge4.Atlanta2.Level3.net
  (4.68.103.14) 48 msec

 10  *  *





With this problem destination IP, our traceroutes look like this:



acs1-con-bb#traceroute xx.xxx.2.226



Type escape sequence to abort.

Tracing th

Re: [c-nsp] Weird Traceroute Issue to Specific Destination

[Heath Jones]

Firstly, i'd ditch the traceroute for now. It opens up paths that
don't need to be walked down.
Are you checking that you have a route, or that the route is correct
to this destination?
Grab the outputs Gert recommended for the 7206VXR, 6500 and also the
device you refer to "back to our location where we can always reach
this website" and post them on here. (both for the trouble IP and some
other public IP like you did cnn.com before).




On 21 September 2010 16:37, Paul Stewart  wrote:
>
> Thank you – the customer is reporting that they cannot reach a particular 
> remote website.  The closest visibility we have is the 7206VXR and 6500 
> physically in the same town – in that town we also have access to a customer 
> network.  On that customer network (fiber fed) they are unable to reach this 
> website neither.
>
>
>
> That location connects back to our location where we can always reach this 
> website.  I have checked each hop along the way and cannot find any null 
> routes etc that would be blocking this traffic.
>
>
>
> So my question is really to confirm – even if the remote site that is 
> unreachable was going something “funky” I should still see a full traceroute 
> at least across the igp correct?
>
> With that in mind, I’m puzzled … maybe it’s simply a matter of “sit back and 
> take a good long look” as Gert just suggested ;)
>
>
>
> Thanks folks …
>
>
>
> Paul
>
>
>
>
>
> From: Heath Jones [mailto:hj1...@gmail.com]
> Sent: Tuesday, September 21, 2010 11:24 AM
>
> To: Paul Stewart
> Cc: cisco-nsp@puck.nether.net
> Subject: Re: [c-nsp] Weird Traceroute Issue to Specific Destination
>
>
>
> If my understanding is correct here, then the DSL user is probably blocking 
> inbound icmp so you would expect the traceroutes you see.. (just constant 
> timeouts).
>
> Lets take a step back here... What problem is the customer reporting?
>
>
>
>
>
>
>
> On 21 September 2010 16:04, Paul Stewart  wrote:
>
> Yes, loopback is in place and the source … yes, loopback in routing table 
> (redistributed via OSPF).  This 7206VXR has been in production for over 4 
> years and we have no issues reaching any other websites.
>
>
>
> I’m confident that if the remote IP was blocking us or something of that 
> nature that the traceroute would at least transverse our igp properly …
>
>
>
> Thanks,
>
>
>
> Paul
>
>
>
>
>
> From: Heath Jones [mailto:hj1...@gmail.com]
> Sent: Tuesday, September 21, 2010 11:00 AM
>
> To: Paul Stewart
> Cc: cisco-nsp@puck.nether.net
> Subject: Re: [c-nsp] Weird Traceroute Issue to Specific Destination
>
>
>
> If it's not a firewall, its probably routing.. Is the 7206VXR using a 
> loopback for the source of the icmp request packets, and do you have a route 
> back to this ip in your igp?
>
>
>
> On 21 September 2010 15:17, Paul Stewart  wrote:
>
> Thank you – good thinking but I checked and there’s nothing in there to limit 
> ICMP at all….;)
>
>
>
> Paul
>
>
>
>
>
> From: Heath Jones [mailto:hj1...@gmail.com]
> Sent: Tuesday, September 21, 2010 10:05 AM
> To: Paul Stewart
> Cc: cisco-nsp@puck.nether.net
> Subject: Re: [c-nsp] Weird Traceroute Issue to Specific Destination
>
>
>
> Hi Paul - perhaps you have a firewall filter preventing the ingress icmp 
> replies (to the 7206VXR)..?
>
>
>
> On 21 September 2010 14:54, Paul Stewart  wrote:
>
> Hi folks..
>
>
>
> We have a customer who is connected over DSL who is having issues getting to
> a certain remote site more often than not.  Sometime they can reach this
> site, but most of the time they cannot.
>
>
>
> They connect to a 7206VXR, which then connects to a 6509 which then
> connections to 6509, 6509, then 7606 out to Internet.  Long story short,
> there is no reported issues along this connectivity at all and we can only
> replicate this complaint to one remote IP address.  Logically, we would push
> this back and say "not our problem" which we're confident it's not *but*
> there's one strange thing that is bugging me and I can't put logic around
> this (I also have a terrible head cold and not thinking straight).
>
>
>
> When logged into the 7206VXR where the customer connects via DSL, a
> traceroute to the Internet loops normally like this:
>
>
>
> acs1-con-bb#traceroute www.cnn.com
>
> Translating "www.cnn.com"...domain server (208.67.222.222) [OK]
>
>
>
> Type escape sequence to abort.
>
> Tracing the route to www.cnn.com (157.166.224.26)
>
>
>
>  1 xx.xxx.7.65 0 msec 0 msec 0 msec
>
>  2 xx.xx.120.25 8 msec 8 msec 8 msec
>
>  3 core2-rtr-to-ge4-12-vl4.nexicom.net (98.124.0.226) 20 msec 16 msec 88
> msec
>
>  4 ge4-0-0.core1.toronto1.nexicom.net (98.124.59.17) 16 msec 20 msec 16
> msec
>
>  5 ge-6-1-113.car1.Toronto2.Level3.net (4.59.180.9) 156 msec 176 msec 88
> msec
>
>  6 ae-8-8.ebr1.Chicago1.Level3.net (4.69.140.245) 32 msec 32 msec 32 msec
>
>  7 ae-1-100.ebr2.Chicago1.Level3.net (4.69.132.42) 32 msec 28 msec 32 msec
>
>  8 ae-3-3.ebr2.Atlanta2.Level3.net (4.69.132.74) 52 msec 52 msec 72 msec
>
>  9 ae-2-52.edge4.Atlanta2.Level3.

Re: [c-nsp] Embeded Packet Capture for DHCP Troubleshooting

[Ryan West]

> -Original Message-
> From: cisco-nsp-boun...@puck.nether.net [mailto:cisco-nsp-
> boun...@puck.nether.net] On Behalf Of Jeff Wojciechowski
> Sent: Tuesday, September 21, 2010 11:24 AM
> To: 'hj1...@gmail.com'
> Cc: 'cisco-nsp@puck.nether.net'
> Subject: Re: [c-nsp] Embeded Packet Capture for DHCP Troubleshooting
> 
> 2801 is doing dhcp - and I wanted to get a packet capture to see why 2 thin
> clients won't get dhcp leases successfully from the 2801 upon boot up (so
> before the client has an ip from dhcp - isn't the dhcp transaction all arp?)
> 
> 

No, it's a broadcast UDP on port 67 from the client.

-ryan

___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: [c-nsp] Weird Traceroute Issue to Specific Destination

[Heath Jones]

I need a coffee or 2, I am misreading absolutely everything today!!

Ok so that IP is not the customer IP - it's the destination on the other
side of the net somewhere..
Gert is correct, the routing and forwarding tables will show you what is
different about that ip.





On 21 September 2010 16:23, Heath Jones  wrote:

>  If my understanding is correct here, then the DSL user is probably
> blocking inbound icmp so you would expect the traceroutes you see.. (just
> constant timeouts).
> Lets take a step back here... What problem is the customer reporting?
>
>
>
>
>
> On 21 September 2010 16:04, Paul Stewart  wrote:
>
>>  Yes, loopback is in place and the source … yes, loopback in routing
>> table (redistributed via OSPF).  This 7206VXR has been in production for
>> over 4 years and we have no issues reaching any other websites.
>>
>>
>>
>> I’m confident that if the remote IP was blocking us or something of that
>> nature that the traceroute would at least transverse our igp properly …
>>
>>
>>
>> Thanks,
>>
>>
>>
>> Paul
>>
>>
>>
>>
>>
>> *From:* Heath Jones [mailto:hj1...@gmail.com]
>> *Sent:* Tuesday, September 21, 2010 11:00 AM
>>
>> *To:* Paul Stewart
>> *Cc:* cisco-nsp@puck.nether.net
>> *Subject:* Re: [c-nsp] Weird Traceroute Issue to Specific Destination
>>
>>
>>
>> If it's not a firewall, its probably routing.. Is the 7206VXR using a
>> loopback for the source of the icmp request packets, and do you have a route
>> back to this ip in your igp?
>>
>>
>>
>>
>>
>> On 21 September 2010 15:17, Paul Stewart  wrote:
>>
>> Thank you – good thinking but I checked and there’s nothing in there to
>> limit ICMP at all….;)
>>
>>
>>
>> Paul
>>
>>
>>
>>
>>
>> *From:* Heath Jones [mailto:hj1...@gmail.com]
>> *Sent:* Tuesday, September 21, 2010 10:05 AM
>> *To:* Paul Stewart
>> *Cc:* cisco-nsp@puck.nether.net
>> *Subject:* Re: [c-nsp] Weird Traceroute Issue to Specific Destination
>>
>>
>>
>> Hi Paul - perhaps you have a firewall filter preventing the ingress icmp
>> replies (to the 7206VXR)..?
>>
>>
>>
>>
>>
>> On 21 September 2010 14:54, Paul Stewart  wrote:
>>
>> Hi folks..
>>
>>
>>
>> We have a customer who is connected over DSL who is having issues getting
>> to
>> a certain remote site more often than not.  Sometime they can reach this
>> site, but most of the time they cannot.
>>
>>
>>
>> They connect to a 7206VXR, which then connects to a 6509 which then
>> connections to 6509, 6509, then 7606 out to Internet.  Long story short,
>> there is no reported issues along this connectivity at all and we can only
>> replicate this complaint to one remote IP address.  Logically, we would
>> push
>> this back and say "not our problem" which we're confident it's not *but*
>> there's one strange thing that is bugging me and I can't put logic around
>> this (I also have a terrible head cold and not thinking straight).
>>
>>
>>
>> When logged into the 7206VXR where the customer connects via DSL, a
>> traceroute to the Internet loops normally like this:
>>
>>
>>
>> acs1-con-bb#traceroute www.cnn.com
>>
>> Translating "www.cnn.com"...domain server (208.67.222.222) [OK]
>>
>>
>>
>> Type escape sequence to abort.
>>
>> Tracing the route to www.cnn.com (157.166.224.26)
>>
>>
>>
>>  1 xx.xxx.7.65 0 msec 0 msec 0 msec
>>
>>  2 xx.xx.120.25 8 msec 8 msec 8 msec
>>
>>  3 core2-rtr-to-ge4-12-vl4.nexicom.net (98.124.0.226) 20 msec 16 msec 88
>> msec
>>
>>  4 ge4-0-0.core1.toronto1.nexicom.net (98.124.59.17) 16 msec 20 msec 16
>> msec
>>
>>  5 
>> ge-6-1-113.car1.Toronto2.Level3.net(4.59.180.9)
>>  156 msec 176 msec 88
>> msec
>>
>>  6 
>> ae-8-8.ebr1.Chicago1.Level3.net(4.69.140.245)
>>  32 msec 32 msec 32 msec
>>
>>  7 
>> ae-1-100.ebr2.Chicago1.Level3.net(4.69.132.42)
>>  32 msec 28 msec 32 msec
>>
>>  8 
>> ae-3-3.ebr2.Atlanta2.Level3.net(4.69.132.74)
>>  52 msec 52 msec 72 msec
>>
>>  9 
>> ae-2-52.edge4.Atlanta2.Level3.net(4.68.103.46)
>>  52 msec 56 msec
>>
>>
>> ae-1-51.edge4.Atlanta2.Level3.net(4.68.103.14)
>>  48 msec
>>
>>  10  *  *
>>
>>
>>
>>
>>
>> With this problem destination IP, our traceroutes look like this:
>>
>>
>>
>> acs1-con-bb#traceroute xx.xxx.2.226
>>
>>
>>
>> Type escape sequence to abort.
>>
>> Tracing the route to xxx.xxx.xxx (xx.xxx.2.226)
>>
>>
>>
>>  1 xx.xxx.7.65 0 msec 0 msec 0 msec
>>
>>  2  *  *  *
>>
>>  3  *  *  *
>>
>>  4  *  *  *
>>
>>
>>
>>
>>
>> So, normally with all "*" it would indicate simply that the route is
>> unreachable. but that's the part that has me puzzled.  The route is
>> reachable across the Internet *and* at the very least, I should see a
>> traceroute going to the fourth hop no matter what.  We run OSPF across
>> these
>> links and have a "default-originate" statement on the 4th hop router.
>>  This
>> is confirmed via t

Re: [c-nsp] Weird Traceroute Issue to Specific Destination

[Paul Stewart]

Thank you - the customer is reporting that they cannot reach a particular
remote website.  The closest visibility we have is the 7206VXR and 6500
physically in the same town - in that town we also have access to a customer
network.  On that customer network (fiber fed) they are unable to reach this
website neither.

 

That location connects back to our location where we can always reach this
website.  I have checked each hop along the way and cannot find any null
routes etc that would be blocking this traffic.

 

So my question is really to confirm - even if the remote site that is
unreachable was going something "funky" I should still see a full traceroute
at least across the igp correct?  

With that in mind, I'm puzzled . maybe it's simply a matter of "sit back and
take a good long look" as Gert just suggested ;)

 

Thanks folks .

 

Paul

 

 

From: Heath Jones [mailto:hj1...@gmail.com] 
Sent: Tuesday, September 21, 2010 11:24 AM
To: Paul Stewart
Cc: cisco-nsp@puck.nether.net
Subject: Re: [c-nsp] Weird Traceroute Issue to Specific Destination

 

If my understanding is correct here, then the DSL user is probably blocking
inbound icmp so you would expect the traceroutes you see.. (just constant
timeouts).

Lets take a step back here... What problem is the customer reporting?

 

 



 

On 21 September 2010 16:04, Paul Stewart  wrote:

Yes, loopback is in place and the source . yes, loopback in routing table
(redistributed via OSPF).  This 7206VXR has been in production for over 4
years and we have no issues reaching any other websites.

 

I'm confident that if the remote IP was blocking us or something of that
nature that the traceroute would at least transverse our igp properly . 

 

Thanks,

 

Paul

 

 

From: Heath Jones [mailto:hj1...@gmail.com] 
Sent: Tuesday, September 21, 2010 11:00 AM 


To: Paul Stewart
Cc: cisco-nsp@puck.nether.net
Subject: Re: [c-nsp] Weird Traceroute Issue to Specific Destination

 

If it's not a firewall, its probably routing.. Is the 7206VXR using a
loopback for the source of the icmp request packets, and do you have a route
back to this ip in your igp?



 

On 21 September 2010 15:17, Paul Stewart  wrote:

Thank you - good thinking but I checked and there's nothing in there to
limit ICMP at all..;)

 

Paul

 

 

From: Heath Jones [mailto:hj1...@gmail.com] 
Sent: Tuesday, September 21, 2010 10:05 AM
To: Paul Stewart
Cc: cisco-nsp@puck.nether.net
Subject: Re: [c-nsp] Weird Traceroute Issue to Specific Destination

 

Hi Paul - perhaps you have a firewall filter preventing the ingress icmp
replies (to the 7206VXR)..?



 

On 21 September 2010 14:54, Paul Stewart  wrote:

Hi folks..



We have a customer who is connected over DSL who is having issues getting to
a certain remote site more often than not.  Sometime they can reach this
site, but most of the time they cannot.



They connect to a 7206VXR, which then connects to a 6509 which then
connections to 6509, 6509, then 7606 out to Internet.  Long story short,
there is no reported issues along this connectivity at all and we can only
replicate this complaint to one remote IP address.  Logically, we would push
this back and say "not our problem" which we're confident it's not *but*
there's one strange thing that is bugging me and I can't put logic around
this (I also have a terrible head cold and not thinking straight).



When logged into the 7206VXR where the customer connects via DSL, a
traceroute to the Internet loops normally like this:



acs1-con-bb#traceroute www.cnn.com  

Translating "www.cnn.com  "...domain server
(208.67.222.222) [OK]



Type escape sequence to abort.

Tracing the route to www.cnn.com   (157.166.224.26)



 1 xx.xxx.7.65 0 msec 0 msec 0 msec

 2 xx.xx.120.25 8 msec 8 msec 8 msec

 3 core2-rtr-to-ge4-12-vl4.nexicom.net
  (98.124.0.226) 20 msec 16
msec 88
msec

 4 ge4-0-0.core1.toronto1.nexicom.net
  (98.124.59.17) 16 msec 20 msec
16
msec

 5 ge-6-1-113.car1.Toronto2.Level3.net
  (4.59.180.9) 156 msec 176
msec 88
msec

 6 ae-8-8.ebr1.Chicago1.Level3.net 
(4.69.140.245) 32 msec 32 msec 32 msec

 7 ae-1-100.ebr2.Chicago1.Level3.net
  (4.69.132.42) 32 msec 28 msec
32 msec

 8 ae-3-3.ebr2.Atlanta2.Level3.net 
(4.69.132.74) 52 msec 52 msec 72 msec

 9 ae-2-52.edge4.Atlanta2.Level3.net
  (4.68.103.46) 52 msec 56 msec

   ae-1-51.edge4.Atlanta2.Level3.net
  (4.68.103.14) 48 msec

 10  *  *





With this problem destination IP, our traceroutes look like this:



acs1-con-bb#traceroute xx.xxx.2.226



Type escape sequence to abort.

Tracing the route to xxx.xxx.xxx (xx.xxx.2.226)



 1 xx.x

Re: [c-nsp] Embeded Packet Capture for DHCP Troubleshooting

[Jeff Wojciechowski]

Yessir - IPV4.

Basically the debugging on the router kept showing the IPV4 address being 
offered to the particular client over and over again - that's why I wanted the 
packet capture to dig into the issue a little deeper.

Thanks,

-Jeff

From: Heath Jones [mailto:hj1...@gmail.com]
Sent: Tuesday, September 21, 2010 10:28 AM
To: Jeff Wojciechowski
Cc: cisco-nsp@puck.nether.net
Subject: Re: [c-nsp] Embeded Packet Capture for DHCP Troubleshooting

It's a L2 & L3 broadcast for requests, then unicast for replies. The clients 
will never have IP addresses before they get a lease from the DHCP server - 
that is what a DHCP lease is (on v4). Are you talking about IPv4?
So your saying that at some point in time the clients are issuing dhcp 
requests, but not seeing replies?



On 21 September 2010 16:23, Jeff Wojciechowski 
mailto:jeff.wojciechow...@midlandpaper.com>>
 wrote:
2801 is doing dhcp - and I wanted to get a packet capture to see why 2 thin 
clients won't get dhcp leases successfully from the 2801 upon boot up (so 
before the client has an ip from dhcp - isn't the dhcp transaction all arp?)


Sent from Midland Paper Company's Blackberry Server

From: Heath Jones [mailto:hj1...@gmail.com]
Sent: Tuesday, September 21, 2010 10:12 AM
To: Jeff Wojciechowski
Cc: cisco-nsp@puck.nether.net 
mailto:cisco-nsp@puck.nether.net>>
Subject: Re: [c-nsp] Embeded Packet Capture for DHCP Troubleshooting

I just re-read and saw it was the 2601 as the dhcp server.

What problem are you trying to solve?
On 21 September 2010 16:10, Heath Jones 
mailto:hj1...@gmail.com>> wrote:
Hi Jeff - which device is performing the dhcp server function and what problem 
are you actually having?



On 21 September 2010 15:46, Jeff Wojciechowski 
mailto:jeff.wojciechow...@midlandpaper.com>>
 wrote:
All:

I would like to use Embedded Packet Capture at a remote site to troubleshoot 
IOS DHCP at a remote site.

(referencing 
http://www.cisco.com/en/US/docs/ios/netmgmt/configuration/guide/nm_packet_capture_ps6441_TSD_Products_Configuration_Guide_Chapter.html#wp1060070)

The concept looks pretty straightforward but since DHCP negotiation isn't IP - 
I am having trouble with building the access list to determine what traffic to 
capture.

The device I would like to perform the capture on is a 2801 and that is the 
device serving DHCP to clients at that site.

Utilization on that router is pretty low so if I *had* to capture all traffic 
on F0/0 and dump it via TFTP to the fileserver at the site - I think that might 
be a viable option? But the fileserver is connected to the switch that 
connected to F0/0 that I want to perform the capture on.

Regular DHCP debugs only show that 2 devices are continually offered IPs but 
then apparently keep re-requesting them. (So maybe I am barking up the wrong 
tree here.)

Diagram:

WANSerial-2801-F0/0Switch--DHCP Clients and Fileserver

Any and all hints would be greatly appreciated.

Thanks much,

Jeff Wojciechowski
LAN, WAN and Telephony Administrator
Midland Paper Company
101 E Palatine Rd
Wheeling, IL 60090
* tel: 847.777.2829
* fax: 847.403.6829
e-mail: 
jeff.wojciechow...@midlandpaper.com>
http://www.midlandpaper.com




 
This electronic mail (including any attachments) may contain information that 
is privileged, confidential, or otherwise protected from disclosure to anyone 
other than its intended recipient(s). Any dissemination or use of this 
electronic mail or its contents (including any attachments) by persons other 
than the intended recipient(s) is strictly prohibited. If you have received 
this message in error, please delete the original message in its entirety 
(including any attachments) and notify us immediately by reply email so that we 
may correct our internal records. Midland Paper Company accepts no 
responsibility for any loss or damage from use of this electronic mail, 
including any damage resulting from a computer virus.
___
cisco-nsp mailing list  
cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/



This electronic mail (including any attachments) may contain information that 
is privileged, confidential, or otherwise protected from disclosure to anyone 
other than its intended recipient(s). Any dissemination or use of this 
electronic mail or its contents (including any attachments) by persons other 
than the intended recipient(s) is strictly prohibited. If you have received 
this message in error, please delete the original message in its entirety 
(including any attachments) and notify us immediately by reply email so that we 
m

Re: [c-nsp] Weird Traceroute Issue to Specific Destination

[Gert Doering]

Hi,

On Tue, Sep 21, 2010 at 11:04:38AM -0400, Paul Stewart wrote:
> I'm confident that if the remote IP was blocking us or something of that
> nature that the traceroute would at least transverse our igp properly . 

local null route ("something left in the config that directs this IP
into a black hole")?

> We have a customer who is connected over DSL who is having issues getting to
> a certain remote site more often than not.  Sometime they can reach this
> site, but most of the time they cannot.

... but that wouldn't actually explain that "sometimes it works".

What I'd do:

 - on the first and second hop routers, try grabbing "show ip route",
   and "show ip cef" for that target IP

 - stare at it, and wonder :-)

gert
-- 
USENET is *not* the non-clickable part of WWW!
   //www.muc.de/~gert/
Gert Doering - Munich, Germany g...@greenie.muc.de
fax: +49-89-35655025g...@net.informatik.tu-muenchen.de


pgp9onuxEEIL2.pgp
Description: PGP signature
___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: [c-nsp] Embeded Packet Capture for DHCP Troubleshooting

[Heath Jones]

It's a L2 & L3 broadcast for requests, then unicast for replies. The clients
will never have IP addresses before they get a lease from the DHCP server -
that is what a DHCP lease is (on v4). Are you talking about IPv4?
So your saying that at some point in time the clients are issuing dhcp
requests, but not seeing replies?



On 21 September 2010 16:23, Jeff Wojciechowski <
jeff.wojciechow...@midlandpaper.com> wrote:

> 2801 is doing dhcp - and I wanted to get a packet capture to see why 2 thin
> clients won't get dhcp leases successfully from the 2801 upon boot up (so
> before the client has an ip from dhcp - isn't the dhcp transaction all arp?)
>
>
> Sent from Midland Paper Company's Blackberry Server
>
> *From*: Heath Jones [mailto:hj1...@gmail.com]
> *Sent*: Tuesday, September 21, 2010 10:12 AM
> *To*: Jeff Wojciechowski
> *Cc*: cisco-nsp@puck.nether.net 
> *Subject*: Re: [c-nsp] Embeded Packet Capture for DHCP Troubleshooting
>
>   I just re-read and saw it was the 2601 as the dhcp server.
>
> What problem are you trying to solve?
>
> On 21 September 2010 16:10, Heath Jones  wrote:
>
>> Hi Jeff - which device is performing the dhcp server function and what
>> problem are you actually having?
>>
>>
>>
>> On 21 September 2010 15:46, Jeff Wojciechowski <
>> jeff.wojciechow...@midlandpaper.com> wrote:
>>
>>> All:
>>>
>>> I would like to use Embedded Packet Capture at a remote site to
>>> troubleshoot IOS DHCP at a remote site.
>>>
>>> (referencing
>>> http://www.cisco.com/en/US/docs/ios/netmgmt/configuration/guide/nm_packet_capture_ps6441_TSD_Products_Configuration_Guide_Chapter.html#wp1060070
>>> )
>>>
>>> The concept looks pretty straightforward but since DHCP negotiation isn't
>>> IP - I am having trouble with building the access list to determine what
>>> traffic to capture.
>>>
>>> The device I would like to perform the capture on is a 2801 and that is
>>> the device serving DHCP to clients at that site.
>>>
>>> Utilization on that router is pretty low so if I *had* to capture all
>>> traffic on F0/0 and dump it via TFTP to the fileserver at the site - I think
>>> that might be a viable option? But the fileserver is connected to the switch
>>> that connected to F0/0 that I want to perform the capture on.
>>>
>>> Regular DHCP debugs only show that 2 devices are continually offered IPs
>>> but then apparently keep re-requesting them. (So maybe I am barking up the
>>> wrong tree here.)
>>>
>>> Diagram:
>>>
>>> WANSerial-2801-F0/0Switch--DHCP Clients and Fileserver
>>>
>>> Any and all hints would be greatly appreciated.
>>>
>>> Thanks much,
>>>
>>> Jeff Wojciechowski
>>> LAN, WAN and Telephony Administrator
>>> Midland Paper Company
>>> 101 E Palatine Rd
>>> Wheeling, IL 60090
>>> * tel: 847.777.2829
>>> * fax: 847.403.6829
>>> e-mail: jeff.wojciechow...@midlandpaper.com>> jeff.wojciechow...@midlandpaper.com>
>>> http://www.midlandpaper.com
>>>
>>>
>>>
>>>
>>>  
>>> This electronic mail (including any attachments) may contain information
>>> that is privileged, confidential, or otherwise protected from disclosure to
>>> anyone other than its intended recipient(s). Any dissemination or use of
>>> this electronic mail or its contents (including any attachments) by persons
>>> other than the intended recipient(s) is strictly prohibited. If you have
>>> received this message in error, please delete the original message in its
>>> entirety (including any attachments) and notify us immediately by reply
>>> email so that we may correct our internal records. Midland Paper Company
>>> accepts no responsibility for any loss or damage from use of this electronic
>>> mail, including any damage resulting from a computer virus.
>>> ___
>>> cisco-nsp mailing list  cisco-nsp@puck.nether.net
>>> https://puck.nether.net/mailman/listinfo/cisco-nsp
>>> archive at http://puck.nether.net/pipermail/cisco-nsp/
>>>
>>
>>
>
> --
> This electronic mail (including any attachments) may contain information
> that is privileged, confidential, or otherwise protected from disclosure to
> anyone other than its intended recipient(s). Any dissemination or use of
> this electronic mail or its contents (including any attachments) by persons
> other than the intended recipient(s) is strictly prohibited. If you have
> received this message in error, please delete the original message in its
> entirety (including any attachments) and notify us immediately by reply
> email so that we may correct our internal records. Midland Paper Company
> accepts no responsibility for any loss or damage from use of this electronic
> mail, including any damage resulting from a computer virus.
>
___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: [c-nsp] Embeded Packet Capture for DHCP Troubleshooting

[Jeff Wojciechowski]

2801 is doing dhcp - and I wanted to get a packet capture to see why 2 thin 
clients won't get dhcp leases successfully from the 2801 upon boot up (so 
before the client has an ip from dhcp - isn't the dhcp transaction all arp?)


Sent from Midland Paper Company's Blackberry Server

From: Heath Jones [mailto:hj1...@gmail.com]
Sent: Tuesday, September 21, 2010 10:12 AM
To: Jeff Wojciechowski
Cc: cisco-nsp@puck.nether.net 
Subject: Re: [c-nsp] Embeded Packet Capture for DHCP Troubleshooting

I just re-read and saw it was the 2601 as the dhcp server.

What problem are you trying to solve?

On 21 September 2010 16:10, Heath Jones 
mailto:hj1...@gmail.com>> wrote:
Hi Jeff - which device is performing the dhcp server function and what problem 
are you actually having?



On 21 September 2010 15:46, Jeff Wojciechowski 
mailto:jeff.wojciechow...@midlandpaper.com>>
 wrote:
All:

I would like to use Embedded Packet Capture at a remote site to troubleshoot 
IOS DHCP at a remote site.

(referencing 
http://www.cisco.com/en/US/docs/ios/netmgmt/configuration/guide/nm_packet_capture_ps6441_TSD_Products_Configuration_Guide_Chapter.html#wp1060070)

The concept looks pretty straightforward but since DHCP negotiation isn't IP - 
I am having trouble with building the access list to determine what traffic to 
capture.

The device I would like to perform the capture on is a 2801 and that is the 
device serving DHCP to clients at that site.

Utilization on that router is pretty low so if I *had* to capture all traffic 
on F0/0 and dump it via TFTP to the fileserver at the site - I think that might 
be a viable option? But the fileserver is connected to the switch that 
connected to F0/0 that I want to perform the capture on.

Regular DHCP debugs only show that 2 devices are continually offered IPs but 
then apparently keep re-requesting them. (So maybe I am barking up the wrong 
tree here.)

Diagram:

WANSerial-2801-F0/0Switch--DHCP Clients and Fileserver

Any and all hints would be greatly appreciated.

Thanks much,

Jeff Wojciechowski
LAN, WAN and Telephony Administrator
Midland Paper Company
101 E Palatine Rd
Wheeling, IL 60090
* tel: 847.777.2829
* fax: 847.403.6829
e-mail: 
jeff.wojciechow...@midlandpaper.com>
http://www.midlandpaper.com




 
This electronic mail (including any attachments) may contain information that 
is privileged, confidential, or otherwise protected from disclosure to anyone 
other than its intended recipient(s). Any dissemination or use of this 
electronic mail or its contents (including any attachments) by persons other 
than the intended recipient(s) is strictly prohibited. If you have received 
this message in error, please delete the original message in its entirety 
(including any attachments) and notify us immediately by reply email so that we 
may correct our internal records. Midland Paper Company accepts no 
responsibility for any loss or damage from use of this electronic mail, 
including any damage resulting from a computer virus.
___
cisco-nsp mailing list  
cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/




This electronic mail (including any attachments) may contain information that 
is privileged, confidential, or otherwise protected from disclosure to anyone 
other than its intended recipient(s). Any dissemination or use of this 
electronic mail or its contents (including any attachments) by persons other 
than the intended recipient(s) is strictly prohibited. If you have received 
this message in error, please delete the original message in its entirety 
(including any attachments) and notify us immediately by reply email so that we 
may correct our internal records. Midland Paper Company accepts no 
responsibility for any loss or damage from use of this electronic mail, 
including any damage resulting from a computer virus.
___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: [c-nsp] Weird Traceroute Issue to Specific Destination

[Heath Jones]

 If my understanding is correct here, then the DSL user is probably blocking
inbound icmp so you would expect the traceroutes you see.. (just constant
timeouts).
Lets take a step back here... What problem is the customer reporting?





On 21 September 2010 16:04, Paul Stewart  wrote:

>  Yes, loopback is in place and the source … yes, loopback in routing table
> (redistributed via OSPF).  This 7206VXR has been in production for over 4
> years and we have no issues reaching any other websites.
>
>
>
> I’m confident that if the remote IP was blocking us or something of that
> nature that the traceroute would at least transverse our igp properly …
>
>
>
> Thanks,
>
>
>
> Paul
>
>
>
>
>
> *From:* Heath Jones [mailto:hj1...@gmail.com]
> *Sent:* Tuesday, September 21, 2010 11:00 AM
>
> *To:* Paul Stewart
> *Cc:* cisco-nsp@puck.nether.net
> *Subject:* Re: [c-nsp] Weird Traceroute Issue to Specific Destination
>
>
>
> If it's not a firewall, its probably routing.. Is the 7206VXR using a
> loopback for the source of the icmp request packets, and do you have a route
> back to this ip in your igp?
>
>
>
>
>
> On 21 September 2010 15:17, Paul Stewart  wrote:
>
> Thank you – good thinking but I checked and there’s nothing in there to
> limit ICMP at all….;)
>
>
>
> Paul
>
>
>
>
>
> *From:* Heath Jones [mailto:hj1...@gmail.com]
> *Sent:* Tuesday, September 21, 2010 10:05 AM
> *To:* Paul Stewart
> *Cc:* cisco-nsp@puck.nether.net
> *Subject:* Re: [c-nsp] Weird Traceroute Issue to Specific Destination
>
>
>
> Hi Paul - perhaps you have a firewall filter preventing the ingress icmp
> replies (to the 7206VXR)..?
>
>
>
>
>
> On 21 September 2010 14:54, Paul Stewart  wrote:
>
> Hi folks..
>
>
>
> We have a customer who is connected over DSL who is having issues getting
> to
> a certain remote site more often than not.  Sometime they can reach this
> site, but most of the time they cannot.
>
>
>
> They connect to a 7206VXR, which then connects to a 6509 which then
> connections to 6509, 6509, then 7606 out to Internet.  Long story short,
> there is no reported issues along this connectivity at all and we can only
> replicate this complaint to one remote IP address.  Logically, we would
> push
> this back and say "not our problem" which we're confident it's not *but*
> there's one strange thing that is bugging me and I can't put logic around
> this (I also have a terrible head cold and not thinking straight).
>
>
>
> When logged into the 7206VXR where the customer connects via DSL, a
> traceroute to the Internet loops normally like this:
>
>
>
> acs1-con-bb#traceroute www.cnn.com
>
> Translating "www.cnn.com"...domain server (208.67.222.222) [OK]
>
>
>
> Type escape sequence to abort.
>
> Tracing the route to www.cnn.com (157.166.224.26)
>
>
>
>  1 xx.xxx.7.65 0 msec 0 msec 0 msec
>
>  2 xx.xx.120.25 8 msec 8 msec 8 msec
>
>  3 core2-rtr-to-ge4-12-vl4.nexicom.net (98.124.0.226) 20 msec 16 msec 88
> msec
>
>  4 ge4-0-0.core1.toronto1.nexicom.net (98.124.59.17) 16 msec 20 msec 16
> msec
>
>  5 
> ge-6-1-113.car1.Toronto2.Level3.net(4.59.180.9)
>  156 msec 176 msec 88
> msec
>
>  6 
> ae-8-8.ebr1.Chicago1.Level3.net(4.69.140.245)
>  32 msec 32 msec 32 msec
>
>  7 
> ae-1-100.ebr2.Chicago1.Level3.net(4.69.132.42)
>  32 msec 28 msec 32 msec
>
>  8 
> ae-3-3.ebr2.Atlanta2.Level3.net(4.69.132.74)
>  52 msec 52 msec 72 msec
>
>  9 
> ae-2-52.edge4.Atlanta2.Level3.net(4.68.103.46)
>  52 msec 56 msec
>
>
> ae-1-51.edge4.Atlanta2.Level3.net(4.68.103.14)
>  48 msec
>
>  10  *  *
>
>
>
>
>
> With this problem destination IP, our traceroutes look like this:
>
>
>
> acs1-con-bb#traceroute xx.xxx.2.226
>
>
>
> Type escape sequence to abort.
>
> Tracing the route to xxx.xxx.xxx (xx.xxx.2.226)
>
>
>
>  1 xx.xxx.7.65 0 msec 0 msec 0 msec
>
>  2  *  *  *
>
>  3  *  *  *
>
>  4  *  *  *
>
>
>
>
>
> So, normally with all "*" it would indicate simply that the route is
> unreachable. but that's the part that has me puzzled.  The route is
> reachable across the Internet *and* at the very least, I should see a
> traceroute going to the fourth hop no matter what.  We run OSPF across
> these
> links and have a "default-originate" statement on the 4th hop router.  This
> is confirmed via the routing table entry for 0.0.0.0
>
>
>
> So what makes this particular destination IP act differently than any other
> within our IGP is my question??
>
>
>
> Cheers,
>
>
>
> Paul
>
>
>
> ___
> cisco-nsp mailing list  cisco-nsp@puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-nsp
> archive at http://puck.nether.net/pipermail/cisco-nsp/
>
>
>
>
>
___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.n

Re: [c-nsp] Embeded Packet Capture for DHCP Troubleshooting

[Heath Jones]

I just re-read and saw it was the 2601 as the dhcp server.

What problem are you trying to solve?

On 21 September 2010 16:10, Heath Jones  wrote:

> Hi Jeff - which device is performing the dhcp server function and what
> problem are you actually having?
>
>
>
> On 21 September 2010 15:46, Jeff Wojciechowski <
> jeff.wojciechow...@midlandpaper.com> wrote:
>
>> All:
>>
>> I would like to use Embedded Packet Capture at a remote site to
>> troubleshoot IOS DHCP at a remote site.
>>
>> (referencing
>> http://www.cisco.com/en/US/docs/ios/netmgmt/configuration/guide/nm_packet_capture_ps6441_TSD_Products_Configuration_Guide_Chapter.html#wp1060070
>> )
>>
>> The concept looks pretty straightforward but since DHCP negotiation isn't
>> IP - I am having trouble with building the access list to determine what
>> traffic to capture.
>>
>> The device I would like to perform the capture on is a 2801 and that is
>> the device serving DHCP to clients at that site.
>>
>> Utilization on that router is pretty low so if I *had* to capture all
>> traffic on F0/0 and dump it via TFTP to the fileserver at the site - I think
>> that might be a viable option? But the fileserver is connected to the switch
>> that connected to F0/0 that I want to perform the capture on.
>>
>> Regular DHCP debugs only show that 2 devices are continually offered IPs
>> but then apparently keep re-requesting them. (So maybe I am barking up the
>> wrong tree here.)
>>
>> Diagram:
>>
>> WANSerial-2801-F0/0Switch--DHCP Clients and Fileserver
>>
>> Any and all hints would be greatly appreciated.
>>
>> Thanks much,
>>
>> Jeff Wojciechowski
>> LAN, WAN and Telephony Administrator
>> Midland Paper Company
>> 101 E Palatine Rd
>> Wheeling, IL 60090
>> * tel: 847.777.2829
>> * fax: 847.403.6829
>> e-mail: jeff.wojciechow...@midlandpaper.com> jeff.wojciechow...@midlandpaper.com>
>> http://www.midlandpaper.com
>>
>>
>>
>>
>>  
>> This electronic mail (including any attachments) may contain information
>> that is privileged, confidential, or otherwise protected from disclosure to
>> anyone other than its intended recipient(s). Any dissemination or use of
>> this electronic mail or its contents (including any attachments) by persons
>> other than the intended recipient(s) is strictly prohibited. If you have
>> received this message in error, please delete the original message in its
>> entirety (including any attachments) and notify us immediately by reply
>> email so that we may correct our internal records. Midland Paper Company
>> accepts no responsibility for any loss or damage from use of this electronic
>> mail, including any damage resulting from a computer virus.
>> ___
>> cisco-nsp mailing list  cisco-nsp@puck.nether.net
>> https://puck.nether.net/mailman/listinfo/cisco-nsp
>> archive at http://puck.nether.net/pipermail/cisco-nsp/
>>
>
>
___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: [c-nsp] Embeded Packet Capture for DHCP Troubleshooting

[Heath Jones]

Hi Jeff - which device is performing the dhcp server function and what
problem are you actually having?



On 21 September 2010 15:46, Jeff Wojciechowski <
jeff.wojciechow...@midlandpaper.com> wrote:

> All:
>
> I would like to use Embedded Packet Capture at a remote site to
> troubleshoot IOS DHCP at a remote site.
>
> (referencing
> http://www.cisco.com/en/US/docs/ios/netmgmt/configuration/guide/nm_packet_capture_ps6441_TSD_Products_Configuration_Guide_Chapter.html#wp1060070
> )
>
> The concept looks pretty straightforward but since DHCP negotiation isn't
> IP - I am having trouble with building the access list to determine what
> traffic to capture.
>
> The device I would like to perform the capture on is a 2801 and that is the
> device serving DHCP to clients at that site.
>
> Utilization on that router is pretty low so if I *had* to capture all
> traffic on F0/0 and dump it via TFTP to the fileserver at the site - I think
> that might be a viable option? But the fileserver is connected to the switch
> that connected to F0/0 that I want to perform the capture on.
>
> Regular DHCP debugs only show that 2 devices are continually offered IPs
> but then apparently keep re-requesting them. (So maybe I am barking up the
> wrong tree here.)
>
> Diagram:
>
> WANSerial-2801-F0/0Switch--DHCP Clients and Fileserver
>
> Any and all hints would be greatly appreciated.
>
> Thanks much,
>
> Jeff Wojciechowski
> LAN, WAN and Telephony Administrator
> Midland Paper Company
> 101 E Palatine Rd
> Wheeling, IL 60090
> * tel: 847.777.2829
> * fax: 847.403.6829
> e-mail: jeff.wojciechow...@midlandpaper.com jeff.wojciechow...@midlandpaper.com>
> http://www.midlandpaper.com
>
>
>
>
>  
> This electronic mail (including any attachments) may contain information
> that is privileged, confidential, or otherwise protected from disclosure to
> anyone other than its intended recipient(s). Any dissemination or use of
> this electronic mail or its contents (including any attachments) by persons
> other than the intended recipient(s) is strictly prohibited. If you have
> received this message in error, please delete the original message in its
> entirety (including any attachments) and notify us immediately by reply
> email so that we may correct our internal records. Midland Paper Company
> accepts no responsibility for any loss or damage from use of this electronic
> mail, including any damage resulting from a computer virus.
> ___
> cisco-nsp mailing list  cisco-nsp@puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-nsp
> archive at http://puck.nether.net/pipermail/cisco-nsp/
>
___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: [c-nsp] Weird Traceroute Issue to Specific Destination

[Paul Stewart]

Yes, loopback is in place and the source . yes, loopback in routing table
(redistributed via OSPF).  This 7206VXR has been in production for over 4
years and we have no issues reaching any other websites.

 

I'm confident that if the remote IP was blocking us or something of that
nature that the traceroute would at least transverse our igp properly . 

 

Thanks,

 

Paul

 

 

From: Heath Jones [mailto:hj1...@gmail.com] 
Sent: Tuesday, September 21, 2010 11:00 AM
To: Paul Stewart
Cc: cisco-nsp@puck.nether.net
Subject: Re: [c-nsp] Weird Traceroute Issue to Specific Destination

 

If it's not a firewall, its probably routing.. Is the 7206VXR using a
loopback for the source of the icmp request packets, and do you have a route
back to this ip in your igp?



 

On 21 September 2010 15:17, Paul Stewart  wrote:

Thank you - good thinking but I checked and there's nothing in there to
limit ICMP at all..;)

 

Paul

 

 

From: Heath Jones [mailto:hj1...@gmail.com] 
Sent: Tuesday, September 21, 2010 10:05 AM
To: Paul Stewart
Cc: cisco-nsp@puck.nether.net
Subject: Re: [c-nsp] Weird Traceroute Issue to Specific Destination

 

Hi Paul - perhaps you have a firewall filter preventing the ingress icmp
replies (to the 7206VXR)..?



 

On 21 September 2010 14:54, Paul Stewart  wrote:

Hi folks..



We have a customer who is connected over DSL who is having issues getting to
a certain remote site more often than not.  Sometime they can reach this
site, but most of the time they cannot.



They connect to a 7206VXR, which then connects to a 6509 which then
connections to 6509, 6509, then 7606 out to Internet.  Long story short,
there is no reported issues along this connectivity at all and we can only
replicate this complaint to one remote IP address.  Logically, we would push
this back and say "not our problem" which we're confident it's not *but*
there's one strange thing that is bugging me and I can't put logic around
this (I also have a terrible head cold and not thinking straight).



When logged into the 7206VXR where the customer connects via DSL, a
traceroute to the Internet loops normally like this:



acs1-con-bb#traceroute www.cnn.com  

Translating "www.cnn.com  "...domain server
(208.67.222.222) [OK]



Type escape sequence to abort.

Tracing the route to www.cnn.com   (157.166.224.26)



 1 xx.xxx.7.65 0 msec 0 msec 0 msec

 2 xx.xx.120.25 8 msec 8 msec 8 msec

 3 core2-rtr-to-ge4-12-vl4.nexicom.net
  (98.124.0.226) 20 msec 16
msec 88
msec

 4 ge4-0-0.core1.toronto1.nexicom.net
  (98.124.59.17) 16 msec 20 msec
16
msec

 5 ge-6-1-113.car1.Toronto2.Level3.net
  (4.59.180.9) 156 msec 176
msec 88
msec

 6 ae-8-8.ebr1.Chicago1.Level3.net 
(4.69.140.245) 32 msec 32 msec 32 msec

 7 ae-1-100.ebr2.Chicago1.Level3.net
  (4.69.132.42) 32 msec 28 msec
32 msec

 8 ae-3-3.ebr2.Atlanta2.Level3.net 
(4.69.132.74) 52 msec 52 msec 72 msec

 9 ae-2-52.edge4.Atlanta2.Level3.net
  (4.68.103.46) 52 msec 56 msec

   ae-1-51.edge4.Atlanta2.Level3.net
  (4.68.103.14) 48 msec

 10  *  *





With this problem destination IP, our traceroutes look like this:



acs1-con-bb#traceroute xx.xxx.2.226



Type escape sequence to abort.

Tracing the route to xxx.xxx.xxx (xx.xxx.2.226)



 1 xx.xxx.7.65 0 msec 0 msec 0 msec

 2  *  *  *

 3  *  *  *

 4  *  *  *





So, normally with all "*" it would indicate simply that the route is
unreachable. but that's the part that has me puzzled.  The route is
reachable across the Internet *and* at the very least, I should see a
traceroute going to the fourth hop no matter what.  We run OSPF across these
links and have a "default-originate" statement on the 4th hop router.  This
is confirmed via the routing table entry for 0.0.0.0



So what makes this particular destination IP act differently than any other
within our IGP is my question??



Cheers,



Paul



___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

 

 

___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: [c-nsp] Embeded Packet Capture for DHCP Troubleshooting

[Gert Doering]

Hi,

On Tue, Sep 21, 2010 at 09:46:59AM -0500, Jeff Wojciechowski wrote:
> The concept looks pretty straightforward but since DHCP negotiation 
> isn't IP 

Last time I looked, it was.  UDP port 67 and 68, to be precise.

(Unless you're talking about DHCPv6, which is IPv6, of course)

gert
-- 
USENET is *not* the non-clickable part of WWW!
   //www.muc.de/~gert/
Gert Doering - Munich, Germany g...@greenie.muc.de
fax: +49-89-35655025g...@net.informatik.tu-muenchen.de


pgpaOOyzkAQsz.pgp
Description: PGP signature
___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: [c-nsp] Weird Traceroute Issue to Specific Destination

[Heath Jones]

If it's not a firewall, its probably routing.. Is the 7206VXR using a
loopback for the source of the icmp request packets, and do you have a route
back to this ip in your igp?



On 21 September 2010 15:17, Paul Stewart  wrote:

>  Thank you – good thinking but I checked and there’s nothing in there to
> limit ICMP at all….;)
>
>
>
> Paul
>
>
>
>
>
> *From:* Heath Jones [mailto:hj1...@gmail.com]
> *Sent:* Tuesday, September 21, 2010 10:05 AM
> *To:* Paul Stewart
> *Cc:* cisco-nsp@puck.nether.net
> *Subject:* Re: [c-nsp] Weird Traceroute Issue to Specific Destination
>
>
>
> Hi Paul - perhaps you have a firewall filter preventing the ingress icmp
> replies (to the 7206VXR)..?
>
>
>
>
>
> On 21 September 2010 14:54, Paul Stewart  wrote:
>
> Hi folks..
>
>
>
> We have a customer who is connected over DSL who is having issues getting
> to
> a certain remote site more often than not.  Sometime they can reach this
> site, but most of the time they cannot.
>
>
>
> They connect to a 7206VXR, which then connects to a 6509 which then
> connections to 6509, 6509, then 7606 out to Internet.  Long story short,
> there is no reported issues along this connectivity at all and we can only
> replicate this complaint to one remote IP address.  Logically, we would
> push
> this back and say "not our problem" which we're confident it's not *but*
> there's one strange thing that is bugging me and I can't put logic around
> this (I also have a terrible head cold and not thinking straight).
>
>
>
> When logged into the 7206VXR where the customer connects via DSL, a
> traceroute to the Internet loops normally like this:
>
>
>
> acs1-con-bb#traceroute www.cnn.com
>
> Translating "www.cnn.com"...domain server (208.67.222.222) [OK]
>
>
>
> Type escape sequence to abort.
>
> Tracing the route to www.cnn.com (157.166.224.26)
>
>
>
>  1 xx.xxx.7.65 0 msec 0 msec 0 msec
>
>  2 xx.xx.120.25 8 msec 8 msec 8 msec
>
>  3 core2-rtr-to-ge4-12-vl4.nexicom.net (98.124.0.226) 20 msec 16 msec 88
> msec
>
>  4 ge4-0-0.core1.toronto1.nexicom.net (98.124.59.17) 16 msec 20 msec 16
> msec
>
>  5 
> ge-6-1-113.car1.Toronto2.Level3.net(4.59.180.9)
>  156 msec 176 msec 88
> msec
>
>  6 
> ae-8-8.ebr1.Chicago1.Level3.net(4.69.140.245)
>  32 msec 32 msec 32 msec
>
>  7 
> ae-1-100.ebr2.Chicago1.Level3.net(4.69.132.42)
>  32 msec 28 msec 32 msec
>
>  8 
> ae-3-3.ebr2.Atlanta2.Level3.net(4.69.132.74)
>  52 msec 52 msec 72 msec
>
>  9 
> ae-2-52.edge4.Atlanta2.Level3.net(4.68.103.46)
>  52 msec 56 msec
>
>
> ae-1-51.edge4.Atlanta2.Level3.net(4.68.103.14)
>  48 msec
>
>  10  *  *
>
>
>
>
>
> With this problem destination IP, our traceroutes look like this:
>
>
>
> acs1-con-bb#traceroute xx.xxx.2.226
>
>
>
> Type escape sequence to abort.
>
> Tracing the route to xxx.xxx.xxx (xx.xxx.2.226)
>
>
>
>  1 xx.xxx.7.65 0 msec 0 msec 0 msec
>
>  2  *  *  *
>
>  3  *  *  *
>
>  4  *  *  *
>
>
>
>
>
> So, normally with all "*" it would indicate simply that the route is
> unreachable. but that's the part that has me puzzled.  The route is
> reachable across the Internet *and* at the very least, I should see a
> traceroute going to the fourth hop no matter what.  We run OSPF across
> these
> links and have a "default-originate" statement on the 4th hop router.  This
> is confirmed via the routing table entry for 0.0.0.0
>
>
>
> So what makes this particular destination IP act differently than any other
> within our IGP is my question??
>
>
>
> Cheers,
>
>
>
> Paul
>
>
>
> ___
> cisco-nsp mailing list  cisco-nsp@puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-nsp
> archive at http://puck.nether.net/pipermail/cisco-nsp/
>
>
>
___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

[c-nsp] Embeded Packet Capture for DHCP Troubleshooting

[Jeff Wojciechowski]

All:

I would like to use Embedded Packet Capture at a remote site to troubleshoot 
IOS DHCP at a remote site.

(referencing 
http://www.cisco.com/en/US/docs/ios/netmgmt/configuration/guide/nm_packet_capture_ps6441_TSD_Products_Configuration_Guide_Chapter.html#wp1060070)

The concept looks pretty straightforward but since DHCP negotiation isn't IP - 
I am having trouble with building the access list to determine what traffic to 
capture.

The device I would like to perform the capture on is a 2801 and that is the 
device serving DHCP to clients at that site.

Utilization on that router is pretty low so if I *had* to capture all traffic 
on F0/0 and dump it via TFTP to the fileserver at the site - I think that might 
be a viable option? But the fileserver is connected to the switch that 
connected to F0/0 that I want to perform the capture on.

Regular DHCP debugs only show that 2 devices are continually offered IPs but 
then apparently keep re-requesting them. (So maybe I am barking up the wrong 
tree here.)

Diagram:

WANSerial-2801-F0/0Switch--DHCP Clients and Fileserver

Any and all hints would be greatly appreciated.

Thanks much,

Jeff Wojciechowski
LAN, WAN and Telephony Administrator
Midland Paper Company
101 E Palatine Rd
Wheeling, IL 60090
* tel: 847.777.2829
* fax: 847.403.6829
e-mail: 
jeff.wojciechow...@midlandpaper.com
http://www.midlandpaper.com




  
This electronic mail (including any attachments) may contain information that 
is privileged, confidential, or otherwise protected from disclosure to anyone 
other than its intended recipient(s). Any dissemination or use of this 
electronic mail or its contents (including any attachments) by persons other 
than the intended recipient(s) is strictly prohibited. If you have received 
this message in error, please delete the original message in its entirety 
(including any attachments) and notify us immediately by reply email so that we 
may correct our internal records. Midland Paper Company accepts no 
responsibility for any loss or damage from use of this electronic mail, 
including any damage resulting from a computer virus.
___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: [c-nsp] n5k caches usernames logged in

[Jared Mauch]

I despise hidden configuration items like those reported here. Either it's a 
local user in the config or not. Looks like a bug or architecture flaw. 

Jared Mauch

On Sep 21, 2010, at 9:24 AM, Tassos Chatzithomaoglou  wrote:

> 
> 
> Lincoln Dale wrote on 20/09/2010 05:20:
>> On 19/09/2010, at 10:12 PM, Tassos Chatzithomaoglou wrote:
>> 
>>   
>>> I noticed that when a remote user logs into a n5k and then logs out, the 
>>> "show user-account" command still displays the user credentials.
>>> Waiting for some time or clearing manually the user from config mode (?), 
>>> fixes the above.
>>> Is there a timer for such cached entries that can be changed? Is it 
>>> possible to disable this caching?
>>> 
>> i'm not sure why you feel this to be an issue?  if the user is provided from 
>> AAA then the AAA server will still be consulted for the 
>> authentication/authorization/roles etc.
>>   
> Maybe it's not an issue.
> The output of "show user-account" gets cluttered with all the users that have 
> logged into the n5k in the past.
> Since this isn't documented anywhere, i found it strange.
> Also having to clear the username from config mode, while "sh run" doesn't 
> include the username, is even more strange.
> 
> n5k(config)# username test role network-admin
> user:test account is already created through remote authentication.
> Please first delete that account using "no" option.
> 
> 
> --
> Tassos
> 
> ___
> cisco-nsp mailing list  cisco-nsp@puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-nsp
> archive at http://puck.nether.net/pipermail/cisco-nsp/

___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: [c-nsp] Weird Traceroute Issue to Specific Destination

[Paul Stewart]

Thank you - good thinking but I checked and there's nothing in there to
limit ICMP at all..;)

 

Paul

 

 

From: Heath Jones [mailto:hj1...@gmail.com] 
Sent: Tuesday, September 21, 2010 10:05 AM
To: Paul Stewart
Cc: cisco-nsp@puck.nether.net
Subject: Re: [c-nsp] Weird Traceroute Issue to Specific Destination

 

Hi Paul - perhaps you have a firewall filter preventing the ingress icmp
replies (to the 7206VXR)..?



 

On 21 September 2010 14:54, Paul Stewart  wrote:

Hi folks..



We have a customer who is connected over DSL who is having issues getting to
a certain remote site more often than not.  Sometime they can reach this
site, but most of the time they cannot.



They connect to a 7206VXR, which then connects to a 6509 which then
connections to 6509, 6509, then 7606 out to Internet.  Long story short,
there is no reported issues along this connectivity at all and we can only
replicate this complaint to one remote IP address.  Logically, we would push
this back and say "not our problem" which we're confident it's not *but*
there's one strange thing that is bugging me and I can't put logic around
this (I also have a terrible head cold and not thinking straight).



When logged into the 7206VXR where the customer connects via DSL, a
traceroute to the Internet loops normally like this:



acs1-con-bb#traceroute www.cnn.com  

Translating "www.cnn.com  "...domain server
(208.67.222.222) [OK]



Type escape sequence to abort.

Tracing the route to www.cnn.com   (157.166.224.26)



 1 xx.xxx.7.65 0 msec 0 msec 0 msec

 2 xx.xx.120.25 8 msec 8 msec 8 msec

 3 core2-rtr-to-ge4-12-vl4.nexicom.net
  (98.124.0.226) 20 msec 16
msec 88
msec

 4 ge4-0-0.core1.toronto1.nexicom.net
  (98.124.59.17) 16 msec 20 msec
16
msec

 5 ge-6-1-113.car1.Toronto2.Level3.net
  (4.59.180.9) 156 msec 176
msec 88
msec

 6 ae-8-8.ebr1.Chicago1.Level3.net 
(4.69.140.245) 32 msec 32 msec 32 msec

 7 ae-1-100.ebr2.Chicago1.Level3.net
  (4.69.132.42) 32 msec 28 msec
32 msec

 8 ae-3-3.ebr2.Atlanta2.Level3.net 
(4.69.132.74) 52 msec 52 msec 72 msec

 9 ae-2-52.edge4.Atlanta2.Level3.net
  (4.68.103.46) 52 msec 56 msec

   ae-1-51.edge4.Atlanta2.Level3.net
  (4.68.103.14) 48 msec

 10  *  *





With this problem destination IP, our traceroutes look like this:



acs1-con-bb#traceroute xx.xxx.2.226



Type escape sequence to abort.

Tracing the route to xxx.xxx.xxx (xx.xxx.2.226)



 1 xx.xxx.7.65 0 msec 0 msec 0 msec

 2  *  *  *

 3  *  *  *

 4  *  *  *





So, normally with all "*" it would indicate simply that the route is
unreachable. but that's the part that has me puzzled.  The route is
reachable across the Internet *and* at the very least, I should see a
traceroute going to the fourth hop no matter what.  We run OSPF across these
links and have a "default-originate" statement on the 4th hop router.  This
is confirmed via the routing table entry for 0.0.0.0



So what makes this particular destination IP act differently than any other
within our IGP is my question??



Cheers,



Paul



___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

 

___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: [c-nsp] Weird Traceroute Issue to Specific Destination

[Heath Jones]

Hi Paul - perhaps you have a firewall filter preventing the ingress icmp
replies (to the 7206VXR)..?



On 21 September 2010 14:54, Paul Stewart  wrote:

> Hi folks..
>
>
>
> We have a customer who is connected over DSL who is having issues getting
> to
> a certain remote site more often than not.  Sometime they can reach this
> site, but most of the time they cannot.
>
>
>
> They connect to a 7206VXR, which then connects to a 6509 which then
> connections to 6509, 6509, then 7606 out to Internet.  Long story short,
> there is no reported issues along this connectivity at all and we can only
> replicate this complaint to one remote IP address.  Logically, we would
> push
> this back and say "not our problem" which we're confident it's not *but*
> there's one strange thing that is bugging me and I can't put logic around
> this (I also have a terrible head cold and not thinking straight).
>
>
>
> When logged into the 7206VXR where the customer connects via DSL, a
> traceroute to the Internet loops normally like this:
>
>
>
> acs1-con-bb#traceroute www.cnn.com
>
> Translating "www.cnn.com"...domain server (208.67.222.222) [OK]
>
>
>
> Type escape sequence to abort.
>
> Tracing the route to www.cnn.com (157.166.224.26)
>
>
>
>  1 xx.xxx.7.65 0 msec 0 msec 0 msec
>
>  2 xx.xx.120.25 8 msec 8 msec 8 msec
>
>  3 core2-rtr-to-ge4-12-vl4.nexicom.net (98.124.0.226) 20 msec 16 msec 88
> msec
>
>  4 ge4-0-0.core1.toronto1.nexicom.net (98.124.59.17) 16 msec 20 msec 16
> msec
>
>  5 
> ge-6-1-113.car1.Toronto2.Level3.net(4.59.180.9)
>  156 msec 176 msec 88
> msec
>
>  6 
> ae-8-8.ebr1.Chicago1.Level3.net(4.69.140.245)
>  32 msec 32 msec 32 msec
>
>  7 
> ae-1-100.ebr2.Chicago1.Level3.net(4.69.132.42)
>  32 msec 28 msec 32 msec
>
>  8 
> ae-3-3.ebr2.Atlanta2.Level3.net(4.69.132.74)
>  52 msec 52 msec 72 msec
>
>  9 
> ae-2-52.edge4.Atlanta2.Level3.net(4.68.103.46)
>  52 msec 56 msec
>
>
> ae-1-51.edge4.Atlanta2.Level3.net(4.68.103.14)
>  48 msec
>
>  10  *  *
>
>
>
>
>
> With this problem destination IP, our traceroutes look like this:
>
>
>
> acs1-con-bb#traceroute xx.xxx.2.226
>
>
>
> Type escape sequence to abort.
>
> Tracing the route to xxx.xxx.xxx (xx.xxx.2.226)
>
>
>
>  1 xx.xxx.7.65 0 msec 0 msec 0 msec
>
>  2  *  *  *
>
>  3  *  *  *
>
>  4  *  *  *
>
>
>
>
>
> So, normally with all "*" it would indicate simply that the route is
> unreachable. but that's the part that has me puzzled.  The route is
> reachable across the Internet *and* at the very least, I should see a
> traceroute going to the fourth hop no matter what.  We run OSPF across
> these
> links and have a "default-originate" statement on the 4th hop router.  This
> is confirmed via the routing table entry for 0.0.0.0
>
>
>
> So what makes this particular destination IP act differently than any other
> within our IGP is my question??
>
>
>
> Cheers,
>
>
>
> Paul
>
>
>
> ___
> cisco-nsp mailing list  cisco-nsp@puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-nsp
> archive at http://puck.nether.net/pipermail/cisco-nsp/
>
___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

[c-nsp] "network-admin" account causes "invalid role vdc-operator" in n5k login

[Tassos Chatzithomaoglou]

I have an user with network-admin role configured (the user is 
configured in a tacacs server, his role is either in the tacacs server 
or locally), but i get the following message when he logs in.


%AUTHPRIV-3-SYSTEM_MSG: invalid role vdc-operator downloaded for user 
test - login[10065]


Any idea why the "vdc-operator" role is being tried too? Bug toolkit 
didn't return anything.

The network-admin role is working fine for this user.

--
Tassos

___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

[c-nsp] Weird Traceroute Issue to Specific Destination

[Paul Stewart]

Hi folks..

 

We have a customer who is connected over DSL who is having issues getting to
a certain remote site more often than not.  Sometime they can reach this
site, but most of the time they cannot.

 

They connect to a 7206VXR, which then connects to a 6509 which then
connections to 6509, 6509, then 7606 out to Internet.  Long story short,
there is no reported issues along this connectivity at all and we can only
replicate this complaint to one remote IP address.  Logically, we would push
this back and say "not our problem" which we're confident it's not *but*
there's one strange thing that is bugging me and I can't put logic around
this (I also have a terrible head cold and not thinking straight).

 

When logged into the 7206VXR where the customer connects via DSL, a
traceroute to the Internet loops normally like this:

 

acs1-con-bb#traceroute www.cnn.com

Translating "www.cnn.com"...domain server (208.67.222.222) [OK]

 

Type escape sequence to abort.

Tracing the route to www.cnn.com (157.166.224.26)

 

  1 xx.xxx.7.65 0 msec 0 msec 0 msec

  2 xx.xx.120.25 8 msec 8 msec 8 msec

  3 core2-rtr-to-ge4-12-vl4.nexicom.net (98.124.0.226) 20 msec 16 msec 88
msec

  4 ge4-0-0.core1.toronto1.nexicom.net (98.124.59.17) 16 msec 20 msec 16
msec

  5 ge-6-1-113.car1.Toronto2.Level3.net (4.59.180.9) 156 msec 176 msec 88
msec

  6 ae-8-8.ebr1.Chicago1.Level3.net (4.69.140.245) 32 msec 32 msec 32 msec

  7 ae-1-100.ebr2.Chicago1.Level3.net (4.69.132.42) 32 msec 28 msec 32 msec

  8 ae-3-3.ebr2.Atlanta2.Level3.net (4.69.132.74) 52 msec 52 msec 72 msec

  9 ae-2-52.edge4.Atlanta2.Level3.net (4.68.103.46) 52 msec 56 msec

ae-1-51.edge4.Atlanta2.Level3.net (4.68.103.14) 48 msec

 10  *  *

 

 

With this problem destination IP, our traceroutes look like this:

 

acs1-con-bb#traceroute xx.xxx.2.226

 

Type escape sequence to abort.

Tracing the route to xxx.xxx.xxx (xx.xxx.2.226)

 

  1 xx.xxx.7.65 0 msec 0 msec 0 msec

  2  *  *  *

  3  *  *  *

  4  *  *  *

  

 

So, normally with all "*" it would indicate simply that the route is
unreachable. but that's the part that has me puzzled.  The route is
reachable across the Internet *and* at the very least, I should see a
traceroute going to the fourth hop no matter what.  We run OSPF across these
links and have a "default-originate" statement on the 4th hop router.  This
is confirmed via the routing table entry for 0.0.0.0

 

So what makes this particular destination IP act differently than any other
within our IGP is my question??

 

Cheers,

 

Paul

 

___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: [c-nsp] n5k caches usernames logged in

[Tassos Chatzithomaoglou]

Lincoln Dale wrote on 20/09/2010 05:20:

On 19/09/2010, at 10:12 PM, Tassos Chatzithomaoglou wrote:

   

I noticed that when a remote user logs into a n5k and then logs out, the "show 
user-account" command still displays the user credentials.
Waiting for some time or clearing manually the user from config mode (?), fixes 
the above.
Is there a timer for such cached entries that can be changed? Is it possible to 
disable this caching?
 

i'm not sure why you feel this to be an issue?  if the user is provided from 
AAA then the AAA server will still be consulted for the 
authentication/authorization/roles etc.
   

Maybe it's not an issue.
The output of "show user-account" gets cluttered with all the users that 
have logged into the n5k in the past.

Since this isn't documented anywhere, i found it strange.
Also having to clear the username from config mode, while "sh run" 
doesn't include the username, is even more strange.


n5k(config)# username test role network-admin
user:test account is already created through remote authentication.
Please first delete that account using "no" option.


--
Tassos

___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: [c-nsp] ASR1000 Routes on ESP2.5 ESP5

[Łukasz Bromirski]

On 2010-09-21 09:06, Garry wrote:


ASR1002-F and ASR-1002 can both handle 512k IPv4 prefixes maximum,
128k IPv6 prefixes maximum or a mix of these. The change in marking
of the ESP from 5 to 2.5 comes from the fact that the useable
bandwidth was halved, not the system capacity to store data.

Whoa ... and $C is marketing this box (1002F) as a provider box?


Where you've read it? ASR is a WAN services box. Positioned
for SPs only as a multi-service edge/SBC/route reflector (with RP2).


I was looking into junking a couple 7200VXR for them, but

> seeing our current prefix total already well above 300k on the
> internet, I'm sort of worried whether this is a good idea ...

This discussion is almost weekly here on cisco-nsp@, but
focuses mainly on the PFC3B/3C vs PFC3BXL/CXL for 6500/7600.

It's an edge not peering box, if you need 1mln IPv4 prefixes in FIB
go for either ASR1002 with ESP10 or 7600S with RSP720 CXL version,
depending on the actual bandwidth you need to push through the box.

--
"Everything will be okay in the end.  | Łukasz Bromirski
 If it's not okay, it's not the end." |  http://lukasz.bromirski.net
___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: [c-nsp] ASR1000 Routes on ESP2.5 ESP5

[Garry]

 On 21.09.2010 07:35, Lukasz Bromirski wrote:
> On 2010-09-21 01:31, David Blundell wrote:
>
>> I am trying to find the number of routes that an ASR1002-F (embedded
>> ESP2.5) and an ASR-1002 with ESP5 can handle.
>
> ASR1002-F and ASR-1002 can both handle 512k IPv4 prefixes maximum,
> 128k IPv6 prefixes maximum or a mix of these. The change in marking
> of the ESP from 5 to 2.5 comes from the fact that the useable
> bandwidth was halved, not the system capacity to store data.
Whoa ... and $C is marketing this box (1002F) as a provider box? I was
looking into junking a couple 7200VXR for them, but seeing our current
prefix total already well above 300k on the internet, I'm sort of
worried whether this is a good idea ... at least for the one that was
going to do our DECIX peering ... add to that VRF prefixes that may also
be present ... with the v4 exhaustion, I reckon the Internet will most
likely be at a combined 500k prefixes between v4 and v6 in a year ... is
$C really this short-sighted?

OTOH, the 1002F will of course still be a nice box in the IGP area in
the event it's not sufficient for the uplink ...

-gg
___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/