Belpak / Beltelecom contact to address a BGP hijacking issue?
Hi all, Does anyone have a technical or peering contact at Belpak / Beltelecom (AS 66697) to address an apparent netblock hijacking issue? AS6697 is advertising the 2.2.2.0/24 address space which is under AS3215 management. We've tried to announce the same prefix but it's difficult to get the traffic back! No answer from people listed in the whois, no peeringDB information. Any suggestions? Thanks in advance, -- sarah
Re: IOS architecture
On Sat, 27 Oct 2012 11:16:10 +0100, Darren O'Connor said: All vendors should be writing in depth architecture books. The Juniper MX book is a great example. Tell us exactly what your product can do and we'll likely use more of it On the flip side, if you document what your product is probably incapable of due to the design architecture, the salescritters won't be able to sell as many of them... :) pgpALf2U241yK.pgp Description: PGP signature
Re: IOS architecture
On 29 October 2012 12:43, valdis.kletni...@vt.edu wrote: On Sat, 27 Oct 2012 11:16:10 +0100, Darren O'Connor said: All vendors should be writing in depth architecture books. The Juniper MX book is a great example. Tell us exactly what your product can do and we'll likely use more of it On the flip side, if you document what your product is probably incapable of due to the design architecture, the salescritters won't be able to sell as many of them... :) I think the biggest problem in that regard is the gap between what the switch or router architecture is capable of and what the current release of IOS actually supports. This is generally what appears on the roadmap but, historically, not all of it gets delivered in a timely manner, and some features aren't delivered at all before the hardware is superseded. Aled
Re: IP tunnel MTU
The core issue here is TCP MSS. PMTUD is a dynamic process for adjusting MSS, but requires that ICMP be permitted to negotiate the connection. The realistic alternative, in a world that filters all ICMP traffic, is to manually rewrite the MSS. In IOS this can be achieved via ip tcp adjust-mss and on Linux-based systems, netfilter can be used to adjust MSS for example. Keep in mind that the MSS will be smaller than your MTU. Consider the following example: ip mtu 1480 ip tcp adjust-mss 1440 tunnel mode ipip IP packets have 20 bytes of overhead, leaving 1480 bytes for data. So for an IP-in-IP tunnel, you'd set your MTU of your tunnel interface to 1480. Subtract another 20 bytes for the tunneled IP header and 20 bytes (typical) for your TCP header and you're left with 1440 bytes for data in a TCP connection. So in this case we write the MSS as 1440. I use IP-in-IP as an example because it's simple. GRE tunnels can be a little more complex. While the GRE header is typically 4 bytes, it can grow up to 16 bytes depending on options used. So for a typical GRE tunnel (4 byte header), you would subtract 20 bytes for the IP header and 4 bytes for the GRE header from your base MTU of 1500. This would mean an MTU of 1476, and a TCP MMS of 1436. Keep in mind that a TCP header can be up to 60 bytes in length, so you may want to go higher than the typical 20 bytes for your MSS if you're seeing problems. On Tue, Oct 23, 2012 at 10:07 AM, Templin, Fred L fred.l.temp...@boeing.com wrote: Hi Roland, -Original Message- From: Dobbins, Roland [mailto:rdobb...@arbor.net] Sent: Monday, October 22, 2012 6:49 PM To: NANOG list Subject: Re: IP tunnel MTU On Oct 23, 2012, at 5:24 AM, Templin, Fred L wrote: Since tunnels always reduce the effective MTU seen by data packets due to the encapsulation overhead, the only two ways to accommodate the tunnel MTU is either through the use of path MTU discovery or through fragmentation and reassembly. Actually, you can set your tunnel MTU manually. For example, the typical MTU folks set for a GRE tunnel is 1476. Yes; I was aware of this. But, what I want to get to is setting the tunnel MTU to infinity. This isn't a new issue; it's been around ever since tunneling technologies have been around, and tons have been written on this topic. Look at your various router/switch vendor Web sites, archives of this list and others, etc. Sure. I've written a fair amount about it too over the span of the last ten years. What is new is that there is now a solution near at hand. So, it's been known about, dealt with, and documented for a long time. In terms of doing something about it, the answer there is a) to allow the requisite ICMP for PMTU-D to work to/through any networks within your span of administrative control and b) That does you no good if there is some other network further beyond your span of administrative control that does not allow the ICMP PTBs through. And, studies have shown this to be the case in a non-trivial number of instances. b) adjusting your own tunnel MTUs to appropriate values based upon experimentation. Adjust it down to what? 1280? Then, if your tunnel with the adjusted MTU enters another tunnel with its own adjusted MTU there is an MTU underflow that might not get reported if the ICMP PTB messages are lost. An alternative is to use IP fragmentation, but recent studies have shown that more and more operators are unconditionally dropping IPv6 fragments and IPv4 fragmentation is not an option due to wrapping IDs at high data rates. Nested tunnels-within-tunnels occur in operational scenarios more and more, and adjusting the MTU for only one tunnel in the nesting does you no good if there are other tunnels that adjust their own MTUs. Enterprise endpoint networks are notorious for blocking *all* ICMP (as well as TCP/53 DNS) at their edges due to 'security' misinformation propagated by Confused Information Systems Security Professionals and their ilk. Be sure that your own network policies aren't part of the problem affecting your userbase, as well as anyone else with a need to communicate with properties on your network via tunnels. Again, all an operator can control is that which is within their own administrative domain. That does no good for ICMPs that are lost beyond their administrative domain. Thanks - Fred fred.l.temp...@boeing.com --- Roland Dobbins rdobb...@arbor.net // http://www.arbornetworks.com Luck is the residue of opportunity and design. -- John Milton -- Ray Patrick Soucy Network Engineer University of Maine System T: 207-561-3526 F: 207-561-3531 MaineREN, Maine's Research and Education Network www.maineren.net
RE: IP tunnel MTU
Hi Ray, MSS rewriting has been well known and broadly applied for a long time now, but only applies to TCP. The subject of MSS rewriting comes up all the time in the IETF wg discussions, but has failed to reach consensus as a long-term alternative. Plus, MSS rewriting does no good for tunnels-within-tunnels. If the innermost tunnel rewrites MSS to a value that *it* thinks is safe there is no guarantee that the packets will fit within any outer tunnels that occur further down the line. What I want to get to is an indefinite tunnel MTU; i.e., admit any packet into the tunnel regardless of its size then make any necessary adaptations from within the tunnel. That is exactly what SEAL does: https://datatracker.ietf.org/doc/draft-templin-intarea-seal/ Thanks - Fred fred.l.temp...@boeing.com -Original Message- From: Ray Soucy [mailto:r...@maine.edu] Sent: Monday, October 29, 2012 7:55 AM To: Templin, Fred L Cc: Dobbins, Roland; NANOG list Subject: Re: IP tunnel MTU The core issue here is TCP MSS. PMTUD is a dynamic process for adjusting MSS, but requires that ICMP be permitted to negotiate the connection. The realistic alternative, in a world that filters all ICMP traffic, is to manually rewrite the MSS. In IOS this can be achieved via ip tcp adjust-mss and on Linux-based systems, netfilter can be used to adjust MSS for example. Keep in mind that the MSS will be smaller than your MTU. Consider the following example: ip mtu 1480 ip tcp adjust-mss 1440 tunnel mode ipip IP packets have 20 bytes of overhead, leaving 1480 bytes for data. So for an IP-in-IP tunnel, you'd set your MTU of your tunnel interface to 1480. Subtract another 20 bytes for the tunneled IP header and 20 bytes (typical) for your TCP header and you're left with 1440 bytes for data in a TCP connection. So in this case we write the MSS as 1440. I use IP-in-IP as an example because it's simple. GRE tunnels can be a little more complex. While the GRE header is typically 4 bytes, it can grow up to 16 bytes depending on options used. So for a typical GRE tunnel (4 byte header), you would subtract 20 bytes for the IP header and 4 bytes for the GRE header from your base MTU of 1500. This would mean an MTU of 1476, and a TCP MMS of 1436. Keep in mind that a TCP header can be up to 60 bytes in length, so you may want to go higher than the typical 20 bytes for your MSS if you're seeing problems. On Tue, Oct 23, 2012 at 10:07 AM, Templin, Fred L fred.l.temp...@boeing.com wrote: Hi Roland, -Original Message- From: Dobbins, Roland [mailto:rdobb...@arbor.net] Sent: Monday, October 22, 2012 6:49 PM To: NANOG list Subject: Re: IP tunnel MTU On Oct 23, 2012, at 5:24 AM, Templin, Fred L wrote: Since tunnels always reduce the effective MTU seen by data packets due to the encapsulation overhead, the only two ways to accommodate the tunnel MTU is either through the use of path MTU discovery or through fragmentation and reassembly. Actually, you can set your tunnel MTU manually. For example, the typical MTU folks set for a GRE tunnel is 1476. Yes; I was aware of this. But, what I want to get to is setting the tunnel MTU to infinity. This isn't a new issue; it's been around ever since tunneling technologies have been around, and tons have been written on this topic. Look at your various router/switch vendor Web sites, archives of this list and others, etc. Sure. I've written a fair amount about it too over the span of the last ten years. What is new is that there is now a solution near at hand. So, it's been known about, dealt with, and documented for a long time. In terms of doing something about it, the answer there is a) to allow the requisite ICMP for PMTU-D to work to/through any networks within your span of administrative control and b) That does you no good if there is some other network further beyond your span of administrative control that does not allow the ICMP PTBs through. And, studies have shown this to be the case in a non-trivial number of instances. b) adjusting your own tunnel MTUs to appropriate values based upon experimentation. Adjust it down to what? 1280? Then, if your tunnel with the adjusted MTU enters another tunnel with its own adjusted MTU there is an MTU underflow that might not get reported if the ICMP PTB messages are lost. An alternative is to use IP fragmentation, but recent studies have shown that more and more operators are unconditionally dropping IPv6 fragments and IPv4 fragmentation is not an option due to wrapping IDs at high data rates. Nested tunnels-within-tunnels occur in operational scenarios more and more, and adjusting the MTU for only one tunnel in the nesting does you no good if there are other tunnels that adjust their own MTUs. Enterprise endpoint networks are notorious for blocking *all* ICMP (as well
the little ssh that (sometimes) couldn't
corruption! http://mina.naguib.ca/blog/2012/10/22/the-little-ssh-that-sometimes-couldnt.html /bill
Re: Belpak / Beltelecom contact to address a BGP hijacking issue?
Hello Sarah Seems like they are not advertising it anymore. AS6697 has transit from Level3 and peering/transit from HE. Both of them show path to AS3215 for that prefix now. http://lookingglass.level3.net/ BGP query on all sites seems OK for now. Also same on results from Oregon as well as HE. Infact HE is taking 1299 5511 3215 ...way longer then direct 6697 if existed. Surely they were getting wrong announcement as per http://bgp.he.net/net/2.2.2.0/24 - which is updated once in 24hrs. So likely issue resolved? On Mon, Oct 29, 2012 at 3:24 PM, Sarah Nataf sarah.na...@gmail.com wrote: Hi all, Does anyone have a technical or peering contact at Belpak / Beltelecom (AS 66697) to address an apparent netblock hijacking issue? AS6697 is advertising the 2.2.2.0/24 address space which is under AS3215 management. We've tried to announce the same prefix but it's difficult to get the traffic back! No answer from people listed in the whois, no peeringDB information. Any suggestions? Thanks in advance, -- sarah -- Anurag Bhatia anuragbhatia.com Linkedin http://in.linkedin.com/in/anuragbhatia21 | Twitterhttps://twitter.com/anurag_bhatia| Google+ https://plus.google.com/118280168625121532854
Re: the little ssh that (sometimes) couldn't
On Mon, Oct 29, 2012 at 10:07 AM, bmann...@vacation.karoshi.com wrote: corruption! http://mina.naguib.ca/blog/2012/10/22/the-little-ssh-that-sometimes-couldnt.html /bill This is an excellent full-stack debugging war story. Thanks for posting it, Bill. -- -george william herbert george.herb...@gmail.com
[NANOG-announce] 2012 NANOG Election Results
*Greetings NANOG Colleagues, As usual for our October meetings, there has been a lot happening with our elections process and more announcements to come over the next few days. We wanted to give you a quick heads-up. Huge thank yous to our Executive Director, Betty Burke, our NANOG Secretariat, Florencia Dazzi and Karen Moore, the Verilan team, the SWANK team, and our technical coordinators Tim Pozar and Matt Peterson for a world-class event in Dallas. Elections Our annual election was held during NANOG 56. Steve Feldman, Dan Golding and Mike Smith were elected to two-year terms on the Board of Directors. All three proposed amendments to our bylaws also passed. The Board appointed its officers for the coming year. I have been re-appointed Chair, Mike Smith is the Vice-Chair, Duane Wessels is Treasurer and DC liaison, Steve Feldman is Secretary, Dan Golding is PC liaison and Steve Gibbard is CC liaison. Committee Appointments The Communications, Development and Program Committees selection process is drawing to a close. Offers have been extended to candidates on October 25 and we expect to have all confirmations in the next 24-48 hours. A formal announcement of their respective composition will follow this week. We wish to thank everyone who volunteered to serve either on the NANOG Board of Directors or on one of its Committees. NANOG is truly an organization that depends on the ongoing enthusiasm and support of its community: thank you for attending NANOG 56 live or via webcast and for contributing regularly to our mailing lists. The coming year is exciting. There is much to be done as we execute our three-year strategic plan. We will communicate progress regularly.* * * *Best regards, * Sylvie -- Sylvie LaPerriere Chair, NANOG Board of Directors www.nanog.org ___ NANOG-announce mailing list nanog-annou...@mailman.nanog.org http://mailman.nanog.org/mailman/listinfo/nanog-announce
Re: the little ssh that (sometimes) couldn't
On Mon, 29 Oct 2012 bmann...@vacation.karoshi.com wrote: corruption! http://mina.naguib.ca/blog/2012/10/22/the-little-ssh-that-sometimes-couldnt.html Bush league. I debugged a similar issue on Sprint's network about 15 years ago, also nailing it down to which router/router hop had the problem (a misconfigured interface that couldn't pass certain bit patterns and was causing a particular file we were hosting for a customer to be non-downloadable by any client who's packets used the bad path), also using ping, but with a pattern much more interesting than large packets of nulls...and I had to figure out the problematic pattern before I could do the ping tests. But if you want really bizzare, this one never got solved to my satisfaction. http://mailman.nanog.org/pipermail/nanog/2008-August/002788.html -- Jon Lewis, MCP :) | I route Senior Network Engineer | therefore you are Atlantic Net| _ http://www.lewis.org/~jlewis/pgp for PGP public key_
Network scan tool/appliance horror stories
We're evaluating several tools at the moment, and one vendor wants to dynamically scan our network to pick up hosts - SNMP, port-scans, WMI, the works. I was curious if anyone had any particularly gruesome horror stories of scanning tools run amok.
Re: the little ssh that (sometimes) couldn't
On 10/29/2012 02:54 PM, Jon Lewis wrote: Bush league. I debugged a similar issue on Sprint's network about 15 years ago, also nailing it down to which router/router hop had the problem When I was working for Sprint about 12 years ago, we had a circuit where the customer complained that we were blocking executable downloads. We essentially dismissed his complaints because they were ridiculous. We would test his T1 and it would show everything fine. I was willing to entertain his concern because it sounded weird and he had a UNIX box I could login to. Running wget I saw the same issues. If I zipped a file I could download it without issue, anything that was an exe would not. We narrowed it down to 2-4 bytes of the exe header that the circuit just wouldn't pass. Called the local telco and had them test the circuit from the customer prem, they found errors on the reverse. We fixed it and he could download executables again. I got an award for persistence and the customer canceled his account.
Re: IP tunnel MTU
Sorry, glanced at this and thought it was someone having problems with tunnel MTU without adjusting TCP MSS. Nice work, though my preference is to avoid tunnels at all costs :-) On Mon, Oct 29, 2012 at 12:39 PM, Templin, Fred L fred.l.temp...@boeing.com wrote: Hi Ray, MSS rewriting has been well known and broadly applied for a long time now, but only applies to TCP. The subject of MSS rewriting comes up all the time in the IETF wg discussions, but has failed to reach consensus as a long-term alternative. Plus, MSS rewriting does no good for tunnels-within-tunnels. If the innermost tunnel rewrites MSS to a value that *it* thinks is safe there is no guarantee that the packets will fit within any outer tunnels that occur further down the line. What I want to get to is an indefinite tunnel MTU; i.e., admit any packet into the tunnel regardless of its size then make any necessary adaptations from within the tunnel. That is exactly what SEAL does: https://datatracker.ietf.org/doc/draft-templin-intarea-seal/ Thanks - Fred fred.l.temp...@boeing.com -Original Message- From: Ray Soucy [mailto:r...@maine.edu] Sent: Monday, October 29, 2012 7:55 AM To: Templin, Fred L Cc: Dobbins, Roland; NANOG list Subject: Re: IP tunnel MTU The core issue here is TCP MSS. PMTUD is a dynamic process for adjusting MSS, but requires that ICMP be permitted to negotiate the connection. The realistic alternative, in a world that filters all ICMP traffic, is to manually rewrite the MSS. In IOS this can be achieved via ip tcp adjust-mss and on Linux-based systems, netfilter can be used to adjust MSS for example. Keep in mind that the MSS will be smaller than your MTU. Consider the following example: ip mtu 1480 ip tcp adjust-mss 1440 tunnel mode ipip IP packets have 20 bytes of overhead, leaving 1480 bytes for data. So for an IP-in-IP tunnel, you'd set your MTU of your tunnel interface to 1480. Subtract another 20 bytes for the tunneled IP header and 20 bytes (typical) for your TCP header and you're left with 1440 bytes for data in a TCP connection. So in this case we write the MSS as 1440. I use IP-in-IP as an example because it's simple. GRE tunnels can be a little more complex. While the GRE header is typically 4 bytes, it can grow up to 16 bytes depending on options used. So for a typical GRE tunnel (4 byte header), you would subtract 20 bytes for the IP header and 4 bytes for the GRE header from your base MTU of 1500. This would mean an MTU of 1476, and a TCP MMS of 1436. Keep in mind that a TCP header can be up to 60 bytes in length, so you may want to go higher than the typical 20 bytes for your MSS if you're seeing problems. On Tue, Oct 23, 2012 at 10:07 AM, Templin, Fred L fred.l.temp...@boeing.com wrote: Hi Roland, -Original Message- From: Dobbins, Roland [mailto:rdobb...@arbor.net] Sent: Monday, October 22, 2012 6:49 PM To: NANOG list Subject: Re: IP tunnel MTU On Oct 23, 2012, at 5:24 AM, Templin, Fred L wrote: Since tunnels always reduce the effective MTU seen by data packets due to the encapsulation overhead, the only two ways to accommodate the tunnel MTU is either through the use of path MTU discovery or through fragmentation and reassembly. Actually, you can set your tunnel MTU manually. For example, the typical MTU folks set for a GRE tunnel is 1476. Yes; I was aware of this. But, what I want to get to is setting the tunnel MTU to infinity. This isn't a new issue; it's been around ever since tunneling technologies have been around, and tons have been written on this topic. Look at your various router/switch vendor Web sites, archives of this list and others, etc. Sure. I've written a fair amount about it too over the span of the last ten years. What is new is that there is now a solution near at hand. So, it's been known about, dealt with, and documented for a long time. In terms of doing something about it, the answer there is a) to allow the requisite ICMP for PMTU-D to work to/through any networks within your span of administrative control and b) That does you no good if there is some other network further beyond your span of administrative control that does not allow the ICMP PTBs through. And, studies have shown this to be the case in a non-trivial number of instances. b) adjusting your own tunnel MTUs to appropriate values based upon experimentation. Adjust it down to what? 1280? Then, if your tunnel with the adjusted MTU enters another tunnel with its own adjusted MTU there is an MTU underflow that might not get reported if the ICMP PTB messages are lost. An alternative is to use IP fragmentation, but recent studies have shown that more and more operators are unconditionally dropping IPv6 fragments and IPv4 fragmentation is not an option due to wrapping IDs at high data rates. Nested
Re: Network scan tool/appliance horror stories
On Mon, 29 Oct 2012, Pedersen, Sean wrote: We're evaluating several tools at the moment, and one vendor wants to dynamically scan our network to pick up hosts - SNMP, port-scans, WMI, the works. I was curious if anyone had any particularly gruesome horror stories of scanning tools run amok. If you have any overloaded/under-powered network gear, such as stateful firewalls and routers that do lots of NAT, you might find them very quickly, depending on how aggressive the scanning tool is. There might also be devices out there that, while possibly lightly loaded, can reach some minimally documented resource threshold under a very aggressive scan, and subsequently tip over. Also, if you're doing IPv6, the performance metrics for many network devices can be a bit more of a moving target. jms
Re: IP tunnel MTU
Hi there, I have the same problem in my network, I have GRE tunnel for transfering users real internet traffic, they have problems with browsing websites like yahoo.com or microsoft.com. I had to set ip mtu 1500 to solve it, and it occurs fragmantation... Thanks On Mon, Oct 29, 2012 at 10:47 PM, Ray Soucy r...@maine.edu wrote: Sorry, glanced at this and thought it was someone having problems with tunnel MTU without adjusting TCP MSS. Nice work, though my preference is to avoid tunnels at all costs :-) On Mon, Oct 29, 2012 at 12:39 PM, Templin, Fred L fred.l.temp...@boeing.com wrote: Hi Ray, MSS rewriting has been well known and broadly applied for a long time now, but only applies to TCP. The subject of MSS rewriting comes up all the time in the IETF wg discussions, but has failed to reach consensus as a long-term alternative. Plus, MSS rewriting does no good for tunnels-within-tunnels. If the innermost tunnel rewrites MSS to a value that *it* thinks is safe there is no guarantee that the packets will fit within any outer tunnels that occur further down the line. What I want to get to is an indefinite tunnel MTU; i.e., admit any packet into the tunnel regardless of its size then make any necessary adaptations from within the tunnel. That is exactly what SEAL does: https://datatracker.ietf.org/doc/draft-templin-intarea-seal/ Thanks - Fred fred.l.temp...@boeing.com -Original Message- From: Ray Soucy [mailto:r...@maine.edu] Sent: Monday, October 29, 2012 7:55 AM To: Templin, Fred L Cc: Dobbins, Roland; NANOG list Subject: Re: IP tunnel MTU The core issue here is TCP MSS. PMTUD is a dynamic process for adjusting MSS, but requires that ICMP be permitted to negotiate the connection. The realistic alternative, in a world that filters all ICMP traffic, is to manually rewrite the MSS. In IOS this can be achieved via ip tcp adjust-mss and on Linux-based systems, netfilter can be used to adjust MSS for example. Keep in mind that the MSS will be smaller than your MTU. Consider the following example: ip mtu 1480 ip tcp adjust-mss 1440 tunnel mode ipip IP packets have 20 bytes of overhead, leaving 1480 bytes for data. So for an IP-in-IP tunnel, you'd set your MTU of your tunnel interface to 1480. Subtract another 20 bytes for the tunneled IP header and 20 bytes (typical) for your TCP header and you're left with 1440 bytes for data in a TCP connection. So in this case we write the MSS as 1440. I use IP-in-IP as an example because it's simple. GRE tunnels can be a little more complex. While the GRE header is typically 4 bytes, it can grow up to 16 bytes depending on options used. So for a typical GRE tunnel (4 byte header), you would subtract 20 bytes for the IP header and 4 bytes for the GRE header from your base MTU of 1500. This would mean an MTU of 1476, and a TCP MMS of 1436. Keep in mind that a TCP header can be up to 60 bytes in length, so you may want to go higher than the typical 20 bytes for your MSS if you're seeing problems. On Tue, Oct 23, 2012 at 10:07 AM, Templin, Fred L fred.l.temp...@boeing.com wrote: Hi Roland, -Original Message- From: Dobbins, Roland [mailto:rdobb...@arbor.net] Sent: Monday, October 22, 2012 6:49 PM To: NANOG list Subject: Re: IP tunnel MTU On Oct 23, 2012, at 5:24 AM, Templin, Fred L wrote: Since tunnels always reduce the effective MTU seen by data packets due to the encapsulation overhead, the only two ways to accommodate the tunnel MTU is either through the use of path MTU discovery or through fragmentation and reassembly. Actually, you can set your tunnel MTU manually. For example, the typical MTU folks set for a GRE tunnel is 1476. Yes; I was aware of this. But, what I want to get to is setting the tunnel MTU to infinity. This isn't a new issue; it's been around ever since tunneling technologies have been around, and tons have been written on this topic. Look at your various router/switch vendor Web sites, archives of this list and others, etc. Sure. I've written a fair amount about it too over the span of the last ten years. What is new is that there is now a solution near at hand. So, it's been known about, dealt with, and documented for a long time. In terms of doing something about it, the answer there is a) to allow the requisite ICMP for PMTU-D to work to/through any networks within your span of administrative control and b) That does you no good if there is some other network further beyond your span of administrative control that does not allow the ICMP PTBs through. And, studies have shown this to be the case in a non-trivial number of instances. b) adjusting your own tunnel MTUs to appropriate values based upon experimentation. Adjust it down to
Re: Network scan tool/appliance horror stories
It all depends on what tools they are using and how you have your system setup. Both NMAP and Nessus can check system\service to see if common accounts have default or non password at all. This can cause these accounts to be locked out. There are other exploits that can cause systems\services to be DOS'd but these normally have to be enabled. Best to get a statement of works from them which should list all the tools including options they will be using. They also should be able to hand over a raw dump of ALL commands run during the testing. On 29 October 2012 19:25, Justin M. Streiner strei...@cluebyfour.orgwrote: On Mon, 29 Oct 2012, Pedersen, Sean wrote: We're evaluating several tools at the moment, and one vendor wants to dynamically scan our network to pick up hosts - SNMP, port-scans, WMI, the works. I was curious if anyone had any particularly gruesome horror stories of scanning tools run amok. If you have any overloaded/under-powered network gear, such as stateful firewalls and routers that do lots of NAT, you might find them very quickly, depending on how aggressive the scanning tool is. There might also be devices out there that, while possibly lightly loaded, can reach some minimally documented resource threshold under a very aggressive scan, and subsequently tip over. Also, if you're doing IPv6, the performance metrics for many network devices can be a bit more of a moving target. jms -- ??? BaconZombie LOAD *,8,1
Re: Network scan tool/appliance horror stories
On 29/10/2012 19:25, Justin M. Streiner wrote: Also, if you're doing IPv6, the performance metrics for many network devices can be a bit more of a moving target. I'd almost be tempted to set up a few machines doing v6 only on the LAN, with some trivial to exploit telnet/SNMP access then invite them to scan the LAN and see if said machines are picked up. My experience of these things a year or two ago was that most of these companies thought everyone had an internal flat IPv4 network in RFC1918 space and that was that. YMMV of course. Paul. -- Paul Thornton
[NANOG-announce] 2012 Program Committee Appointments Announcement
*Greetings NANOG Colleagues, * * The Board has completed the Program Committee selection process. This year, twenty members submitted their candidacies for eight available positions. We want to thank each and every one of them for considering this important service to our community and encourage them to try for the next selection cycle. We are pleased to announce the two-year term appointment of Philippe Couture, Greg Dendy, Ryan Donnelly, Chris Grundemann, Elisa Jasinska, Anton Kapela, John van Oppen and Dave Temkin to the Program Committee. We also want to thank and recognize the contribution of Kevin Epperson for his service during 2010-2012 and the contributions of Nina Bargisen, Tom Daly, Dave Meyer and Tom Scholl for their 4-year service on the 2008-2012 Program Committees. In the coming weeks, the new Program Committee will hold its first meeting and select a Chair and a Vice-Chair. On behalf of the Board, *Sylvie -- Sylvie LaPerriere Chair, NANOG Board of Directors www.nanog.org ___ NANOG-announce mailing list nanog-annou...@mailman.nanog.org http://mailman.nanog.org/mailman/listinfo/nanog-announce
Re: Network scan tool/appliance horror stories
I heard a story in the past year of someone that had a system get scanned and it opened a ticket with their IT department for each time they scanned them. Eventually the IT department system crashed due to the excessive number of tickets being opened by their scanning tool. The network was properly exempted from the future scans after the system had to be recovered from backup. - Jared LOAD *,8,1 ^ yay
Re: Network scan tool/appliance horror stories
On 10/29/12 12:10 -0700, Pedersen, Sean wrote: We're evaluating several tools at the moment, and one vendor wants to dynamically scan our network to pick up hosts - SNMP, port-scans, WMI, the works. I was curious if anyone had any particularly gruesome horror stories of scanning tools run amok. http://www.tulsaworld.com/news/article.aspx?subjectid=334articleid=20121002_11_A1_CUTLIN325691 A layer 7 failure. Make sure all members of your organization are aware of your plans. -- Dan White
Re: IP tunnel MTU
Templin, Fred L wrote: Yes; I was aware of this. But, what I want to get to is setting the tunnel MTU to infinity. Essentially, its time the network matured to the point where inter-networking actually works (again), seamlessly. I agree. Joe
Re: IP tunnel MTU
On Oct 29, 2012, at 3:46 PM, Joe Maimon jmai...@ttec.com wrote: Templin, Fred L wrote: Yes; I was aware of this. But, what I want to get to is setting the tunnel MTU to infinity. Essentially, its time the network matured to the point where inter-networking actually works (again), seamlessly. I agree. Certainly fixing all the buggy host stacks, firewall and compliance devices to realize that ICMP isn't bad won't be hard. - Jared
Re: IP tunnel MTU
On Mon, Oct 29, 2012 at 4:01 PM, Jared Mauch ja...@puck.nether.net wrote: On Oct 29, 2012, at 3:46 PM, Joe Maimon jmai...@ttec.com wrote: Templin, Fred L wrote: Yes; I was aware of this. But, what I want to get to is setting the tunnel MTU to infinity. Essentially, its time the network matured to the point where inter-networking actually works (again), seamlessly. I agree. Certainly fixing all the buggy host stacks, firewall and compliance devices to realize that ICMP isn't bad won't be hard. - Jared Wait till you get started on fixing the security consultants. -- Tim:
[NANOG-announce] 2012 Development Committee Appointments Announcement
*Greetings NANOG Colleagues, * * * *The Board has completed the Development Committee selection process for 2012. We are pleased to announce the two-year term appointment of Michael Buchner, Jezzibell Gilmore, Gina Haspillaire and Misako Manca and the one-year term appointment of Michael Rascoe to the Development Committee. We also want to thank and recognize Cat Rodery for designing the members benefits during her tenure as Membership Vice Chair. In the coming weeks, the new Development Committee will hold its first meeting and select a Chair and a Vice-Chair. On behalf of the Board, * -- Sylvie LaPerriere Chair, NANOG Board of Directors www.nanog.org ___ NANOG-announce mailing list nanog-annou...@mailman.nanog.org http://mailman.nanog.org/mailman/listinfo/nanog-announce
Re: IP tunnel MTU
On Mon, Oct 29, 2012 at 03:46:57PM -0400, Joe Maimon wrote: Templin, Fred L wrote: Yes; I was aware of this. But, what I want to get to is setting the tunnel MTU to infinity. Essentially, its time the network matured to the point where inter-networking actually works (again), seamlessly. I agree. Joe you mean its safe to turn off the VPNs? /bill
Re: IP tunnel MTU
Jared Mauch wrote: On Oct 29, 2012, at 3:46 PM, Joe Maimon jmai...@ttec.com wrote: Templin, Fred L wrote: Yes; I was aware of this. But, what I want to get to is setting the tunnel MTU to infinity. Essentially, its time the network matured to the point where inter-networking actually works (again), seamlessly. I agree. Certainly fixing all the buggy host stacks, firewall and compliance devices to realize that ICMP isn't bad won't be hard. - Jared ICMP is just not the way it is ever going to work. Joe
Re: IP tunnel MTU
bmann...@vacation.karoshi.com wrote: On Mon, Oct 29, 2012 at 03:46:57PM -0400, Joe Maimon wrote: Templin, Fred L wrote: Yes; I was aware of this. But, what I want to get to is setting the tunnel MTU to infinity. Essentially, its time the network matured to the point where inter-networking actually works (again), seamlessly. I agree. Joe you mean its safe to turn off the VPNs? /bill Quite the reverse. Joe
Re: IP tunnel MTU
On Oct 29, 2012, at 4:43 PM, Joe Maimon jmai...@ttec.com wrote: Jared Mauch wrote: On Oct 29, 2012, at 3:46 PM, Joe Maimon jmai...@ttec.com wrote: Templin, Fred L wrote: Yes; I was aware of this. But, what I want to get to is setting the tunnel MTU to infinity. Essentially, its time the network matured to the point where inter-networking actually works (again), seamlessly. I agree. Certainly fixing all the buggy host stacks, firewall and compliance devices to realize that ICMP isn't bad won't be hard. - Jared ICMP is just not the way it is ever going to work. I wish you luck in getting your host IP stacks to work properly without ICMP, especially as you deploy IPv6. - Jared
RE: Network scan tool/appliance horror stories
During scans at various times in the past (and depending on throttling and settings of that scan) we've seen: 1) small remote site firewalls doing site to site vpns drop a small number of packets 2) locally installed remote control service popup a 'user has been disconnected' error on PCs when port scanned 3) some devices send alerts like 'Unauthorized attempt to gain access' when their SNMP ports are hit with non-standard community strings 4) logging on some devices that causes concern for the admin of that device (Is someone hacking my device?) 5) out of date/non-patched (yet critical) applications and/or web servers crashing/locking up (this occurred on specific nessus scans, not a generic port/snmp scan) 6) large stacks of 3750s (six or more members) have issues around CPU during certain SNMP commands (I want to say some sort of getbulk type of command) The first four were pretty minor although #3 could generate a lot of calls to the support center. #5 was a big deal due to the nature of the application. #6 was impactful because we dropped routing neighbors for about 10 seconds but this was a couple of years ago so may have been an old IOS bug. -Original Message- From: Pedersen, Sean [mailto:sean.peder...@usairways.com] Sent: Monday, October 29, 2012 12:11 PM To: nanog@nanog.org Subject: Network scan tool/appliance horror stories We're evaluating several tools at the moment, and one vendor wants to dynamically scan our network to pick up hosts - SNMP, port-scans, WMI, the works. I was curious if anyone had any particularly gruesome horror stories of scanning tools run amok.
RE: IP tunnel MTU
I wish you luck in getting your host IP stacks to work properly without ICMP, especially as you deploy IPv6. From what I've heard, ICMPv6 is already being filtered, including PTBs. I have also heard that IPv6 fragments are also being dropped unconditionally along some paths. So, if neither ICMPv6 PTB nor IPv6 fragmentation works then the tunnel endpoints have to take matters into their own hands. That's where SEAL comes in. Thanks - Fred fred.l.temp...@boeing.com - Jared
Re: IP tunnel MTU
Jared Mauch wrote: ICMP is just not the way it is ever going to work. I wish you luck in getting your host IP stacks to work properly without ICMP, especially as you deploy IPv6. - Jared Precisely the state we are in. Looking for luck. Joe
Re: IP tunnel MTU
On Mon, Oct 29, 2012 at 04:44:40PM -0400, Joe Maimon wrote: bmann...@vacation.karoshi.com wrote: On Mon, Oct 29, 2012 at 03:46:57PM -0400, Joe Maimon wrote: Templin, Fred L wrote: Yes; I was aware of this. But, what I want to get to is setting the tunnel MTU to infinity. Essentially, its time the network matured to the point where inter-networking actually works (again), seamlessly. I agree. Joe you mean its safe to turn off the VPNs? /bill Quite the reverse. Joe so its tunnels all the way down... maybe we should just go back to a circuit oriented network, eh? /bill
[NANOG-announce] 2012 Communications Committee Appointments Announcement
*Greetings NANOG Colleagues, * * The Board has completed the Communications Committee selection process for 2012. We are pleased to announce the two-year term appointment of Larry Blunk, Colin Corbett and Andrew Koch to the Communications Committee. We also want to thank and recognize Randy Epstein for serving four years on the CC and for his leadership as Chair this past year. In the coming weeks, the new Development Committee will hold its first meeting and select a Chair and a Vice-Chair. On behalf of the Board, * Sylvie -- Sylvie LaPerriere Chair, NANOG Board of Directors www.nanog.org ___ NANOG-announce mailing list nanog-annou...@mailman.nanog.org http://mailman.nanog.org/mailman/listinfo/nanog-announce
Re: IP tunnel MTU
bmann...@vacation.karoshi.com wrote: you mean its safe to turn off the VPNs? /bill Quite the reverse. Joe so its tunnels all the way down... maybe we should just go back to a circuit oriented network, eh? /bill Its not safe to turn on VPNs. Joe
Re: IP tunnel MTU
On Mon, Oct 29, 2012 at 10:54 AM, Ray Soucy r...@maine.edu wrote: The core issue here is TCP MSS. PMTUD is a dynamic process for adjusting MSS, but requires that ICMP be permitted to negotiate the connection. The realistic alternative, in a world that filters all ICMP traffic, is to manually rewrite the MSS. In IOS this can be achieved via ip tcp adjust-mss and on Linux-based systems, netfilter can be used to adjust MSS for example. Longer term, the ideal solution would be a replacement algorithm that allows TCP to adjust its MSS with or without negative acknowledgement from intermediate routers. The ICMP-didn't-get-there problem is only going to get worse and things like private IPs on routers and encapsulation mechanisms where the intermediate router isn't dealing with an IP packet directly are as much at fault these days as foolish firewall admins. Perhaps my understanding of end-to-end is flawed, but I suspect it means that an endpoint shouldn't depend on direct communication with an intermediate system for its successful communication with another endpoint. Maybe something as simple as clearing the don't fragment flag and adding a TCP option to report receipt of a fragmented packet along with the fragment sizes back to the sender so he can adjust his mss to avoid fragmentation. Regards, Bill Herrin -- William D. Herrin her...@dirtside.com b...@herrin.us 3005 Crane Dr. .. Web: http://bill.herrin.us/ Falls Church, VA 22042-3004
RE: IP tunnel MTU
Hi Bill, Maybe something as simple as clearing the don't fragment flag and adding a TCP option to report receipt of a fragmented packet along with the fragment sizes back to the sender so he can adjust his mss to avoid fragmentation. That is in fact what SEAL is doing, but there is no guarantee that the size of the largest fragment is going to be an accurate reflection of the true path MTU. RFC1812 made sure of that when it more or less gave IPv4 routers permission to fragment packets pretty much any way they want. Thanks - Fred fred.l.temp...@boeing.com
Re: IP tunnel MTU
True, but it could be used as an alternative PMTUD algorithm - raise the segment size and wait for the I got this as fragments option to show up... Of course, this only works for IPv4. IPv6 users are SOL if something in the middle is dropping ICMPv6. -C On Oct 29, 2012, at 4:02 PM, Templin, Fred L wrote: Hi Bill, Maybe something as simple as clearing the don't fragment flag and adding a TCP option to report receipt of a fragmented packet along with the fragment sizes back to the sender so he can adjust his mss to avoid fragmentation. That is in fact what SEAL is doing, but there is no guarantee that the size of the largest fragment is going to be an accurate reflection of the true path MTU. RFC1812 made sure of that when it more or less gave IPv4 routers permission to fragment packets pretty much any way they want. Thanks - Fred fred.l.temp...@boeing.com
Re: Network scan tool/appliance horror stories
On Mon, Oct 29, 2012 at 2:10 PM, Pedersen, Sean sean.peder...@usairways.com wrote: I was curious if anyone had any particularly gruesome horror stories of scanning tools run amok. A particular model of ShoreTel voice switches I used to administer (running VxWorks, IIRC) would reliably lock up hard when hit with nmap's OS/service detection on a particular port. Required pulling the plug to restore service. The truly odd thing was that it didn't seem like a resource exhaustion issue, it could be triggered with a single well-crafted probe or two. After several long nights of painful troubleshooting with their level III support, we came to the conclusion that if it hurts, you probably shouldn't do it, and mitigating ACLs were put in place. -n
Re: IPv4 address length technical design
On 10/03/2012 09:52 AM, Seth Mos wrote: Op 3-10-2012 18:33, Kevin Broderick schreef: I'll add that in the mid-90's, in a University Of Washington lecture hall, Vint Cerf expressed some regret over going with 32 bits. Chuckle worthy and at the time, and a fond memory - K Pick a number between this and that. It's the 80's and you can still count the computers in the world. :) Oops... And that was not quite what Mr Cerf meant to do. I finally got around to finding a reply Vint Cerf wrote to a thread I started a year or two ago. The url is http://mailman.nanog.org/pipermail/nanog/2010-April/020488.html and quoted below in full for future prosperity. This gives a great behind the scenes view and a clear idea on the thought processes involved and why things evolved the way they did. Interestingly ipv6 is 128 bits, and I personally would have loved to see variable length address structures being implemented, alas. Maybe when ipv6 is in need of replacement... * Begin quote * Hi, Vint Cerf kindly sent through some more explanation. Regards, Mark. Begin forwarded message: Date: Sat, 3 Apr 2010 08:17:28 -0400 From: Vint Cerf vint at google.com To: Mark Smith nanog at 85d5b20a518b8f6864949bd940457dc124746ddc.nosense.org Cc: Andrew Gray 3356 at blargh.com, NANOG List nanog at nanog.org Subject: Re: legacy /8 When the Internet design work began, there were only a few fairly large networks around. ARPANET was one. The Packet Radio and Packet Satellite networks were still largely nascent. Ethernet had been implemented in one place: Xerox PARC. We had no way to know whether the Internet idea was going to work. We knew that the NCP protocol was inadequate for lossy network operation (think: PRNET and Ethernet in particular). This was a RESEARCH project. We assumed that national scale networks were expensive so there would not be too many of them. And we certainly did not think there would be many built for a proof of concept. So 8 bits seemed reasonable. Later, with local networks becoming popular, we shifted to the class A-D address structure and when class B was near exhaustion, the NSFNET team (I think specifically Hans-Werner Braun but perhaps others also) came up with CIDR and the use of masks to indicate the size of the network part of the 32 bit address structure. By 1990 (7 years after the operational start of the Internet and 17 years since its basic design), it seemed clear that the 32 bit space would be exhausted and the long debate about IPng that became IPv6 began. CIDR slowed the rate of consumption through more efficient allocation of network addresses but now, in 2010, we face imminent exhaustion of the 32 bit structure and must move to IPv6. Part of the reason for not changing to a larger address space sooner had to do with the fact that there were a fairly large number of operating systems in use and every one of them would have had to be modified to run a new TCP and IP protocol. So the hacks seemed the more convenient alternative. There had been debates during the 1976 year about address size and proposals ranged from 32 to 128 bit to variable length address structures. No convergence appeared and, as the program manager at DARPA, I felt it necessary to simply declare a choice. At the time (1977), it seemed to me wasteful to select 128 bits and variable length address structures led to a lot of processing overhead per packet to find the various fields of the IP packet format. So I chose 32 bits. vint * end quote * -- Earthquake Magnitude: 4.7 Date: Monday, October 29, 2012 23:51:42 UTC Location: Flores region, Indonesia Latitude: -8.1762; Longitude: 123.4122 Depth: 19.60 km
Re: the little ssh that (sometimes) couldn't
: :corruption! : : :http://mina.naguib.ca/blog/2012/10/22/the-little-ssh-that-sometimes-couldnt.html I ran into a similar issue with a customer just a few days ago! The customer's theory was that there was something badly wrong with their dorky gateway/switch (which we sold and support sigh). ssh was timing out, with a SSH2_MSG_KEX_DH_GEX_GROUP hang/failure during the ssh protocol exchange. Based on that, some wireshark captures, and and stray Google droppings, I advised them to ratchet down the MTU to make things work. Through bisectional MTU settings and pinging, we arrived at an MTU of 850. And I initially started cursing at the switch (because that helps move packets, really :) ). Turns out -- the ssh server in question was running RHEL 5.x Linux, and that was the key. Even though ip route show cache looked sane, ip route flush cache (which I had them run, just on a lark) made the problem go away. So it probably wasn't my switch (unless it had done something untoward in the distant past that induced some weird Linux stack bug). I'm mostly posting this because I was wondering if anyone else had run into an MTU of 850 before. Is that a magic number that rings any bells (or perhaps has seen the Linux route cache behavior I did). -Mike -- Michael J. O'Connor m...@dojo.mi.org =--==--==--==--==--==--==--==--==--==--==--==--==--==--==--==--==--==--==--= It is now the age of now.-Non Campus Mentis pgpco3nCOlAoW.pgp Description: PGP signature
Re: IP tunnel MTU
Templin, Fred L wrote: I wish you luck in getting your host IP stacks to work properly without ICMP, especially as you deploy IPv6. From what I've heard, ICMPv6 is already being filtered, including PTBs. As v6 PTBs are specified to be generated even against multicast packets, it is of course that they are dropped to prevent ICMP implosions. But, it is a very serious problem of not only tunnels but entire IPv6. That is, if PMTUD is unavailable, IPv6 hosts are prohibited to send packets larger than 1280B. Then, ignoring the prohibition, tunnel end points may send packets a little larger than 1280B, which means physical link MTU of 1500B or a little smaller than that is enough for nested tunnels. Thus, no new tunneling protocol is necessary. The harder part of the job is to disable PMTUD on all the IPv6 implementations. I have also heard that IPv6 fragments are also being dropped unconditionally along some paths. Again, it is not a problem of tunnels only. If that is the operational reality, specifications on fragmentation must be dropped from IPv6 specification. Masataka Ohta
Re: Network scan tool/appliance horror stories
On Oct 29, 2012, at 3:55 PM, Rutis, Cameron 6) large stacks of 3750s (six or more members) have issues around CPU during certain SNMP commands (I want to say some sort of getbulk type of command) The first four were pretty minor although #3 could generate a lot of calls to the support center. #5 was a big deal due to the nature of the application. #6 was impactful because we dropped routing neighbors for about 10 seconds but this was a couple of years ago so may have been an old IOS bug. Saw the same. All of our 3750 stacks (which are small) committed suicide during a trial of Foglight. We had discovery timings turned way down, but it still caused a reload on a mix of the last supposedly really stable releases of 12.x. Not confidence inspiring. TAC was useless and suggested a v15 upgrade despite no known fix. The proposed v15 upgrade sent our lab boxes into continuous reload unless you broke the stack and manually wiped each switch. Oh, and port 28 was invisible on each switch after upgrade, and Gi2/0/28 would throw a syntax error. Wait for new releases, lather, rinse, repeat. Total time to resolution in production was several man-weeks on our side, and a few months calendar time, all because the discovery scan revealed how great a software company Cisco has become.
Re: Network scan tool/appliance horror stories
On Mon, Oct 29, 2012 at 12:10:40PM -0700, Pedersen, Sean wrote: We're evaluating several tools at the moment, and one vendor wants to dynamically scan our network to pick up hosts - SNMP, port-scans, WMI, the works. I was curious if anyone had any particularly gruesome horror stories of scanning tools run amok. Check your netmask on the to-be-discovered network and what the rate of discovery is. I have seen internal systems attempt to scan and discover nodes in a /16 and promptly set off a flood of alarms on all PDUs (6 per rack) and plenty of other devices that thought they are being attacked. -andreas
