I had never considered the remote end. Just for grins I put overridemtu=1200 on the remote end ipsec.conf and low and behold data transfers!!!
I suspect I have patched the problem, but not addressed it. Does this change any of the steps I should be doing to continue troubleshooting or should I just tweak the remote overridemtu=1200 until I find the max that works? Thank you Charles for a huge chunk of your time!!! - Todd > -----Original Message----- > From: Charles Steinkuehler [mailto:[EMAIL PROTECTED]] > Sent: Thursday, February 13, 2003 4:14 PM > To: Todd Pearsall > Cc: [EMAIL PROTECTED] > Subject: Re: [leaf-user] PPPoE, IPSec and MTU size problems > > > Todd Pearsall wrote: > > Below are tcpdumps from the eth1 and then the ipsec0 > interfaces. Any > > help in making some sense would be great. Let me know if > other options, > > test, etc. would be more useful. If it gets too wrapped in > e-mail I can > > make it available on the web. > > It's not too hard to piece back together. > > > User on the local end makes a FTP connection remote server > > > > These dumps include the user performing an 'ls' on a small > directory and > > then a 'get' for a file. The get hangs and eventually times out. > > > # tcpdump -n -i eth1 net 172.30.85.0 mask 255.255.255.0 > > tcpdump: listening on eth1 > > 15:13:02.074186 192.168.5.13.4426 > 172.30.85.100.21: P > > 2956246103:2956246129(26) ack 3802495246 win 63620 (DF) > > Your problem is immediately apparent. The (DF) in *EVERY ONE* of the > packets below means "Don't Fragment", which prevents the router from > breaking appart your oversized packets and sending them in > multiple pieces. > > Combined with the MSS of 1460 (which I think is larger than the data > size left after PPPoE and IPSec overhead) and broken ICMP > delivery, and > you wind up with the results you've been seeing. > > The question now is *WHY* is the local client requesting "Don't > Fragment" on all traffic, why it doesn't get handled automatically by > the network stacks, and what can be done to fix it. > > NOTE: The don't fragment option could be getting set as part of Path > MTU discovery, which is a good thing. For this to work properly, > however, the ICMP error messages about a packet with don't > fragment set > getting dropped need to make it back to the sending host. > > At this point, you need to: > > - Figure out why the don't fragment bit is getting set. > > - Figure out why ICMP error messages are not being returned to the > sender, or why the sender is ignoring them (note the ICMP > error would be > generated by a system somewhere between the remote end and your local > firewall, and should be headed back to the remote FTP server). > > - Run a trace on both ends of the connection simultaniously, > so we can > see what's going on. Looks like most of the "fun stuff" is > happening on > the server side... > > Start with tracing the data on both ends, so we can digest it while > you're looking into the other issues. > > At this point, if I had to hazzard a guess, I suspect the ftp return > traffic (which will be a large packet) is sent by your remote FTP > system, encrypted into a 1500 byte (or there abouts) packet at your > firewall and thrown out to the internet. It then makes its > way to your > PPPoE system, where it can't be squeezed down the PPPoE link w/o > fragmenting. At this point, an ICMP error message should be > generated > by your ISP's router (which has to drop the packet, since the > DF flag is > set), but they are probably blocking all ICMP traffic at > their boarder, > so the ICMP error packet never makes it back to the FTP > server, causing > Path MTU discovery to break, and your connection "hangs" as a result. > > *WARNING* The above is still just a guess...we really need to see the > server-side traffic dump. > > You might also try setting the MSS on shorewall to whatever a > 1500 byte > packet minus the PPPoP and IPSec wrappers comes out to > (should be online > somewhere). > > -- > Charles Steinkuehler > [EMAIL PROTECTED] > > ------------------------------------------------------- This SF.NET email is sponsored by: FREE SSL Guide from Thawte are you planning your Web Server Security? Click here to get a FREE Thawte SSL guide and find the answers to all your SSL security issues. http://ads.sourceforge.net/cgi-bin/redirect.pl?thaw0026en ------------------------------------------------------------------------ leaf-user mailing list: [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/leaf-user SR FAQ: http://leaf-project.org/pub/doc/docmanager/docid_1891.html
