The issue has gone away as per my last email. I think that maybe the NIC tuning fixed it. I did replace a squashed UTP cable, but if that was causing problems I would have expected to see some Ethernet faults during my packet sniffing session.
I really don't like it when there is no black and white problem/solution. I am just waiting for this to come back and haunt me. Unfortunately, when there is no symptom, it's hard to continue looking for the problem. One benefit of this exercise was finding a couple of un-related mistakes/omissions in a few of the router configs. There is a positive in every negative :) Thanks again, Symon -----Original Message----- From: Priscilla Oppenheimer [mailto:[EMAIL PROTECTED] Sent: 07 March 2003 19:07 To: [EMAIL PROTECTED] Subject: RE: SMTP Time outs and re-transmissions, multiple [7:64617] Symon Thurlow wrote: > > HI Priscilla, > > Thanks for your response. This problem is driving me nuts. > > Looking at sh int, sh frame pvc etc I can't see any significant errors > or problems. > > We have been using auth for some time. No significant changes, other > than a power down on Friday, problem notice don Monday. I have gone > over the router configs and nothing looks wrong. No routes flapping, > all times on routes are consistent. There are no firewalls or access > lists in this equation. > > From the client side, this is the packet flow: > > 3 way handshake ok > Final ack from the client (WINXP) declares a 62420 byte window size. > SMTP starts, ehlo, auth, mail from, etc all ok. > After "send data..." from the server, client sends an ack, > with Windows size 63795 (I guess the client is thinking at this > point that it has a good fast connection, and the server is > emptying it's buffers quickly?). > > I then see 5 packets of SMTP data at > > ARRRGHHH!!!! It's 8.15pm, i'm still at the clients site and my laptop > just blue screened (it has the captures...). yay. > > Right, I then see 5 packets of SMTP data (from client to > server) at 1514 bytes a piece. I then see one packet with smtp data > from client to server at 946 bytes. not sure why.> > Then I see 3 acks from the server to the client, ack'ing the > 3rd, 5th, and get this the NEXT SMTP data packet. TCP uses a forward acknowledgement, so that may be normal. Please remember also that TCP ACKs bytes, not packets. The sequence number identifies the byte number in the stream of bytes. It identifies the byte number of the first byte in the segment being sent. The ACK number indentifies the next expected byte. > It looks like > the packet and ack were recieved at the same time. THe acks all have a > window size of 8760. This tells me that either the recieve buffers are > full or set too small on the server? Usually servers have larger receive windows than that these days, but not necessarily. I've seen a lot that have a similar size, and it shouldn't really matter with SMTP I don't think. To tune TCP's window size you would have to tune the TCP stack in the registry probably. You mentioned tuning the buffers on the NIC. I don't think that helps TCP. I don't think TCP knows about the NIC's buffers. That's not to say that tuning the NIC buffers won't help the problem. It might cause the NIC to drop fewer frames, which would certainly help. > > The above 5 packets and 3 acks happens another 4 times (the email is > about 400 Kbytes), then the re-transmissions start. > > I get 6 retransmissions of the 4th to last SMTP data packet from the > client to the server, then the whole sequence starts all over again > from 3 way handshake. Sounds like the server stopped ACKing or left a hole in what it has ACKed? You'll have to study the ACK numbers carefully. It sure sounds like the server has some problems and is losing packets somewhere, either hardware or software. Maybe you need to replace its NIC? You said you tuned the buffers which might help too. ? No rst or fin or anything. > > On the mail server I can see it logging that it is waiting for the > message body, and times out waiting. > > Perhaps the buffer fills up on the server and it can;t empty it > anymore? In this case I would expect to see some sort of notification > from the server to stop sending (a reduced window size perhpas?) Yes, that's what you should expect if TCP gets behind. More evidence that the problem is lower down, with the NIC perhaps?? > > I'm going to train it home (with my nose in TCP/IP illustrated) then > study the captures some more. Sounds like a good idea. TCP/IP Illustrated is great. > If anyone is interested, I > can send you the captures, they are from Sniffer Pro. If you get stuck, I'll take a look. I predict you'll figure it out yourself, though. Good luck. Priscilla > > Cheers, > > Symon > > > > ________________________________ > > From: Priscilla Oppenheimer [mailto:[EMAIL PROTECTED] > Sent: Thu 06/03/2003 6:32 PM > To: [EMAIL PROTECTED] > Subject: RE: SMTP Time outs and re-transmissions, multiple TCP > [7:64617] > > > Symon Thurlow wrote: > > > > Hi all, > > > > I am trying to problem solve and issue where I have multiple frame > > sites > > (1750's) all connecting to a site (3640) to send and receive email > > (POP3 SMTP). > > > > They are having problems sending SMTP. Looking at a packet trace > > from a site trying to send SMTP, I see lots of re-transmissions. > > > > I have checked with the Frame provider and they assure me that no > > packets are being dropped, even though there are a lot getting > > marked > > DE. > > > > I have pretty much excluded the server from being an issue. > > > > IN the packet sniff I also see multiple acks with the same sequence > > number, one after the other. This does not look right to me. I > > am about > > to delve in to TCP/IP illustrated to find out about it, but > > does this > > behaviour trigger anything in your minds? > > Multiple ACKs with the same sequence number are probably just TCP > keepalives. The original TCP RFC didn't mention a keepalive > process but the > Host Requirements RFC does. That's how it works. After some > time of silence > a host just sends the last ACK over and over again for a couple > hours > (typical timeout). > > Is the client side sending these multiple ACKs? > > If the server were sending them, and you're seeing them on the client > side, then you would have some proof that the server can at least get > through, but > I bet it's the client side sending them? > > If you're sniffing on the client side and you see it send > retransmissions and multiple ACKs, but you don't see much from the > server, I would suspect > that the server's packets are getting lost. But what would > cause SMTP to get > lost but not POP3? A firewall misconfiguration? An MTU issue? > > What is the last packet exchange you see for SMTP? > > What error do the users see? > > For SMTP, did you recently start using authentication or any other new > features? We recently started requiring our users to use > authentication and > had lots of strange problems due to the mix of operating > systems. Some of > them didn't behave correctly at all, although their misbehavior > wasn't > exactly what you're describing. > > Can the users still receive mail? I think you implied that they can. > POP3 is based on TCP too. Does it still work? If it still works, but > SMTP doesn't, > that's a huge clue that TCP and the internetwork are fine. > > If POP has problems too, start bugging the Frame provider more and do > some checking yourself with "show int serial?" Are there lots of > errors? > > Actually, before you blame the Frame provider, though, if possible do > some sniffing on the server side and make sure the server is really > still sending > packets and they are making it through local routers and > firewalls. Make > sure it is sending packets with data, not just ACKs. It might > be hard to > sniff on the outside side of the firewall, but if you can that > would a good > spot. > > Speaking of firewalls, are there any in the picture that could be > messing with TCP and causing problems or timing out? It scares me how > much you tell > a firewall to mess with TCP. > > What else has changed recently, if anything? > > Feel free to send us more info. Thanks, > > Priscilla > > > > > > Cheers, > > > > Symon > ============================================= > > This email has been content filtered and > subject to spam filtering. If you consider > this email is unsolicited please forward > the email to [EMAIL PROTECTED] and > request that the sender's domain be > blocked from sending any further emails. > > ============================================= ============================================= This email has been content filtered and subject to spam filtering. If you consider this email is unsolicited please forward the email to [EMAIL PROTECTED] and request that the sender's domain be blocked from sending any further emails. ============================================= Message Posted at: http://www.groupstudy.com/form/read.php?f=7&i=64776&t=64617 -------------------------------------------------- FAQ, list archives, and subscription info: http://www.groupstudy.com/list/cisco.html Report misconduct and Nondisclosure violations to [EMAIL PROTECTED]
