A good app / stack / whatever will not create packets larger than the network MTU. If I have an ExaByte to send, I SHOULD create a whole bunch of "mtu<overhead>" packets; not create a bunch of maximum size datagrams (4GB?) and force the host(s) to frag / defrag them. Can you imagine? I know of several ISP's that drop fragments and don't allow their devices to perform frag / defrags. Typically those processes are done in software and it's WAY too easy to DOS a system.
I'll enable a packet capture on the cisco router and post it. I'm sure it will concur with what tcpdump says. Thanks for the help thus far! G -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Rick Jones Sent: Tuesday, May 31, 2011 4:33 PM To: [email protected] Subject: Re: [Ntop] Cisco netflow packet size - adjusting? On Tue, 2011-05-31 at 16:05 -0500, Gary Gatten wrote: > Some interesting notes. Assuming I agree for now, I adjusted my filter: > tcpdump -vvv -nn -s 1524 host 1.2.3.4. > > Same results: > tcpdump: listening on rl0, link-type EN10MB (Ethernet), capture size 1524 > bytes > 16:34:41.421480 IP (tos 0x0, ttl 253, id 14766, offset 0, flags [+], proto: > UDP (17), length: 1396) 1.2.3.4.50769 > 5.5.5.5.2060: [bad udp cksum e710!] > UDP, length 1464 If that is all that is being captured, then something is swallowing the second IP datagram fragment of the IP datagram carrying the UDP datagram carrying the netflow PDU that describes the flows that Jack built... > This capture was on a FBSD box (vs RHEL) and outside the PIX FW. > > The whole issue that brought me here is several exporters NOT showing > up in ntop. The common thread was those with high enough flows that > they "need" large packets. My guess is ntop is not seeing 70% of my > netflow packets from remote offices due to this issue. Not sure how > to tell, except maybe turn on debug for netflow and listen to only a > single office, else it would be WAY too much data... > > Yes - if the MTU of the interface is n, it should not format a packet > larger than n. Even M$ does this correctly! You still haven't got data showing that anyone was actually trying to send *packets* on the wire larger than the MTU. (aka, IIRC, giants) What data there is, suggests that at least as far as UDP and IP over Ethernet are concerned there are no *packets* (eg frames) being sent which are larger than the MTU. I believe we are having a Can vs May vs Should discussion, intertwined with the responsibility and terminology of the various layers and perhaps what constitutes a "packet." Applications are allowed to ask UDP to send up to just shy of 64 KB of data per message, regardless of MTU (May). They have the socket calls etc with which to do so (Can). Now, that said, indeed it is not a Good Idea (tm) for an application to send a message via UDP that would cause IP to have to fragment the UDP datagram carrying the user's message (Should not). But that is not an all-or-nothing "should not" - there is leeway for considering circumstances. The main reason for the "should not" is the "all or nothing" nature of IP fragmentation and reassembly - IP does not retransmit, and all the fragments of an IP datagram must arrive at the receiver to reassemble the datagram. An underlying packet loss rate then is "amplified" into a larger IP datagram loss rate. But, it is still *perfectly* *legal* (can and may) as far as any of the specifications are concerned for an application to send messages larger than link-local MTU at one time via UDP. UDP checks the user's message against its maximum message size limit (16 bits including headers), and adds the UDP header and passes to IP if that check passes. IP then checks the resulting datagram against the link-local MTU, sees the datagram is larger, and then fragments into N fragments each of which fits in the MTU. Those are then passed to Ethernet, which sends the frames on the wire - messages to datagrams, datagrams to frames, the "packets" (frames) are all <= MTU in size. Now, that does not mean that applications cannot use stack-specific means to determine link-local MTU and adjust their behaviour, but such applications must then be able to, in essence, do the same segmentation and reassembly that TCP does. And, short of additional headers in the application's PDU (eg netflow or sflow) to say "this PDU is incomplete, wait for the rest of it" a protocol such as netflow or sflow (or DNS) could have a damn difficult time of it if it were *required* to honor the link-local MTU when using UDP. rick (OK, perhaps sflow/netflow isn't the *best* example, because their PDU's often hold several, smaller, otherwise independent things, which could be sent separately, but I hope the gist of what I'm trying to convey is there) > > G > > > -----Original Message----- > From: [email protected] > [mailto:[email protected]] On Behalf Of Rick Jones > Sent: Tuesday, May 31, 2011 3:57 PM > To: [email protected] > Subject: Re: [Ntop] Cisco netflow packet size - adjusting? > > On Tue, 2011-05-31 at 15:20 -0500, Gary Gatten wrote: > > Tcpdump extract: notice all the 1464 byte packets are F'd, others OK: > > > > 15:16:24.818262 IP (tos 0x0, ttl 252, id 14766, offset 0, flags [none], > > proto: UDP (17), length: 1012) 10.15.26.200.50769 > > > monpapp1.waddell.com.netflow-Regn15: [udp sum ok] UDP, length 984 > > > > 15:16:28.822210 IP (tos 0x0, ttl 251, id 14766, offset 0, flags [+], proto: > > UDP (17), length: 1396) 10.15.26.200.50769 > > > monpapp1.waddell.com.netflow-Regn15: [bad udp cksum 55bc!] UDP, length 1464 > > > > 15:16:39.822211 IP (tos 0x0, ttl 252, id 14766, offset 0, flags [none], > > proto: UDP (17), length: 1060) 10.15.26.200.50769 > > > monpapp1.waddell.com.netflow-Regn15: [udp sum ok] UDP, length 1032 > > > > 15:16:46.822499 IP (tos 0x0, ttl 251, id 14766, offset 0, flags [+], proto: > > UDP (17), length: 1396) 10.15.26.200.50769 > > > monpapp1.waddell.com.netflow-Regn15: [bad udp cksum 6409!] UDP, length 1464 > > > > 15:16:50.821026 IP (tos 0x0, ttl 251, id 14766, offset 0, flags [+], proto: > > UDP (17), length: 1396) 10.15.26.200.50769 > > > monpapp1.waddell.com.netflow-Regn15: [bad udp cksum 9a8e!] UDP, length 1464 > > > > 15:16:55.819812 IP (tos 0x0, ttl 251, id 14766, offset 0, flags [+], proto: > > UDP (17), length: 1396) 10.15.26.200.50769 > > > monpapp1.waddell.com.netflow-Regn15: [bad udp cksum 2191!] UDP, length 1464 > > I don't believe that tcpdump will do IP datagram fragment reassembly, so > it cannot really validate the UDP checksum when the UDP datagram is > being carried in a fragmented IP datagram. Of course, one might argue > that when tcpdump has only the first fragment (the "+" means the More > Fragments bit is set) it should not report anything about the UDP > checksum in the first place. That is something you might bring-up to > the tcpdump-workers mailing list. And btw, a filter expression matching > on UDP port number (eg "port netflow" or "port sflow" will not capture > the other fragments of the IP datagram carrying the UDP datagram > because, as it is IP doing the fragmentation, the UDP header, which is > just payload to IP, is not replicated. > > So, from that tcpdump output at least, you don't really know that the > 1464 byte packets were indeed "F'd" as you so eloquently put it :) You > need to see UDP checksum failures reported by UDP at the receiver (eg in > something like netstat statistics) after IP fragment reassembly has > completed and the entire datagram has been passed to UDP. And/or IP > fragment reassembly errors increasing. > > Also, from your previous message: > > > By "not honor": If the traffic is generated by the local system, it > > should look at the MTU of the output interface before forming a packet > > and "honor" the MTU size of said interface. I have set the source and > > dest interface to have an MTU of 1400, yet it still sends 1464 byte > > packets. Not very nice! > > I assume you mean - sends it unfragmented right? It is perfectly > "legal" (if not necessarily advisable) for an application using UDP to > send messages larger than the link-local MTU. > > FWIW, that 1464 in the tcpdump output above is the *UDP* datagram > length. The length of the IP datagram is earlier in the line, and > coupled with the "[+]" it reporting 1396 does suggest that the IP > datagram carrying the UDP datagram was fragmented > > rick > > > -----Original Message----- > > From: [email protected] > > [mailto:[email protected]] On Behalf Of Rick Jones > > Sent: Tuesday, May 31, 2011 2:48 PM > > To: [email protected] > > Subject: Re: [Ntop] Cisco netflow packet size - adjusting? > > > > On Tue, 2011-05-31 at 14:37 -0500, Gary Gatten wrote: > > > Hello, > > > > > > Anyone know how to force the Cisco IOS, specifically netflow, to use > > > packet sizes smaller than 1464 bytes? It doesn't seem to be honoring > > > the MTU size of the interfaces. It gets fragmented and then Checksum > > > is invalid, and then ntop doesn't see the packets. Annoying.... > > > > Um, gets fragmented by *what* and where? Fragmentation isn't *supposed* > > to corrupt datagrams, by what is the checksum being invalid being > > reported? > > > > Also, whence 1464? The maximum UDP payload on a standard 1500 byte MTU > > Ethernet-like network is 1472 bytes before the IPv4 datagram carrying > > the UDP datagram carrying the user message has to be fragmented. Does > > your network have a smaller than 1500 byte MTU? > > > > While IP fragmentation is indeed frowned upon, "not honoring the MTU > > size" is a bit strong - UDP-using applications can indeed send messages > > large enough to require fragmentation by IP. For example EDNS in DNS > > will do so, and it isn't considered dishonorable :) > > > > rick jones > > > > > I've googled for hours and haven't found anything yet. > > > > > > TIA > > > > > > Gary > > > > > > Tags: netflow, ntop, exporter not recognized, cant see netflow traffic > > > > > > > > > > > > "This email is intended to be reviewed by only the intended recipient > > > and may contain information that is privileged and/or confidential. If > > > you are not the intended recipient, you are hereby notified that any > > > review, use, dissemination, disclosure or copying of this email and > > > its attachments, if any, is strictly prohibited. If you have received > > > this email in error, please immediately notify the sender by return > > > email and delete this email from your system." > > > _______________________________________________ > > > Ntop mailing list > > > [email protected] > > > http://listgateway.unipi.it/mailman/listinfo/ntop > > > > > > _______________________________________________ > > Ntop mailing list > > [email protected] > > http://listgateway.unipi.it/mailman/listinfo/ntop > > > > > > > > > > > > <font size="1"> > > <div style='border:none;border-bottom:double windowtext 2.25pt;padding:0in > > 0in 1.0pt 0in'> > > </div> > > "This email is intended to be reviewed by only the intended recipient > > and may contain information that is privileged and/or confidential. > > If you are not the intended recipient, you are hereby notified that > > any review, use, dissemination, disclosure or copying of this email > > and its attachments, if any, is strictly prohibited. If you have > > received this email in error, please immediately notify the sender by > > return email and delete this email from your system." > > </font> > > > > _______________________________________________ > > Ntop mailing list > > [email protected] > > http://listgateway.unipi.it/mailman/listinfo/ntop > > > _______________________________________________ > Ntop mailing list > [email protected] > http://listgateway.unipi.it/mailman/listinfo/ntop > > > > > > <font size="1"> > <div style='border:none;border-bottom:double windowtext 2.25pt;padding:0in > 0in 1.0pt 0in'> > </div> > "This email is intended to be reviewed by only the intended recipient > and may contain information that is privileged and/or confidential. > If you are not the intended recipient, you are hereby notified that > any review, use, dissemination, disclosure or copying of this email > and its attachments, if any, is strictly prohibited. If you have > received this email in error, please immediately notify the sender by > return email and delete this email from your system." > </font> > > _______________________________________________ > Ntop mailing list > [email protected] > http://listgateway.unipi.it/mailman/listinfo/ntop _______________________________________________ Ntop mailing list [email protected] http://listgateway.unipi.it/mailman/listinfo/ntop <font size="1"> <div style='border:none;border-bottom:double windowtext 2.25pt;padding:0in 0in 1.0pt 0in'> </div> "This email is intended to be reviewed by only the intended recipient and may contain information that is privileged and/or confidential. If you are not the intended recipient, you are hereby notified that any review, use, dissemination, disclosure or copying of this email and its attachments, if any, is strictly prohibited. If you have received this email in error, please immediately notify the sender by return email and delete this email from your system." </font> _______________________________________________ Ntop mailing list [email protected] http://listgateway.unipi.it/mailman/listinfo/ntop
