Re: How to disable software TCP checksumming?
"Louis A. Mamakos" wrote: > > Setting aside the degree to which you choose to be paranoid about > where data can be corrupted, and the likelyhood thereof-- there > is an architectural issue here, which is that the CRC provided > by your friendly neighborhood Ethernet NIC card only protects the > data over one Ethernet subnetwork. The TCP checksum, and TCP in > general, provides end-to-end semantics, independent of the > concatenation of physical links the TCP segments might transit from > the source to the destination. If you have an end-to-end protocol, > with end-to-end semantics, then you need and end-to-end mechanism. I think you need to read the Tigon II driver in the /sys/pci directory, and come back when you realize that it supports offloading the TCP checksum. FWIW: The Intel Gigabit, Tigon III, and several other Gigabit cards support this, as well. -- Terry To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: How to disable software TCP checksumming?
Setting aside the degree to which you choose to be paranoid about where data can be corrupted, and the likelyhood thereof-- there is an architectural issue here, which is that the CRC provided by your friendly neighborhood Ethernet NIC card only protects the data over one Ethernet subnetwork. The TCP checksum, and TCP in general, provides end-to-end semantics, independent of the concatenation of physical links the TCP segments might transit from the source to the destination. If you have an end-to-end protocol, with end-to-end semantics, then you need and end-to-end mechanism. Remember years ago when Sun by default shipped their OS with UDP checksums disabled for UDP-based NFS traffic. The position was that you had this strong 32 bit CRC protecting your NFS traffic between the client and server (on the same LAN, of course) and the UDP checksum was wasteful and degrading performance. And people had mysterious things happen to their files when running UDP/NFS over WAN links and other interesting media and circumstances. Everything old is new again. louie To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: How to disable software TCP checksumming?
In a message dated 06/09/2001 1:21:45 PM Eastern Daylight Time, [EMAIL PROTECTED] writes: > [EMAIL PROTECTED] wrote: > > > > This thread is baffling. The bottom line is that you cant > > trust data coming into your machine, and you have to > > checksum it. The link level check only verifies that what > > was sent by the last forwarding point is the same as what > > you got, but in NO WAY implies that all of the data is > > valid. A link level checksum pass is a necessary, but not > > a sufficient condition for the data being acceptable. There > > are scads of reasons that it could be bad. Disabling > > checksumming is a kludge that you may chose to do, but its > > never the right thing to do. > > Tell me: at what point are you willing to trust the data? > > Is "the card plus the motherboard" my machine, or is it > just "the motherboard"? > > The cards support offloading the checksum to hardware; > you are saying that it is never reasonable to do this. > > Fine. That basically means that we need to checksum the > contents of memory once they have been loaded into the > L2 cache, to make sure they arrived safely from memory. > > Wait... now we need to checksum them in L1 cache, once > they have arrived from L2. > no, if your system is bad, you are sunk in general. This discussion was transforming into a "the link level check is good enough" discussion, and that what I was commenting on. Of course if the hardware does the IP and tcp checksumming its adaquate, although unreported DMA errors are not that uncommon, so it is safer to do the checks AFTER the data has been copied to memory. It may be a mistake to assume that all bus transfers are 100% reliable. Bryan To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: How to disable software TCP checksumming?
> Tell me: at what point are you willing to trust the data? > > Is "the card plus the motherboard" my machine, or is it > just "the motherboard"? > > The cards support offloading the checksum to hardware; > you are saying that it is never reasonable to do this. No, that's not what's being said. > Fine. That basically means that we need to checksum the > contents of memory once they have been loaded into the > L2 cache, to make sure they arrived safely from memory. That's why you have an ECC memory datapath. > Wait... now we need to checksum them in L1 cache, once > they have arrived from L2. And that's why you run ECC on the L2 cache as well. > Wait... now we need to checksum them in the pipeline, as > they have arrived from our untrustworthy L1 cache... And... yes, you got it. Some people will even ECC the L1 cache. > At some point you have to draw the boundary between the > "inside" and the "outside". I am happy to draw this > boundary at the etherenet cable; why are you insisting > that I draw it at the PCI bus instead? Well, if PCI parity worked, you could trust the PCI bus more, but the bottom line is simply that you can't trust *anything*, and the more that you do trust, the more you expose yourself to risk. > Does your requirement apply to devices integrated on the > motherboard, or is your concern solely with the mechanical > PCI connectors between the boards and the motherboard? PCI is typically implemented as a no-parity bus. Anything from poor drive to coupled noise to a simple alpha-decay error in a pipeline FIFO somewhere will cause PCI bus data corruption. -- ... every activity meets with opposition, everyone who acts has his rivals and unfortunately opponents also. But not because people want to be opponents, rather because the tasks and relationships force people to take different points of view. [Dr. Fritz Todt] V I C T O R Y N O T V E N G E A N C E To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: How to disable software TCP checksumming?
[EMAIL PROTECTED] wrote: > > This thread is baffling. The bottom line is that you cant > trust data coming into your machine, and you have to > checksum it. The link level check only verifies that what > was sent by the last forwarding point is the same as what > you got, but in NO WAY implies that all of the data is > valid. A link level checksum pass is a necessary, but not > a sufficient condition for the data being acceptable. There > are scads of reasons that it could be bad. Disabling > checksumming is a kludge that you may chose to do, but its > never the right thing to do. Tell me: at what point are you willing to trust the data? Is "the card plus the motherboard" my machine, or is it just "the motherboard"? The cards support offloading the checksum to hardware; you are saying that it is never reasonable to do this. Fine. That basically means that we need to checksum the contents of memory once they have been loaded into the L2 cache, to make sure they arrived safely from memory. Wait... now we need to checksum them in L1 cache, once they have arrived from L2. Wait... now we need to checksum them in the pipeline, as they have arrived from our untrustworthy L1 cache... At some point you have to draw the boundary between the "inside" and the "outside". I am happy to draw this boundary at the etherenet cable; why are you insisting that I draw it at the PCI bus instead? Does your requirement apply to devices integrated on the motherboard, or is your concern solely with the mechanical PCI connectors between the boards and the motherboard? -- Terry To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: How to disable software TCP checksumming?
Assar Westerlund wrote: > > Terry Lambert <[EMAIL PROTECTED]> writes: > > This all came from IP headers being 14 bytes long, instead > > of 16. > > Hu? An IPv4 header (not including options) is 20 bytes long. Sorry; ethernet header. The problem is the 14 bytes making the code unaligned on a 32 or 64 bit boundary... -- Terry To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: How to disable software TCP checksumming?
In article [EMAIL PROTECTED]> you write: >This thread is baffling. The bottom line is that you cant trust data coming >into your machine, and you have to checksum it. The link level check only >verifies that what was sent by the last forwarding point is the same as what >you got, but in NO WAY implies that all of the data is valid. A link level >checksum pass is a necessary, but not a sufficient condition for the data >being acceptable. There are scads of reasons that it could be bad. Disabling >checksumming is a kludge that you may chose to do, but its never the right >thing to do. I think there's some confusion here. There are two types of checksums: 1. CRC link-level checksum on the ethernet packet. 2. Upper layer checksums (TCP/IP) All hardware that I'm aware of automatically performs #1, I'm not sure if there is anything that will permit disabling this. Step 2 is usually known as the "end-to-end" checksum, and is traditionally performed by the host CPU. Depending on your view of what constitutes an "endpoint", these checksums may be performed by the network adapter itself. I believe that this is what the original meant by disabling software checksums, in the sense that his hardware would do this work. Dispensing with TCP/IP checksum altogether is a fairly bad idea, but I don't think that is what was being proposed here. -- Jonathan To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: How to disable software TCP checksumming?
This thread is baffling. The bottom line is that you cant trust data coming into your machine, and you have to checksum it. The link level check only verifies that what was sent by the last forwarding point is the same as what you got, but in NO WAY implies that all of the data is valid. A link level checksum pass is a necessary, but not a sufficient condition for the data being acceptable. There are scads of reasons that it could be bad. Disabling checksumming is a kludge that you may chose to do, but its never the right thing to do. bryan To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: How to disable software TCP checksumming?
Assar Westerlund wrote: > > Ruslan Ermilov <[EMAIL PROTECTED]> writes: > > On Fri, Jun 08, 2001 at 02:24:23PM +0200, Assar Westerlund wrote: > > > Terry Lambert <[EMAIL PROTECTED]> writes: > > > > This all came from IP headers being 14 bytes long, instead > > > > of 16. > > > > > > Hu? An IPv4 header (not including options) is 20 bytes long. > > > > > Terry meant the length of an IPv4 header up to but not including > > the checksum field. no he meant to say "ethernet header" -- __--_|\ Julian Elischer / \ [EMAIL PROTECTED] ( OZ) World tour 2000-2001 ---> X_.---._/ v To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: How to disable software TCP checksumming?
Ruslan Ermilov <[EMAIL PROTECTED]> writes: > On Fri, Jun 08, 2001 at 02:24:23PM +0200, Assar Westerlund wrote: > > Terry Lambert <[EMAIL PROTECTED]> writes: > > > This all came from IP headers being 14 bytes long, instead > > > of 16. > > > > Hu? An IPv4 header (not including options) is 20 bytes long. > > > Terry meant the length of an IPv4 header up to but not including > the checksum field. I'm confused. From rfc791: 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |Version| IHL |Type of Service| Total Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Identification|Flags| Fragment Offset| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Time to Live |Protocol | Header Checksum | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Source Address | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |Destination Address| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |Options|Padding| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Which to me says that an IP header up to but not including the checksum is 10 bytes? /assar To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: How to disable software TCP checksumming?
On Fri, Jun 08, 2001 at 02:24:23PM +0200, Assar Westerlund wrote: > Terry Lambert <[EMAIL PROTECTED]> writes: > > This all came from IP headers being 14 bytes long, instead > > of 16. > > Hu? An IPv4 header (not including options) is 20 bytes long. > Terry meant the length of an IPv4 header up to but not including the checksum field. Cheers, -- Ruslan Ermilov Oracle Developer/DBA, [EMAIL PROTECTED] Sunbay Software AG, [EMAIL PROTECTED] FreeBSD committer, +380.652.512.251Simferopol, Ukraine http://www.FreeBSD.org The Power To Serve http://www.oracle.com Enabling The Information Age To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: How to disable software TCP checksumming?
Terry Lambert <[EMAIL PROTECTED]> writes: > This all came from IP headers being 14 bytes long, instead > of 16. Hu? An IPv4 header (not including options) is 20 bytes long. /assar To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: How to disable software TCP checksumming?
Bill Fenner wrote: > > >The TCP checksum protects more than just the contents of the packet > >on the wire; it's also a (somewhat) weak check on the contents > >of your packet sitting in memory, and as it's going over the bus > >in your computer between memory and peripherals and for other > >end-to-end sorts of issues. > > In fact, Jonathan Stone at Stanford did some measurements of > checksum errors for his thesis, and found that on a given link > in the Stanford residences there were a surprising (small, but > measurable) number of packets with IP header checksum errors > with what appeared to be DMA errors -- e.g. 4 bytes missing > from the middle of the header. When the end to end checksum > is really memory to memory, it can catch errors like this... You need to read the ethernet drivers in /sys/pci/if_*... Specifically, look for m_devget() uses, and the comments about why they are necessary. I will bet you that the 4 byte "holes" that appear to be "DMA errors" are actually driver errors, with drivers sending unaligned packets down to the the DMA engine. Further, if you characterize them, I bet you will see that the majority of them are Linux machines. The problem is that many cards require 4 byte alignment (one of the gigabit cards requires 64 bit alignment) for the DMA engine on the card to function properly. This annoys the piss out of TCP/IP stacks in host machines, which like to cast headers to pointers, and directly access the packet header contents. Naieve driver writers will do a 2 byte align on their assembled TCP/IP packets (causing the IP address to be on a 4 + 2 byte boundary, instead of a 4 byte boundary, and the DMA engine will choke. This all came from IP headers being 14 bytes long, instead of 16. Why card manufacturers can't make their DMA engine use an offset shif buffer is beyond me, but they don't do that, and so unaligned accesses happen. This slows down host processing, best case; worst case, the host is not careful to make the cards happy, and "holes" appear. In fact, it would not surprise me if the person conducting the research had the problem on his system, and the stuff on the network itself was actually all right, since it is generally a receive processing introduced error. 8-). In any case, I still would like an option to turn on the processing in the card, and let me assume that my hardware is monolithic, my card supports checksum offloading, and if I have problems with corruption between my ethernet card and my CPU, I have really, reall bad problems because the CPU and the etherent interface are sitting on the same silicon die which they were fabricated to share. -- Terry To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: How to disable software TCP checksumming?
This thread is baffling. The bottom line is that you cant trust data coming into your machine, and you have to checksum it. The link level check only verifies that what was sent by the last forwarding point is the same as what you got, but in NO WAY implies that all of the data is valid. A link level checksum pass is a necessary, but not a sufficient condition for the data being acceptable. There are scads of reasons that it could be bad. Disabling checksumming is a kludge that you may chose to do, but its never the right thing to do. bryan To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: How to disable software TCP checksumming?
>The TCP checksum protects more than just the contents of the packet >on the wire; it's also a (somewhat) weak check on the contents >of your packet sitting in memory, and as it's going over the bus >in your computer between memory and peripherals and for other end-to-end >sorts of issues. In fact, Jonathan Stone at Stanford did some measurements of checksum errors for his thesis, and found that on a given link in the Stanford residences there were a surprising (small, but measurable) number of packets with IP header checksum errors with what appeared to be DMA errors -- e.g. 4 bytes missing from the middle of the header. When the end to end checksum is really memory to memory, it can catch errors like this... Bill To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: How to disable software TCP checksumming?
As the originator of this thread (sorry), I thought I'd clear up some of the speculation as to what I'm trying to do. :-) Our adapter card is a GSN adapter, focused on the ST protocol over a hippi-6400 link. We support IP on this adapter as well but that's not its expected primary use. In asking the original question, I was trying to find out how to disable the IP/TCP software checksums and rely on the link layer LCRC and ECRC (I think that's what they're called) checks. The LCRC is a hop to hop CRC that is applied to each 32 byte micropacket that is transmitted. The ECRC is an overll message CRC that covers all micropackets of the message. It is my expectation that these two CRCs will be adaquate to catch any errors that occur within the GSN network. Note that our card has no IP specific checksum support (it does for ST) so for packets originating from, or destined to, outside of the GSN network we _must_ do the software checksuming. I understand this. My purpose in being able to turn software checksums off was strictly to measure the performance advantage of doing so. In any real network environment the software checksums are required with this card. Thanks, Bob On Wed, Jun 06, 2001 at 05:15:40PM -0300, Daniel C. Sobral wrote: > Louis A. Mamakos wrote: > > > > > If I'm not mistaken the message that started this thread inquired > > about adding an option to prevent TCP from doing a checksum, since > > the fancy gigabit hardware performed reliable link-level error detection > > > I do not recall he ever mentioning "link-level". Either my mind removed > it, or yours inserted it. :-) > > > > itself. I argue that since TCP is an end-to-end transport protocol that > > individual error detection on a per-hop basis is not sufficient either > > theoretically or practically. My last message illustrated a number of > > cases where the transport of the packet over a link was reliably > > done, but the contents of the packet were corrupted by malfunctioning > > software or hardware which the end-to-end TCP checksum detected. > > > > I have less of an issue with the endpoints of the TCP connection > > offloading checksum computation to the NIC card, though you're > > still exposed to a certain class of error, like the PR I referenced. > > The problem is what happend to your data in intermediate network > > elements (routers, etc.) between the endpoints of the TCP connection. > > > > Louis Mamakos > > > > > > To Unsubscribe: send mail to [EMAIL PROTECTED] > > with "unsubscribe freebsd-hackers" in the body of the message > > > > > -- > Daniel C. Sobral (8-DCS) > [EMAIL PROTECTED] > [EMAIL PROTECTED] > [EMAIL PROTECTED] > [EMAIL PROTECTED] > > ... if the church put in half the time on covetousness that it does > on lust, this would be a better world. > -- Garrison Keillor, "Lake Wobegon Days" -- Bob Willcox All men profess honesty as long as they can. [EMAIL PROTECTED]To believe all men honest would be folly. Austin, TX To believe none so is something worse. -- John Quincy Adams To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: How to disable software TCP checksumming?
> "Louis A. Mamakos" wrote: > > > > > > > > It seems to me to be kind of moot to check the same value twice, unless > > > you suspect hardware problems. Aren't you talking about two different > > > checks over the same data instead of checksum off-loading? > > > > Suspect hardware problem? Of course you should! That's why memory > > systems have parity or ECC, and I/O buses are similarlly protected. At > > least on real computers. > > > > The link-level CRC only protects the data as it goes over the link > ^^ > > After reading the rest of his messages, I'm not so sure, but I would > think he was talking about _transport_ level check sum, and verifying > that with hardward (NIC) instead of software (IP stack). If I'm not mistaken the message that started this thread inquired about adding an option to prevent TCP from doing a checksum, since the fancy gigabit hardware performed reliable link-level error detection itself. I argue that since TCP is an end-to-end transport protocol that individual error detection on a per-hop basis is not sufficient either theoretically or practically. My last message illustrated a number of cases where the transport of the packet over a link was reliably done, but the contents of the packet were corrupted by malfunctioning software or hardware which the end-to-end TCP checksum detected. I have less of an issue with the endpoints of the TCP connection offloading checksum computation to the NIC card, though you're still exposed to a certain class of error, like the PR I referenced. The problem is what happend to your data in intermediate network elements (routers, etc.) between the endpoints of the TCP connection. Louis Mamakos To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: How to disable software TCP checksumming?
"Louis A. Mamakos" wrote: > > > > > It seems to me to be kind of moot to check the same value twice, unless > > you suspect hardware problems. Aren't you talking about two different > > checks over the same data instead of checksum off-loading? > > Suspect hardware problem? Of course you should! That's why memory > systems have parity or ECC, and I/O buses are similarlly protected. At > least on real computers. > > The link-level CRC only protects the data as it goes over the link ^^ After reading the rest of his messages, I'm not so sure, but I would think he was talking about _transport_ level check sum, and verifying that with hardward (NIC) instead of software (IP stack). > between the link-level hardware which generates the CRC and the box > on the other end that receives it. It does not protect the data > end-to-end, so if it's gets corrupted whilst sitting in memory in > an intermediate node, you won't detect that. > > Why would it get corrupted? > > 1. Software. Random unrelated other software does a random store in > the buffer containing your packet sitting in memory. Fancy device drivers > that do scatter-gather I/O gets it wrong every now and then and points > off into space. mbuf cluster which should be treated as read-only isn't, > and some other code hoses it. (See PR kern/27782 at > http://www.FreeBSD.org/cgi/query-pr.cgi?pr=27782 for an example of this.) > > 2. Hardware. I found broken UNIBUS hardware 15 or 20 years ago this > way. I had some broken hardware which took frame relay frames on > a HSSI interface and turned them into AAL5 ATM cells that would get > the cell reassembly wrong if you pushed too hard. Or just plain > broken memory that results in packet corruption. > > I've personally experienced all these problems. Maybe I'm just > unluckly. One common thread of my experiences is being close to the > bleeding edge of technology where stuff isn't as mature as many people > are used to. > > This end-to-end issue is not a theoretical consideration. While the > 1's complement internet checksum isn't that strong, it does detect a > bunch of bug-like behavior like this. > > Louis Mamakos -- Daniel C. Sobral(8-DCS) [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED] wow regex humor... I'm a geek To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: How to disable software TCP checksumming?
In a message dated 06/05/2001 10:25:09 AM Eastern Daylight Time, [EMAIL PROTECTED] writes: > Suspect hardware problem? Of course you should! That's why memory > systems have parity or ECC, and I/O buses are similarlly protected. At > least on real computers. Your view of the world is a bit misguided. I know of a situation a few years back where cascade switches were doing ATM to Frame Relay conversion and sending out mal-formed packets due to a bug in their reassembly procedure. Cisco routers (which pass data without checksuming and dont checksum pings) never saw a problem, but our unix boxes were complaining regularly. A similar situation occurred with cheap ethernet bridges that a service provider was using to colocate to a building across the street to uunet truncated packets with volumes over 2mb/s. Checksumming is there for a reason. You premise that "just because there are no physical errors, the data must be good" is simply defective. Bryan To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: How to disable software TCP checksumming?
> Giorgos Keramidas wrote: > > > > There are good reasons why checksumming in upper layers should not be disabled > > even if some lower layer does checksumming of its own. I recall reading some > > good points on this one at "TCP/IP Illustrated, Volume I" from (now late) > > Richard W. Stevens. > > It seems to me to be kind of moot to check the same value twice, unless > you suspect hardware problems. Aren't you talking about two different > checks over the same data instead of checksum off-loading? Suspect hardware problem? Of course you should! That's why memory systems have parity or ECC, and I/O buses are similarlly protected. At least on real computers. The link-level CRC only protects the data as it goes over the link between the link-level hardware which generates the CRC and the box on the other end that receives it. It does not protect the data end-to-end, so if it's gets corrupted whilst sitting in memory in an intermediate node, you won't detect that. Why would it get corrupted? 1. Software. Random unrelated other software does a random store in the buffer containing your packet sitting in memory. Fancy device drivers that do scatter-gather I/O gets it wrong every now and then and points off into space. mbuf cluster which should be treated as read-only isn't, and some other code hoses it. (See PR kern/27782 at http://www.FreeBSD.org/cgi/query-pr.cgi?pr=27782 for an example of this.) 2. Hardware. I found broken UNIBUS hardware 15 or 20 years ago this way. I had some broken hardware which took frame relay frames on a HSSI interface and turned them into AAL5 ATM cells that would get the cell reassembly wrong if you pushed too hard. Or just plain broken memory that results in packet corruption. I've personally experienced all these problems. Maybe I'm just unluckly. One common thread of my experiences is being close to the bleeding edge of technology where stuff isn't as mature as many people are used to. This end-to-end issue is not a theoretical consideration. While the 1's complement internet checksum isn't that strong, it does detect a bunch of bug-like behavior like this. Louis Mamakos To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: How to disable software TCP checksumming?
Giorgos Keramidas wrote: > > There are good reasons why checksumming in upper layers should not be disabled > even if some lower layer does checksumming of its own. I recall reading some > good points on this one at "TCP/IP Illustrated, Volume I" from (now late) > Richard W. Stevens. It seems to me to be kind of moot to check the same value twice, unless you suspect hardware problems. Aren't you talking about two different checks over the same data instead of checksum off-loading? -- Daniel C. Sobral(8-DCS) [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED] wow regex humor... I'm a geek To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: How to disable software TCP checksumming?
On Wed, May 30, 2001 at 10:04:08PM -0400, Louis A. Mamakos wrote: > The TCP checksum protects more than just the contents of the packet > on the wire; it's also a (somewhat) weak check on the contents > of your packet sitting in memory, and as it's going over the bus > in your computer between memory and peripherals and for other end-to-end > sorts of issues. Given that we're talking about this for >=1Gbps devices where all devices do hardware checksuming that argument holds little water. No one is proposing turing off checksums for everything, but instead for a set of specalized, tightly coupled applications. I certaintly wouldn't turn it off for packets traversing the internet. -- Brooks -- Any statement of the form "X is the one, true Y" is FALSE. PGP fingerprint 655D 519C 26A7 82E7 2529 9BF0 5D8E 8BE9 F238 1AD4 PGP signature
Re: How to disable software TCP checksumming?
The TCP checksum protects more than just the contents of the packet on the wire; it's also a (somewhat) weak check on the contents of your packet sitting in memory, and as it's going over the bus in your computer between memory and peripherals and for other end-to-end sorts of issues. Of course, with rude things like NAT boxes and other network elements taking liberties with your data, this becomes less and less useful as time goes on, sigh. louie To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: How to disable software TCP checksumming?
On Tue, May 29, 2001 at 11:52:15PM -0500, Bob Willcox wrote: > This is probably all well and good, but our adapter is a 10 Gb/s link > and includes hardware CRC (actually two forms of this, LCRC on a per > micropacket [32 byte] basis and ECRC over the entire message). Right > now our goal is to see how fast we can get it to run on PC hardware > on FreeBSD with IP. We don't really expect IP to allow us to reach > it's theoretical limit of about 800 MB/s, but the closer we can get the > better. :-) (ST and a system with faster main memory and dual PCI-X > busses should allow us to get much closer.) I was just thinking I wanted this feature for a cluster. I was thinking the knob should be three state. Default, on for all packets. Next, off for localy connected networks and finally, off for all packets. For what it's worth, I agree it's just a waste of CPU to checksum TCP packets that will only travel over 1Gb or 10Gb Ethernet, or similar links. This 10Gb Ethernet presentation talkes about mean time between false packet acceptance being around 60 billion years: http://www.best.com/~walker/pdfs.talks/albuquerque.pdf -- Brooks -- Any statement of the form "X is the one, true Y" is FALSE. PGP fingerprint 655D 519C 26A7 82E7 2529 9BF0 5D8E 8BE9 F238 1AD4 PGP signature
Re: How to disable software TCP checksumming?
On Wed, May 30, 2001 at 04:52:00AM +0300, Giorgos Keramidas wrote: > On Tue, May 29, 2001 at 03:52:12PM -0500, Bob Willcox wrote: > | > | and may prevent the upper layers from checking the CRC on input. > > There are good reasons why checksumming in upper layers should not be disabled > even if some lower layer does checksumming of its own. I recall reading some > good points on this one at "TCP/IP Illustrated, Volume I" from (now late) > Richard W. Stevens. > > In fact, this is exactly what the sysctl knob net.inet.udp.checksum is there > for. The knob having a default value of 1 (checksumming enabled), means that > checksumming is used. > > If you do change checksumming in TCP, please provide a similar knob for that > protocol, and have it set to a default of 1 (still enabled). This way if > someone feels comfortable to have checksums disabled in TCP, or some higher > layer, they can still do it, but the default behavior is the good ole' FreeBSD > we all know and love. This is probably all well and good, but our adapter is a 10 Gb/s link and includes hardware CRC (actually two forms of this, LCRC on a per micropacket [32 byte] basis and ECRC over the entire message). Right now our goal is to see how fast we can get it to run on PC hardware on FreeBSD with IP. We don't really expect IP to allow us to reach it's theoretical limit of about 800 MB/s, but the closer we can get the better. :-) (ST and a system with faster main memory and dual PCI-X busses should allow us to get much closer.) Bob > > --giorgos -- Bob Willcox Egotist, n.: [EMAIL PROTECTED] A person of low taste, more interested in himself than me. Austin, TX-- Ambrose Bierce, "The Devil's Dictionary" To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: How to disable software TCP checksumming?
On Tue, May 29, 2001 at 03:52:12PM -0500, Bob Willcox wrote: | | and may prevent the upper layers from checking the CRC on input. There are good reasons why checksumming in upper layers should not be disabled even if some lower layer does checksumming of its own. I recall reading some good points on this one at "TCP/IP Illustrated, Volume I" from (now late) Richard W. Stevens. In fact, this is exactly what the sysctl knob net.inet.udp.checksum is there for. The knob having a default value of 1 (checksumming enabled), means that checksumming is used. If you do change checksumming in TCP, please provide a similar knob for that protocol, and have it set to a default of 1 (still enabled). This way if someone feels comfortable to have checksums disabled in TCP, or some higher layer, they can still do it, but the default behavior is the good ole' FreeBSD we all know and love. --giorgos To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: How to disable software TCP checksumming?
Thanks Drew! This helps alot! Bob On Tue, May 29, 2001 at 05:00:43PM -0400, Andrew Gallatin wrote: > > > Jesper Skriver writes: > > On Tue, May 29, 2001 at 02:41:14PM -0500, Bob Willcox wrote: > > > Hi, > > > > > > I am working on a device driver for a GSN adapter that has hardware CRC > > > checking and need to know if there is a way to disable the software CRC > > > checking for TCP? This is on a FreeBSD 4.2-stable system. > > > > Eegads. I think the original poster wanted to be able to use the > hardware CRC features of his nic, not ignore checksums altogther. > > Bob -- Take a look at the /sys/pci/if_ti.c driver for an example of > how to use hardware checksum assist. > > On the recieve side, you want to set the m_pkthdr.csum_flags > appropriately (depending on what your device can do) on each recieve, > as well as fill in the actual checksum in m_pkthdr.csum_data. > > On the send side, you need to specify what your device is capable of > assisting with in the if_hwassist field of your driver's ifp struct. > Packets will come down w/o those fields filled in. The stack will > expect your device to calculate those fields in hardware. > > I beleive these features appeared around 4.1, so if this is a 3rd > party driver, you may want to check __FreeBSD_version >= 41. > > > Hope this helps, > > Drew > > -- > Andrew Gallatin, Sr Systems Programmerhttp://www.cs.duke.edu/~gallatin > Duke University Email: [EMAIL PROTECTED] > Department of Computer SciencePhone: (919) 660-6590 -- Bob Willcox Egotist, n.: [EMAIL PROTECTED] A person of low taste, more interested in himself than me. Austin, TX-- Ambrose Bierce, "The Devil's Dictionary" To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: How to disable software TCP checksumming?
Jesper Skriver writes: > On Tue, May 29, 2001 at 02:41:14PM -0500, Bob Willcox wrote: > > Hi, > > > > I am working on a device driver for a GSN adapter that has hardware CRC > > checking and need to know if there is a way to disable the software CRC > > checking for TCP? This is on a FreeBSD 4.2-stable system. > Eegads. I think the original poster wanted to be able to use the hardware CRC features of his nic, not ignore checksums altogther. Bob -- Take a look at the /sys/pci/if_ti.c driver for an example of how to use hardware checksum assist. On the recieve side, you want to set the m_pkthdr.csum_flags appropriately (depending on what your device can do) on each recieve, as well as fill in the actual checksum in m_pkthdr.csum_data. On the send side, you need to specify what your device is capable of assisting with in the if_hwassist field of your driver's ifp struct. Packets will come down w/o those fields filled in. The stack will expect your device to calculate those fields in hardware. I beleive these features appeared around 4.1, so if this is a 3rd party driver, you may want to check __FreeBSD_version >= 41. Hope this helps, Drew -- Andrew Gallatin, Sr Systems Programmer http://www.cs.duke.edu/~gallatin Duke University Email: [EMAIL PROTECTED] Department of Computer Science Phone: (919) 660-6590 To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: How to disable software TCP checksumming?
Thanks for the info, but I don't think this will do what I want. I
have been looking at the Tigon driver which seems to support hardware
assisted CRC checking. I'm could be way off base here, but this snippit
of code from if_ti.c:
if (ifp->if_hwassist) {
m->m_pkthdr.csum_flags |= CSUM_IP_CHECKED |
CSUM_DATA_VALID;
if ((cur_rx->ti_ip_cksum ^ 0x) == 0)
m->m_pkthdr.csum_flags |= CSUM_IP_VALID;
m->m_pkthdr.csum_data = cur_rx->ti_tcp_udp_cksum;
}
looks like it might be related to what I want to do, and may prevent
the upper layers from checking the CRC on input. I also would like to
disable the CRC generation for output but, so far, can't seem to find
how that can be done. Note that I'm not above changing the kernel if
that's the only way, but would like to find some way in my driver to
turn off the checksums if possible.
Thanks,
Bob
On Tue, May 29, 2001 at 10:11:07PM +0200, Jesper Skriver wrote:
> On Tue, May 29, 2001 at 02:41:14PM -0500, Bob Willcox wrote:
> > Hi,
> >
> > I am working on a device driver for a GSN adapter that has hardware CRC
> > checking and need to know if there is a way to disable the software CRC
> > checking for TCP? This is on a FreeBSD 4.2-stable system.
>
> Not without modifying the source, the below should do it from a
> quick look.
>
> Index: src/sys/netinet/tcp_input.c
> ===
> RCS file: /home/ncvs/src/sys/netinet/tcp_input.c,v
> retrieving revision 1.130
> diff -u -r1.130 tcp_input.c
> --- src/sys/netinet/tcp_input.c 2001/05/29 19:54:45 1.130
> +++ src/sys/netinet/tcp_input.c 2001/05/29 20:07:22
> @@ -401,6 +401,7 @@
> th = (struct tcphdr *)((caddr_t)ip + off0);
> tlen = ip->ip_len;
>
> +#if 0
> if (m->m_pkthdr.csum_flags & CSUM_DATA_VALID) {
> if (m->m_pkthdr.csum_flags & CSUM_PSEUDO_HDR)
> th->th_sum = m->m_pkthdr.csum_data;
> @@ -423,6 +424,7 @@
> tcpstat.tcps_rcvbadsum++;
> goto drop;
> }
> +#endif
> #ifdef INET6
> /* Re-initialization for later version check */
> ip->ip_v = IPVERSION;
>
> /Jesper
>
> --
> Jesper Skriver, jesper(at)skriver(dot)dk - CCIE #5456
> Work:Network manager @ AS3292 (Tele Danmark DataNetworks)
> Private: FreeBSD committer @ AS2109 (A much smaller network ;-)
>
> One Unix to rule them all, One Resolver to find them,
> One IP to bring them all and in the zone to bind them.
--
Bob Willcox Egotist, n.:
[EMAIL PROTECTED] A person of low taste, more interested in himself than me.
Austin, TX-- Ambrose Bierce, "The Devil's Dictionary"
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message
Re: How to disable software TCP checksumming?
On Tue, May 29, 2001 at 02:41:14PM -0500, Bob Willcox wrote:
> Hi,
>
> I am working on a device driver for a GSN adapter that has hardware CRC
> checking and need to know if there is a way to disable the software CRC
> checking for TCP? This is on a FreeBSD 4.2-stable system.
Not without modifying the source, the below should do it from a
quick look.
Index: src/sys/netinet/tcp_input.c
===
RCS file: /home/ncvs/src/sys/netinet/tcp_input.c,v
retrieving revision 1.130
diff -u -r1.130 tcp_input.c
--- src/sys/netinet/tcp_input.c 2001/05/29 19:54:45 1.130
+++ src/sys/netinet/tcp_input.c 2001/05/29 20:07:22
@@ -401,6 +401,7 @@
th = (struct tcphdr *)((caddr_t)ip + off0);
tlen = ip->ip_len;
+#if 0
if (m->m_pkthdr.csum_flags & CSUM_DATA_VALID) {
if (m->m_pkthdr.csum_flags & CSUM_PSEUDO_HDR)
th->th_sum = m->m_pkthdr.csum_data;
@@ -423,6 +424,7 @@
tcpstat.tcps_rcvbadsum++;
goto drop;
}
+#endif
#ifdef INET6
/* Re-initialization for later version check */
ip->ip_v = IPVERSION;
/Jesper
--
Jesper Skriver, jesper(at)skriver(dot)dk - CCIE #5456
Work:Network manager @ AS3292 (Tele Danmark DataNetworks)
Private: FreeBSD committer @ AS2109 (A much smaller network ;-)
One Unix to rule them all, One Resolver to find them,
One IP to bring them all and in the zone to bind them.
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message
