Re: Possible problem with zero-copy TCP and sendfile()

2001-04-17 Thread Jan Kasprzak
Wolfgang Rohdewald wrote: : On Tuesday 17 April 2001 22:36, Jan Kasprzak wrote: : > +    if (len == -1 || len > 0 && len < count) { : : are you sure there are no missing () ? : : if ((len == -1) || (len > 0) && (len < count)) { : : assumig that && has precedence over || (I believe so)

Re: Possible problem with zero-copy TCP and sendfile()

2001-04-17 Thread Martin Josefsson
On Tue, 17 Apr 2001, Wolfgang Rohdewald wrote: > On Tuesday 17 April 2001 22:36, Jan Kasprzak wrote: > > +    if (len == -1 || len > 0 && len < count) { > > are you sure there are no missing () ? > > if ((len == -1) || (len > 0) && (len < count)) { > > assumig that && has precedence over || (I

Re: Possible problem with zero-copy TCP and sendfile()

2001-04-17 Thread Wolfgang Rohdewald
On Tuesday 17 April 2001 22:36, Jan Kasprzak wrote: > +    if (len == -1 || len > 0 && len < count) { are you sure there are no missing () ? if ((len == -1) || (len > 0) && (len < count)) { assumig that && has precedence over || (I believe so) - To unsubscribe from this list: send the line "uns

Re: Possible problem with zero-copy TCP and sendfile()

2001-04-17 Thread David S. Miller
Jesse S Sipprell writes: > On error, -1 is returned in the usual fashion and offset is purported to be > updated to point to the next byte following the last one sent. > > Will the zerocopy patches break this? No, they should not. Later, David S. Miller [EMAIL PROTECTED] - To unsubscribe f

Re: Possible problem with zero-copy TCP and sendfile()

2001-04-17 Thread Jesse S Sipprell
On Tue, Apr 17, 2001 at 01:23:07PM -0700, David S. Miller wrote: > One more subtle note, for the case of error handling. There is a > change to sendfile() in the zerocopy patches which causes sendfile() > to act more like sendmsg() when errors occur. How is this likely to affect applications?

Re: Possible problem with zero-copy TCP and sendfile()

2001-04-17 Thread Jan Kasprzak
Jesse S Sipprell wrote: : After cursory examination of proftpd, it appears that there is a misuse of the : sendfile() call under Linux, which may be responsible for the corruption. The : code was originally based on BSD semantics. Under Linux, the offset argument : is not being used correctly to

Re: Possible problem with zero-copy TCP and sendfile()

2001-04-17 Thread David S. Miller
Jesse S Sipprell writes: > A patch will be coming out soon, as it is a fairly trivial fix. Thank you for tracking this down. One more subtle note, for the case of error handling. There is a change to sendfile() in the zerocopy patches which causes sendfile() to act more like sendmsg() when er

Re: Possible problem with zero-copy TCP and sendfile()

2001-04-17 Thread Jesse S Sipprell
On Tue, Apr 17, 2001 at 06:15:24PM +0200, Jan Kasprzak wrote: > Alan Cox wrote: > : > : but once a fixed BIOS is out for your board that would be a good first step. > : > : If it still does it then, its worth digging for kernel naughties > : > : > : > I don't think I have 686b southbridge. I ha

Re: Possible problem with zero-copy TCP and sendfile()

2001-04-17 Thread Pekka Pietikainen
On Tue, Apr 17, 2001 at 06:15:24PM +0200, Jan Kasprzak wrote: > Some more progress: I now downgraded to proftpd without sendfile(). > The CPU usage is now nearly 100% (with ~170 FTP users; with sendfile() > it was under 50% with >320 FTP users). But nevertheless, the downloaded > images now

Re: Possible problem with zero-copy TCP and sendfile()

2001-04-17 Thread Jan Kasprzak
Jan Kasprzak wrote: : $ cmp -cl seawolf-sendfile.iso seawolf-i386-SRPMS.iso [...] : : Which simply means, that at 160628609 it started to send : the CD image from the beginning. Well, I did strace of proftpd, and it _may_ be a mis-interpretation of the sendfile(2) semantics on the

Re: Possible problem with zero-copy TCP and sendfile()

2001-04-17 Thread Jan Kasprzak
Andi Kleen wrote: : I guess to debug this problem it would be useful to get some idea about the : nature of the corruption. Could you enable sendfile() again, and when a : user complains ask to download it again and provide a : cmp -cl fileA fileB | head -500 listing of their differences?

Re: Possible problem with zero-copy TCP and sendfile()

2001-04-17 Thread Jan Kasprzak
Alan Cox wrote: : > : but once a fixed BIOS is out for your board that would be a good first step. : > : If it still does it then, its worth digging for kernel naughties : > : : > I don't think I have 686b southbridge. I have 686 (without "b"): : : Ok. What revision of 3c90x card do you have

Re: Possible problem with zero-copy TCP and sendfile()

2001-04-17 Thread Alan Cox
> : but once a fixed BIOS is out for your board that would be a good first step. > : If it still does it then, its worth digging for kernel naughties > : > I don't think I have 686b southbridge. I have 686 (without "b"): Ok. What revision of 3c90x card do you have ? - To unsubscribe from

Re: Possible problem with zero-copy TCP and sendfile()

2001-04-17 Thread Jan Kasprzak
Andi Kleen wrote: : On Tue, Apr 17, 2001 at 03:10:07PM +0200, Jan Kasprzak wrote: : > 00:0c.0 Ethernet controller: 3Com Corporation 3c905C-TX [Fast Etherlink] (rev 74) : : IIRC the problem came up earlier. Some versions of 3com NICs seem to make : problems with the hardware checksum. There were s

Re: Possible problem with zero-copy TCP and sendfile()

2001-04-17 Thread Jan Kasprzak
Alan Cox wrote: : > The long story: My server is Athlon 850 on ASUS A7V, 256M RAM. : > Seven IDE discs, one SCSI disc. The controllers and NIC are as follows : > (output of lspci): : : See the VIA chipset report on www.theregister.co.uk about corruption problems : with VIA chipsets. The cases

Re: Possible problem with zero-copy TCP and sendfile()

2001-04-17 Thread Alan Cox
> The long story: My server is Athlon 850 on ASUS A7V, 256M RAM. > Seven IDE discs, one SCSI disc. The controllers and NIC are as follows > (output of lspci): See the VIA chipset report on www.theregister.co.uk about corruption problems with VIA chipsets. The cases seen on Linux included sh

Re: Possible problem with zero-copy TCP and sendfile()

2001-04-17 Thread Andi Kleen
On Tue, Apr 17, 2001 at 03:10:07PM +0200, Jan Kasprzak wrote: > 00:0c.0 Ethernet controller: 3Com Corporation 3c905C-TX [Fast Etherlink] (rev 74) IIRC the problem came up earlier. Some versions of 3com NICs seem to make problems with the hardware checksum. There were some fixes in the driver la

Possible problem with zero-copy TCP and sendfile()

2001-04-17 Thread Jan Kasprzak
Hello, I have discovered a possible problem on my host. The short story is: When downloading ISO images from this host (which runs 2.4.3 + zerocopy and ProFTPd with sendfile()), the image is sometimes corrupted (MD5 checksum of the downloaded file does not match). The lon

Re: zero-copy TCP

2000-09-12 Thread Jamie Lokier
Todd wrote: > While I agree with what's going on right now, the recent purchase of > Alteon by Nortel (primarily for their switch line, not for the > NICs) leaves quite a bit of doubt in my mind about the future of the card > and the openness of the firmware in particular. Why not raise your conc

Re: zero-copy TCP

2000-09-12 Thread Jes Sorensen
> "Todd" == Todd <[EMAIL PROTECTED]> writes: >> > Jes Sorensen wrote: > > It took me a little while in the >> beginning to convince Alteon to open > > up and provide docs, but >> since they saw the light they have been > > extremely helpful and >> went much further in their openness than I h

Re: zero-copy TCP

2000-09-12 Thread Todd
> > Jes Sorensen wrote: > > > It took me a little while in the beginning to convince Alteon to open > > > up and provide docs, but since they saw the light they have been > > > extremely helpful and went much further in their openness than I had > > > ever expected or dared to hope for. > > > > A

Re: zero-copy TCP

2000-09-12 Thread Alan Cox
> Jes Sorensen wrote: > > It took me a little while in the beginning to convince Alteon to open > > up and provide docs, but since they saw the light they have been > > extremely helpful and went much further in their openness than I had > > ever expected or dared to hope for. > > And now it's re

Re: zero-copy TCP

2000-09-12 Thread Jamie Lokier
Jes Sorensen wrote: > It took me a little while in the beginning to convince Alteon to open > up and provide docs, but since they saw the light they have been > extremely helpful and went much further in their openness than I had > ever expected or dared to hope for. And now it's really showing i

Re: zero-copy TCP

2000-09-12 Thread Jes Sorensen
> "Jamie" == Jamie Lokier <[EMAIL PROTECTED]> writes: Jamie> According to group legend here (I wasn't around but will repeat Jamie> what I was told), we spent about 1 year trying to get docs on Jamie> Intel's i960 based gigabit card so we could program it. Jamie> Eventually we gave up and mov

Re: zero-copy TCP

2000-09-06 Thread Stephen Williams
Alan Cox wrote: > I've spent over 2 years trying to extract eepro100 server docs out of Intel > and failed. [EMAIL PROTECTED] said: > Sounds familiar :-) Very familiar. I have a whole development environment for the i960RP (we use the processor on boards we make) and by looking at pictures I wa

Re: zero-copy TCP

2000-09-06 Thread Jamie Lokier
Alan Cox wrote: > I've spent over 2 years trying to extract eepro100 server docs out of Intel > and failed. Sounds familiar :-) According to group legend here (I wasn't around but will repeat what I was told), we spent about 1 year trying to get docs on Intel's i960 based gigabit card so we coul

Re: zero-copy TCP

2000-09-05 Thread Alan Cox
> > True the i960 based one I didn't think of, however Intel never > > provided docs for it. > > ??? I find this surprising. Email [EMAIL PROTECTED] and ask him if > they will give them to you. I'm sure they would for Linux. I've spent over 2 years trying to extract eepro100 server docs out of

Re: zero-copy TCP

2000-09-05 Thread Jes Sorensen
> "Jeff" == Jeff V Merkey <[EMAIL PROTECTED]> writes: Jeff> IPX is a really good LAN protocol (but totally sucks for Jeff> internet). A full blown NCP server in-kernel that's toughtly Jeff> coupled to the page cache running over IPX would make flames Jeff> shoot out of the back of a Linux se

Re: zero-copy TCP

2000-09-05 Thread Jes Sorensen
> "Jeff" == Jeff V Merkey <[EMAIL PROTECTED]> writes: Jeff> Intel Nitro Card - i960 processor on the card. The SMP Jeff> debugging involved the use of a bus analyser since this card had Jeff> a piggish memory bus footprint (i960 processors do not have an Jeff> IO address space, so everything

Re: zero-copy TCP

2000-09-05 Thread Henning P . Schmiedehausen
On Tue, Sep 05, 2000 at 11:25:12AM -0700, Dan Hollis wrote: > On 5 Sep 2000, Henning P. Schmiedehausen wrote: > > [EMAIL PROTECTED] (Jeff V. Merkey) writes: > > >IPX is a really good LAN protocol (but totally sucks for internet). A > > Jeff, Netware is dead. Please leave it there. IP won. The num

Re: zero-copy TCP

2000-09-05 Thread Jes Sorensen
> "Jeff" == Jeff V Merkey <[EMAIL PROTECTED]> writes: Jeff> Jes Sorensen wrote: >> True the i960 based one I didn't think of, however Intel never >> provided docs for it. Jeff> ??? I find this surprising. Email [EMAIL PROTECTED] and ask Jeff> him if they will give them to you. I'm sure th

Re: zero-copy TCP

2000-09-05 Thread Jeff V. Merkey
Jes Sorensen wrote: > > > "Jeff" == Jeff V Merkey <[EMAIL PROTECTED]> writes: > > Jeff> Intel Nitro Card - i960 processor on the card. The SMP > Jeff> debugging involved the use of a bus analyser since this card had > Jeff> a piggish memory bus footprint (i960 processors do not have an >

Re: zero-copy TCP

2000-09-05 Thread Jeff V. Merkey
Dear Stu, I apologize for bothering you, but how possible would it be to let the Linux folks get access to the Nitro docs and docs for Intel ethernet cards in general. There seems to be interest from folks in getting this to improve Linux support for Intel products. Any help would be appreciat

Re: zero-copy TCP

2000-09-05 Thread Dan Hollis
On Tue, 5 Sep 2000, Henning P . Schmiedehausen wrote: > On Tue, Sep 05, 2000 at 11:25:12AM -0700, Dan Hollis wrote: > > I think you mean IPX is dead. Netware *could* work over TCP or UDP. > > IP is definitely king. Even micro$haft gave up on NetBEUI. > Yep, thats' what I meant. Sorry that I was no

Re: zero-copy TCP

2000-09-05 Thread Dan Hollis
On 5 Sep 2000, Henning P. Schmiedehausen wrote: > [EMAIL PROTECTED] (Jeff V. Merkey) writes: > >IPX is a really good LAN protocol (but totally sucks for internet). A > Jeff, Netware is dead. Please leave it there. IP won. The number of > new Netware Installations (as compared to existing or just

Re: zero-copy TCP

2000-09-05 Thread Jeff V. Merkey
Only Linux makes the lights flash with IPX RIP/SAP. NetWare uses NLSP routing and has since 1993 for IPX/SPX. I agree if someone is running NetWare 3 or NetWare 4.1 or earlier there's a lot of RIP/SAP traffic, but not the NLSP versions -- they do not use RIP/SAP but NLSP. Jeff Jes Sorensen wr

Re: zero-copy TCP

2000-09-05 Thread Jes Sorensen
> "Jeff" == Jeff V Merkey <[EMAIL PROTECTED]> writes: Jeff> Only Linux makes the lights flash with IPX RIP/SAP. NetWare Jeff> uses NLSP routing and has since 1993 for IPX/SPX. I agree if Jeff> someone is running NetWare 3 or NetWare 4.1 or earlier there's a Jeff> lot of RIP/SAP traffic, but

Re: zero-copy TCP

2000-09-05 Thread Jeff V. Merkey
NT only uses RIP/SAP. Jeff Jes Sorensen wrote: > > > "Jeff" == Jeff V Merkey <[EMAIL PROTECTED]> writes: > > Jeff> Only Linux makes the lights flash with IPX RIP/SAP. NetWare > Jeff> uses NLSP routing and has since 1993 for IPX/SPX. I agree if > Jeff> someone is running NetWare 3 or

Re: zero-copy TCP

2000-09-05 Thread Ingo Molnar
On Wed, 6 Sep 2000, Chris Wedgwood wrote: > [...] The point is that a write() is only used if some sort of > dynamic data is generated on the fly. > > There are exsiting applications out there that use mmap+write > (caching the maps), it would be nice for the authors of these not to > h

Re: zero-copy TCP

2000-09-05 Thread dean gaudet
On Mon, 4 Sep 2000, Jamie Lokier wrote: > Alan Cox wrote: > > > It's not faster than card->card DMA, which falls out naturally from my > > > zero-copy proposal :-) > > > > We already support card->card DMA for routing with fastrouting > > ..but not for user space proxies which was the above's c

Re: zero-copy TCP

2000-09-05 Thread dean gaudet
On Mon, 4 Sep 2000, Linus Torvalds wrote: > On Mon, 4 Sep 2000, Jamie Lokier wrote: > > > Linus Torvalds wrote: > > > > > Basically any copy <= 4 cache lines is "free" compared to trying to be > > > clever. > > > > We're obviously interested in larger packets than 128 bytes. > > "obviously"?

Re: zero-copy TCP

2000-09-05 Thread Henning P. Schmiedehausen
[EMAIL PROTECTED] (Jeff V. Merkey) writes: >Linus Torvalds wrote: >> >> >> >> Basically, only TCP and UDP really matter. Decnet, IPX, etc don't really >> make a big selling point any more. >> >> >Linus, >IPX is a really good LAN protocol (but totally sucks for internet). A >full blown N

Re: zero-copy TCP

2000-09-05 Thread Derek Fawcus
On Tue, Sep 05, 2000 at 04:35:43AM -0600, Jeff V. Merkey wrote: > > IPX is a really good LAN protocol (but totally sucks for internet). A > full blown NCP server in-kernel that's toughtly coupled to the page > cache running over IPX would make flames shoot out of the back of a > Linux server, and

Re: zero-copy TCP

2000-09-05 Thread Jeff V. Merkey
Ingo Molnar wrote: > > On Tue, 5 Sep 2000, Jeff V. Merkey wrote: > > > The origin of this comment was related to a comparison of the > > MSM/TSM/CSM layer in NetWare and Linux. I've already said that Alan's > > code handles fast paths well and from what I've seen is comparable to > > NetWare.

Re: zero-copy TCP

2000-09-05 Thread Ingo Molnar
On Tue, 5 Sep 2000, Jeff V. Merkey wrote: > The origin of this comment was related to a comparison of the > MSM/TSM/CSM layer in NetWare and Linux. I've already said that Alan's > code handles fast paths well and from what I've seen is comparable to > NetWare. [...] can we thus take this as a r

Re: zero-copy TCP

2000-09-05 Thread Jeff V. Merkey
You opened your mouth. :-) Jeff Ingo Molnar wrote: > > btw., - the maintainers of the 2.4 networking and TCP/IP code are Alexey > Kuznetsov and David S. Miller - please direct your findings towards them, > not me :-) > > Ingo - To unsubscribe from this list: send the line "unsubscr

Re: zero-copy TCP

2000-09-05 Thread Jeff V. Merkey
The origin of this comment was related to a comparison of the MSM/TSM/CSM layer in NetWare and Linux. I've already said that Alan's code handles fast paths well and from what I've seen is comparable to NetWare. The areas I saw where sideband cases and issues of fragment re-assembly. It

Re: zero-copy TCP

2000-09-05 Thread Ingo Molnar
btw., - the maintainers of the 2.4 networking and TCP/IP code are Alexey Kuznetsov and David S. Miller - please direct your findings towards them, not me :-) Ingo - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Plea

Re: zero-copy TCP

2000-09-05 Thread Ingo Molnar
On Tue, 5 Sep 2000, Jeff V. Merkey wrote: > Alright Ingo, you asked for it. I am going through it now and going > over ALL my notes. I will catalog ALL of them and post it. Is this > what you really want? yes, this would be the best indeed, to get those places fixed. But if you dont want to spe

Re: zero-copy TCP

2000-09-05 Thread Jeff V. Merkey
Alright Ingo, you asked for it. I am going through it now and going over ALL my notes. I will catalog ALL of them and post it. Is this what you really want? :-) Jeff Ingo Molnar wrote: > > On Tue, 5 Sep 2000, Jeff V. Merkey wrote: > > > > > while (x) > > > > { > > > > x = x->next >

Re: zero-copy TCP

2000-09-05 Thread Jeff V. Merkey
Linus Torvalds wrote: > > > > Basically, only TCP and UDP really matter. Decnet, IPX, etc don't really > make a big selling point any more. > > Linus, IPX is a really good LAN protocol (but totally sucks for internet). A full blown NCP server in-kernel that's toughtly coupled to the page

Re: zero-copy TCP

2000-09-05 Thread Ingo Molnar
On Tue, 5 Sep 2000, Jeff V. Merkey wrote: > > > while (x) > > > { > > > x = x->next > > > } > > > > > > all over the place that increases latency. [...] > > > > i challenge you to show one such place in the 2.4.0-test8-pre2 kernel. If > > it's all over the place and if it increases latency, y

Re: zero-copy TCP

2000-09-05 Thread Jeff V. Merkey
Intel Nitro Card - i960 processor on the card. The SMP debugging involved the use of a bus analyser since this card had a piggish memory bus footprint (i960 processors do not have an IO address space, so everything is memory mapped. The big weakness of early I2O stuff was that the I960 running

Re: zero-copy TCP

2000-09-05 Thread Jeff V. Merkey
Ingo, When I have time to do this exercise, I will. I've finished merging Alan's Code into MANOS (completed last night). Most of the cases I saw where there were copies were not fast path. It takes some time to go through all this code you guys have written. It is actually looking good. J

Re: zero-copy TCP

2000-09-05 Thread Jeff V. Merkey
Was a complete overhaul of the TCP/IP stack > needed like you claim? Not at all - the Linux TCP/IP code was so generic, > that to my surprise i saw the first zero-copy TCP transfer after only one > day of hacking. > > Ingo > > - > To unsubscribe from this list: sen

Re: zero-copy TCP

2000-09-04 Thread Dan Kegel
[EMAIL PROTECTED] wrote: > Few month ago, I gathered precice data and posted it on lk-ml. In our > experiment, I used four 100Base cards, the Web-Bench gained nearly 5% > performance by the patch. The CPU load reached over 95%. > > I want to show the reference to experiments results, But unfor

Re: zero-copy TCP

2000-09-04 Thread Linus Torvalds
On Mon, 4 Sep 2000, Jamie Lokier wrote: > Linus Torvalds wrote: > > > Basically any copy <= 4 cache lines is "free" compared to trying to be > > clever. > > We're obviously interested in larger packets than 128 bytes. "obviously"? Take a look at some common traffic. Yes, even in servers.

Re: zero-copy TCP

2000-09-04 Thread Jes Sorensen
> "Richard" == Richard Gooch <[EMAIL PROTECTED]> writes: Richard> I thought you said some of the GigE drivers supported this? Richard> Or were you just saying that the GigE cards were some of the Richard> few which supported scatter/gather DMA and IP checksumming? The latter. Jes - To unsub

Re: zero-copy TCP

2000-09-04 Thread Stephen C. Tweedie
Hi, On Sun, Sep 03, 2000 at 07:29:56PM +0200, Ingo Molnar wrote: > > On Sun, 3 Sep 2000, Andi Kleen wrote: > > > I did the same for fragment RX some months ago (simple fragment lists > > that were copy-checksummed to user space). Overall it is probably > > better to use a kiovec, because that c

Re: zero-copy TCP

2000-09-04 Thread Richard Gooch
Jes Sorensen writes: > > "Richard" == Richard Gooch <[EMAIL PROTECTED]> writes: > > Richard> Andrew Morton writes: > >> All of them except the 3c905 provide hardware Rx and Tx > >> checksumming of IP, TCP and UDP headers. No 64 bit addressing > >> support. > > Richard> And does the driver s

Re: zero-copy TCP

2000-09-04 Thread Jes Sorensen
> "Ingo" == Ingo Molnar <[EMAIL PROTECTED]> writes: Ingo> On Mon, 4 Sep 2000 [EMAIL PROTECTED] wrote: >> The experiment showed the following prefetching could reduce 20-30% >> of csum_partial_copy_generic() execution time. Ingo> Please test it and post the numbers. csum_partial_copy_generic

Re: zero-copy TCP

2000-09-04 Thread Jes Sorensen
> "Jamie" == Jamie Lokier <[EMAIL PROTECTED]> writes: Jamie> Nice point! Only valid for TCP & UDP though. Jamie> When people want _real_ low latency, they don't use TCP or UDP, Jamie> and they certainly don't put data checksums at the start. They Jamie> still aim for zero copies. That pas

Re: zero-copy TCP

2000-09-04 Thread Jes Sorensen
> "Ingo" == Ingo Molnar <[EMAIL PROTECTED]> writes: Ingo> On Sun, 3 Sep 2000, Andi Kleen wrote: >> I did the same for fragment RX some months ago (simple fragment >> lists that were copy-checksummed to user space). Overall it is >> probably better to use a kiovec, because that can be more ea

Re: zero-copy TCP

2000-09-04 Thread Jes Sorensen
> "Jamie" == Jamie Lokier <[EMAIL PROTECTED]> writes: Jamie> It's not faster than card->card DMA, which falls out naturally Jamie> from my zero-copy proposal :-) Except that many cards and PCI bridges don't work and you lose the buffering aspect in this case. Jes - To unsubscribe from this

Re: zero-copy TCP

2000-09-04 Thread Jes Sorensen
> "Ingo" == Ingo Molnar <[EMAIL PROTECTED]> writes: Ingo> i believe such zero-copy send should only be allowed for drivers Ingo> which can guarantee correct checksums. (ie. cards which do Ingo> Tx-checksums) The other drivers will still copy. I dont think Ingo> this is a problem - the number

Re: zero-copy TCP

2000-09-04 Thread Jes Sorensen
> "Richard" == Richard Gooch <[EMAIL PROTECTED]> writes: Richard> Andrew Morton writes: >> All of them except the 3c905 provide hardware Rx and Tx >> checksumming of IP, TCP and UDP headers. No 64 bit addressing >> support. Richard> And does the driver support it? Has anyone benchmarked the

Re: zero-copy TCP

2000-09-04 Thread kumon
Ingo Molnar writes: > On Mon, 4 Sep 2000 [EMAIL PROTECTED] wrote: > > The experiment showed the following prefetching could reduce 20-30% of > > csum_partial_copy_generic() execution time. > > Please test it and post the numbers. csum_partial_copy_generic() already > does prefetching - the r

Re: zero-copy TCP

2000-09-04 Thread Jamie Lokier
Linus Torvalds wrote: > (The "invalidate on write" is the sane way of doing SMP cache coherency, > which is probably why. Trying to have shared dirty cache-lines is just > not a viable option in the end). With DMA from a device -- "snoop and update" still results in only one owner of the dirty ca

Re: zero-copy TCP

2000-09-04 Thread Jamie Lokier
Alan Cox wrote: > > struct page of course). Note that it doesn't matter if another thread, > > and this includes truncate/write in another thread, clobbers the page > > data. That's just the normal effect of two concurrent writers to the > > same memory. > > Oh it does matter. You might send ou

Re: zero-copy TCP

2000-09-04 Thread Alan Cox
> > those pages out of cache which isnt so ideal if the CPU wants > > them. Some of the results are suprisingly counter-intuitive like this > > Does it flush the CPU cache? I thought the CPU just snooped the bus and > updated its cache with new data. In general no. > struct page of course).

Re: zero-copy TCP

2000-09-04 Thread Ingo Molnar
On Mon, 4 Sep 2000 [EMAIL PROTECTED] wrote: > The experiment showed the following prefetching could reduce 20-30% of > csum_partial_copy_generic() execution time. Please test it and post the numbers. csum_partial_copy_generic() already does prefetching - the real test would be to check lat_tcp

Re: zero-copy TCP

2000-09-03 Thread kumon
The experiment showed the following prefetching could reduce 20-30% of csum_partial_copy_generic() execution time. This suggests CPU cause heavy cache miss at checksum calclation and it is also supported by the PMC (performance monitoring counter) measurements. So I think the following statement i

Re: zero-copy TCP

2000-09-03 Thread Linus Torvalds
On Sun, 3 Sep 2000, Lincoln Dale wrote: > > many people (myself included) have been experimenting with zerocopy > infrastructures. > in my case, i've been working on it as time permits for quite a few months > now, and am about on my fourth rewrite. Heh. > i've found exactly what you state

Re: zero-copy TCP

2000-09-03 Thread Lincoln Dale
At 22:53 03/09/00, Linus Torvalds wrote: > >Ugh. User space DMA gets complicated quickly. The performance question > >is, perhaps, can you do this without a TLB flush (but with locking the > >struct page of course). Note that it doesn't matter if another thread, > >and this includes truncate/wr

Re: zero-copy TCP

2000-09-03 Thread Linus Torvalds
In article <[EMAIL PROTECTED]>, Jamie Lokier <[EMAIL PROTECTED]> wrote: >Alan Cox wrote: >> > read/recv block while the NIC DMAs into user space main memory. >> >> Thats actually not always a win either. A DMA to user space flushes >> those pages out of cache which isnt so ideal if the CPU wants

Re: zero-copy TCP

2000-09-03 Thread Jamie Lokier
Alan Cox wrote: > > read/recv block while the NIC DMAs into user space main memory. > > Thats actually not always a win either. A DMA to user space flushes > those pages out of cache which isnt so ideal if the CPU wants > them. Some of the results are suprisingly counter-intuitive like this Does

Re: zero-copy TCP

2000-09-03 Thread Jamie Lokier
Alan Cox wrote: > > It's not faster than card->card DMA, which falls out naturally from my > > zero-copy proposal :-) > > We already support card->card DMA for routing with fastrouting ..but not for user space proxies which was the above's context. Still, the fastrouting proves card->card DMA a

Re: zero-copy TCP

2000-09-03 Thread Alan Cox
> It's not faster than card->card DMA, which falls out naturally from my > zero-copy proposal :-) We already support card->card DMA for routing with fastrouting > - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read

Re: zero-copy TCP

2000-09-03 Thread Alan Cox
> read/recv block while the NIC DMAs into user space main memory. Thats actually not always a win either. A DMA to user space flushes those pages out of cache which isnt so ideal if the CPU wants them. Some of the results are suprisingly counter-intuitive like this > (Can't DMA earlier because w

Re: zero-copy TCP

2000-09-03 Thread Linus Torvalds
On Sun, 3 Sep 2000, Jamie Lokier wrote: > > Nice point! Only valid for TCP & UDP though. Yeah. But "we need oxygen" is only a valid point for carbon-based life-forms. You might as well argue that oxygen is not avalid criteria for being livable, because it's only valid for the particular kind

Re: zero-copy TCP

2000-09-03 Thread Jamie Lokier
Ingo Molnar wrote: > > At CERN we had a bunch of applications where this would be a win, data > > aquisition servers taking data in from on some custom hardware and > > sending out data over the wire on another card. You never really want > > to touch data in memory with the CPU but because of the

Re: zero-copy TCP

2000-09-03 Thread Jamie Lokier
Linus Torvalds wrote: > Proof: the data to be sent out is in RAM. In fact, often it is cached > in the CPU these days. In order to start sending out the packet, the > smart card has to move all of the data from RAM/cache over the bus to > the card. It can only start actually sending afte

Re: zero-copy TCP

2000-09-03 Thread Jamie Lokier
Andi Kleen wrote: > On Sun, Sep 03, 2000 at 05:22:44AM +0200, Jamie Lokier wrote: > > I just thought I'd mention that you can do zero copy TCP in and out > > *without* any page marking schemes. All you need is a network card with > > quite a lot of RAM and some intelli

Re: zero-copy TCP

2000-09-03 Thread Jamie Lokier
Alan Cox wrote: > > I just thought I'd mention that you can do zero copy TCP in and out > > *without* any page marking schemes. All you need is a network card with > > No > > > quite a lot of RAM and some intelligence. An Alteon could do it, with > > ex

Re: zero-copy TCP

2000-09-03 Thread Alan Cox
> i think it's a quality of implementation issue. The csum_copy_generic > thing is a bug. Allowing incorrect checksums to be sent out would be a > design bug. I think some RFCs do even forbid the sending of incorrect > packets? You are perfectly at liberty to send invalidly checksummed packets. I

Re: zero-copy TCP

2000-09-03 Thread Dan Kegel
Ingo Molnar ([EMAIL PROTECTED]) wrote: > yep i agree - in this case a receivefile() implementation would be handy > (we are 100% ready in 2.4 to introduce it - from the pagecache and VFS > point of view, it's just not there yet), thus you could receivefile() your > data into a temporary file, a

Re: zero-copy TCP

2000-09-03 Thread Andi Kleen
On Sun, Sep 03, 2000 at 07:42:53PM +0200, Ingo Molnar wrote: > > On Sun, 3 Sep 2000, Andi Kleen wrote: > > > You can already cause incorrect checksums on the wire just by passing > > a partly unmapped address (the zero-the-rest exception handler in > > csum_copy_generic in i386 forgets to add in

Re: zero-copy TCP

2000-09-03 Thread Werner Almesberger
Ingo Molnar wrote: > i believe such zero-copy send should only be allowed for drivers which can > guarantee correct checksums. Hmm, I think this shouldn't be tied too closely to TCP. E.g. you can probably play wonderful ALF tricks with raw sockets. For TCP/UDP, such a restriction may be useful,

Re: zero-copy TCP

2000-09-03 Thread Ingo Molnar
On Sun, 3 Sep 2000, Andi Kleen wrote: > You can already cause incorrect checksums on the wire just by passing > a partly unmapped address (the zero-the-rest exception handler in > csum_copy_generic in i386 forgets to add in the carry) > > I do not believe it is a big deal, packets with bad chec

Re: zero-copy TCP

2000-09-03 Thread Andi Kleen
On Sun, Sep 03, 2000 at 07:21:50PM +0200, Ingo Molnar wrote: > > On Sun, 3 Sep 2000 [EMAIL PROTECTED] wrote: > > > If we go for a Linux-specific solution anyway, maybe one could add > > another send{,to,msg} flag that makes send*(2)'s buffer access > > non-atomic. That way, the kernel only needs

Re: zero-copy TCP

2000-09-03 Thread Ingo Molnar
On 3 Sep 2000, Jes Sorensen wrote: > At CERN we had a bunch of applications where this would be a win, data > aquisition servers taking data in from on some custom hardware and > sending out data over the wire on another card. You never really want > to touch data in memory with the CPU but beca

Re: zero-copy TCP

2000-09-03 Thread Ingo Molnar
On Sun, 3 Sep 2000, Andi Kleen wrote: > I did the same for fragment RX some months ago (simple fragment lists > that were copy-checksummed to user space). Overall it is probably > better to use a kiovec, because that can be more easily used in nfsd > and sendfile. the basic fragment type introd

Re: zero-copy TCP

2000-09-03 Thread Ingo Molnar
On Sun, 3 Sep 2000 [EMAIL PROTECTED] wrote: > If we go for a Linux-specific solution anyway, maybe one could add > another send{,to,msg} flag that makes send*(2)'s buffer access > non-atomic. That way, the kernel only needs to make sure the pages > don't disappear, but there's no need for expens

Re: zero-copy TCP

2000-09-03 Thread Richard Gooch
Andrew Morton writes: > Jes Sorensen wrote: > > > > I only know of a few 100baseT cards that can do it such as the > > Adaptec Starfire and the 3C905B > > (though I am not sure what it provides is sufficient). > > The 3c905, 3c905B, 3c905C and all 3Com Cardbus NICs will do > scatter and gather o

Re: zero-copy TCP

2000-09-03 Thread Jes Sorensen
> "Andrew" == Andrew Morton <[EMAIL PROTECTED]> writes: Andrew> Jes Sorensen wrote: >> I only know of a few 100baseT cards that can do it such as the >> Adaptec Starfire and the 3C905B (though I am not sure what it >> provides is sufficient). Andrew> The 3c905, 3c905B, 3c905C and all 3Com C

Re: zero-copy TCP

2000-09-03 Thread Andrew Morton
Jes Sorensen wrote: > > I only know of a few 100baseT cards that can do it such as the > Adaptec Starfire and the 3C905B > (though I am not sure what it provides is sufficient). The 3c905, 3c905B, 3c905C and all 3Com Cardbus NICs will do scatter and gather of up to 63 fragments per packet with b

Re: zero-copy TCP

2000-09-03 Thread almesber
Ingo Molnar wrote: > On 2 Sep 2000, Jes Sorensen wrote: > > Besides that you need to do copy-on-write if you want to be able to do > > zero copy on write() from user space [...] > > i agree that this is hard - i'm not sure wether we want to go the pain to > enable anonymous-buffer write()s do zer

Re: zero-copy TCP

2000-09-03 Thread Jes Sorensen
> "Ingo" == Ingo Molnar <[EMAIL PROTECTED]> writes: Ingo> On 2 Sep 2000, Jes Sorensen wrote: >> You can't DMA directly from a file cache page unless you have a >> network card that does scatter/gather DMA and surprise surprise, >> 80-90% of the cards on the market don't support this. [...]

Re: zero-copy TCP

2000-09-03 Thread Jes Sorensen
> "Jeff" == Jeff V Merkey <[EMAIL PROTECTED]> writes: Jeff> There's been a few cards around since about 1995, but I don't Jeff> remember all of them. I do remember having to debug SMP code on Jeff> them though -- yec I wouldn't be surprised but I would prefer names. Doing SMP aware

Re: zero-copy TCP

2000-09-03 Thread Alan Cox
> I just thought I'd mention that you can do zero copy TCP in and out > *without* any page marking schemes. All you need is a network card with No > quite a lot of RAM and some intelligence. An Alteon could do it, with > extra RAM or an impressively underloaded network. P

  1   2   >