Re: Van Jacobson's network stack restructure
Hi, Frank. re: http://lists.freebsd.org/pipermail/freebsd-hackers/2006-February/015365.html Who's calling? :-) Frank The original email to which you link above occurred in a discussion regarding the performance, architecture, evolution (or somesuch) of the FreeBSD network stack. The FreeBSD network stack is arguably about the most direct descendent of the TCP/IP stack that was done (forked from a BBN stack, I guess) at UC Berkeley. One reason the UCB (and FreeBSD) stack is of interest is because it essentially became the reference implementation for the modern TCP/IPv4 implementation. In university networking classes, noting that the first TCP link outside of a single lab was between Stanford and London (and thus that the Internet has always been International) makes a good story. Unfortunately it is hard to answer the invariable follow-up questions. I found very little info available the last time I looked (a few years ago). But maybe I missed the obvious. Did you do the UCL PDP-9 implementation on the first link? Was it a stand-alone program? What did it do? Kind of a ping? A test or performance program to debug things? What was the network device like? Did this program evolve, or was the PDP-9 just too small? Was your TCP written in Babbage? Was Babbage an assembler or a structured assembly language with control flow and expressions? Is there any published doc on this language (or programming environment)? How did you work with the Stanford folks? Was the work with the protocol done to a wire spec, or did you look at their implementation? Did you work with their BCPL version of TCP, or hand-port BCPL to Babbage assembly code? etc. etc... Was anything written up and published? Where? (I did try to contact UCL, to no avail.) Alas, all to often today if it's not electronically indexed it's as if it didn't exist. Was this info classified at the time? (I saw somewhere that the big push for a portable standard digital network link to Britain was that Norway happened to have an existing seismic-monitoring network close to a particularly interesting Arctic test site... is this true?) Sorry to go on at length with so many questions. There was some nice information about a new study of the Antikythera mechanism today, surely the mysterious PDP-9 and it's Babbage programming environment can yield a few clues! :) Please correct me anyone if I've got something wrong. - bruce ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Van Jacobson's network stack restructure
http://lists.freebsd.org/pipermail/freebsd-hackers/2006-February/015365.html Who's calling? :-) Frank ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Van Jacobson's network stack restructure
Frank Deignan wrote: http://lists.freebsd.org/pipermail/freebsd-hackers/2006-February/015365.html Who's calling? :-) using the everyone knows someone who is closer to the answer than they are theory, I suggest we forward it to Kirk to forward to Van or Mike.. :-) (CC'd to Kirk (I think I remember the address)) Frank ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Van Jacobson's network stack restructure
Greg 'groggy' Lehey wrote: Last week, at the Linux.conf.au in Dunedin, Van Jacobson presented some slides about work he has been doing rearchitecting the Linux network stack. He claims to have reduced the CPU usage by 80% and doubled network throughput (he expects more, but it was limited by memory bandwidth). The approach looks like it would work on FreeBSD as well. I spoke to him and he confirmed. He's currently trying to get the code released as open source, but in the meantime his slides are up on http://www.lemis.com/grog/Documentation/vj/. Yes, this is my web site. The conference organizers are going to put it up on their web site soon, but in the meantime he's asked me to put it were I can. Comments? It's an interesting approach. However there are a few caveats which put its probable overall performance on par or again with the traditional sockets approach. In his model the buffer (window) resides within user space and is shared with the kernel. This is very losely related to our zero-copy page flipping socket buffer. However this doesn't solve the problem of socket buffer memory overcommit. In fact with his model memory actually in use at any given point in time may be a lot more than the always fully committed socket buffer (in userland share with kernel) plus a number of outstanding packets waiting in the socket queue. The shared user/kernel socket buffer should not be paged out and thus must stay resident. With a large numbers of connections on a machine this gets inefficient because all buffer memory is always committed and not just when it is needed. Memory overcommit goes away. Processing the TCP segments on the same CPU as the userland resides (provided it doesn't migrate [too often]) is certainly beneficial and something we are looking at for some time already. However we are not there yet and have still some work on the TCP stack to do for this to become a reality. Processing the TCP segments within the process CPU quantum and only when it gets selected by the scheduler is a very interesting idea. It has a couple of true advantages and theoretical disadvantages. On the good side it accounts the work in the TCP stack to the process, aggregates processing all segments that arrived between process runs and keeps good cpu/cache locality. On the potential negative side it increases segment latency and has to maintain not only the socket buffer but also another unprocessed-packet buffer. The packet buffer has to be limited or we open ourselfs up to memory exhaustion attacks. When many packets for a connection arrive and the process doesn't get scheduled quickly enough we may get packet loss because packet queue overflows. This can be dealt with in relatively good ways though. Summary: Some gems in there and we are certainly looking at a couple of those ideas to adapt to our network stack in the future. -- Andre ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Van Jacobson's network stack restructure
Greg 'groggy' Lehey wrote: Last week, at the Linux.conf.au in Dunedin, Van Jacobson presented some slides about work he has been doing rearchitecting the Linux network stack. He claims to have reduced the CPU usage by 80% and doubled network throughput (he expects more, but it was limited by memory bandwidth). The approach looks like it would work on FreeBSD as well. I spoke to him and he confirmed. Comments? The approach there is very similar to Lazy Receiver Processing. More details are available here : http://www.cs.rice.edu/CS/Systems/LRP/ http://www.cs.rice.edu/CS/Systems/ScalaServer/code/rescon-lrp/README.html Maybe people want to look at that as well. Aniruddha ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Van Jacobson's network stack restructure
Hi, hope this isn't too off-topic, but it's a reasonably hackery follow-up re a minor historical question instigated by Van Jacobson's Slide 6, which contains the following point: First TCP/IP stack done on Multics (1980) Presumably this means the first version of the specific TCP/IP stack with the architecture that evolved/forked/more-or-less-directly-was-reimplmented to become the BSD kernel-resident TCP/IP stack. 1. Was this first Multics stack written in PL/1? Based on literature/web, etc., the very first implementaton of TCP was a 24 Kbyte BCPL program written by Richard Karp at Stanford for the ELF operating system (this was in Cerf's lab). In 1975 the first test of TCP outside a single lab occured when a link was established between this stack and a TCP program of some sort written in Babbage by Frank Deignan on a very small PDP-9 at University College London. BBN did a 3rd independent BCPL implementation of TCP under (user-level) TENEX. All 3 TCP stacks first inter-operated in 1977. TCP was split into TCP and IP in 1979. The sliding window protocol comes from Pouzin's Cyclades system (which became operational in France sometime between 1972-1974; Cerf and Kahn acknowledged the influence of Cyclades on TCP); Pouzin had worked at MIT on CTSS and coined the term shell. 2. Was the Multics stack based on one of the BCPL stacks? 3. Does anyone know anything about the UCL Babbage TCP implmentation, or anything about the Babbage language? (I assume it was a structured assembly language, similar to PL/360 or SUPMAC). 4. Does anyone know anything about Frank Deignan? Just curious, have tried to find out information about this for awhile. Thanks in advance if you know anything relating to any of this. - bruce ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Van Jacobson's network stack restructure
Last week, at the Linux.conf.au in Dunedin, Van Jacobson presented some slides about work he has been doing rearchitecting the Linux network stack. He claims to have reduced the CPU usage by 80% and doubled network throughput (he expects more, but it was limited by memory bandwidth). The approach looks like it would work on FreeBSD as well. I spoke to him and he confirmed. He's currently trying to get the code released as open source, but in the meantime his slides are up on http://www.lemis.com/grog/Documentation/vj/. Yes, this is my web site. The conference organizers are going to put it up on their web site soon, but in the meantime he's asked me to put it were I can. Comments? Greg -- See complete headers for address and phone numbers. pgpgvFs1DpfT6.pgp Description: PGP signature
Re: Van Jacobson's network stack restructure
Greg 'groggy' Lehey wrote this message on Wed, Feb 01, 2006 at 11:50 +1030: Last week, at the Linux.conf.au in Dunedin, Van Jacobson presented some slides about work he has been doing rearchitecting the Linux network stack. He claims to have reduced the CPU usage by 80% and doubled network throughput (he expects more, but it was limited by memory bandwidth). The approach looks like it would work on FreeBSD as well. I spoke to him and he confirmed. He's currently trying to get the code released as open source, but in the meantime his slides are up on http://www.lemis.com/grog/Documentation/vj/. Yes, this is my web site. The conference organizers are going to put it up on their web site soon, but in the meantime he's asked me to put it were I can. Comments? I must say that the idea of requiring the userland to provide receive buffers before you can open up a window is a cool idea... This means that instead of the normal failure mode of a box that can't handle all the data it's receiving because of lack of cpu processing, the kernel buffers won't fill up.. only the allocated userland buffers will.. This also has interesting possibilities for smarter ethernet cards where the card can dump it directly into the userland buffer w/o having to do the special page flipping thing we can do now... definately some interesting ideas... -- John-Mark Gurney Voice: +1 415 225 5579 All that I will do, has been done, All that I have, has not. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
