[naviserver-devel] Socket buffer sizes
Hi! I am in the process of evaluating the possibility to correct the current behaviour WRT manipulating socket buffer sizes... So far I could understand from reading tons of docs found all over the internet, the socket buffer sizes are crucial for optimizing the network peformance related to fast, high-latency links. We have customers that would benefit from this so I need to find the way to make this happen in the Naviserver. I did find out that different OS'es handle such cases differently. Most of them require admins to tune either system or per-app buffer sizes to achieve good BDP (Bandwidh-Delay Product). Some (newer Linux'es) do this automatically. Currently we do already manipulate the socket buffer sizes in the main driver code (only). Unfortunately this manipulation happens (at least for the read-buffer-size) too late! As I understand, the socket buffers for reading must be setup BEFORE listen() call. Whereas we manipulate the sizes AFTER the connection is established. Furthermore, the Tcl API commands related to sockets (like ns_socklisten et.al.) also need to be expanded to allow for explicit buffer size manipulation. This all would definitely require some internal changes perhaps even adding new Ns API calls. I wonder if anybody already thought about this topic and what was the outcome? Cheers Zoran -- ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] Socket buffer sizes
On 10.05.2010, at 12:55, Stephen Deasey wrote: Which code are you looking at? IIRC, you removed this code years ago: http://bitbucket.org/naviserver/naviserver/changeset/a6e8a6348da0 We do still have people using old'er OS'es and by scrapping this out they were forced to change buffer sizes on the OS level (system-wide) which created side-effects we are now fighting with. And it is very expensive to do this on the customer site :-( Hence I thought to re-vive the socket buffers manipulation and extend it to all API (including Tcl). But this time I wanted to make it correct. Mostly people using HTTP services only will not be really caring too much about this. But we use HTTP connections to shuffle TB of data accross networks and we are very interested in doing all sorts of optimisations that we can. What I have in mind is to: a. make necessary changes to the driver code so that socket options are set before we actually call listen(). b. add ?-socketbuffersize size? option to all Tcl API commands that work with sockets. The a. is really the most work as it will require either chaning existing (sub-optimal) and/or adding new (more optimal) C-APIs. The b. is relatively harmless and should be straight-forward once the a. is done. The default should be no changes as mostly the OS'es would do the right thing. BTW... I have really found out that only newer Linuxes do this automatically (re-size buffers to accomondate to high-latency links). So far neither Solaris nor Mac OSX nor Windows do this. I may be wrong but I was not able to find anything related on the internet. Cheers Zoran -- ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] Socket buffer sizes
On Mon, May 10, 2010 at 7:22 PM, Andrew Piskorski a...@piskorski.com wrote: On Mon, May 10, 2010 at 11:39:26AM +0200, Vasiljevic Zoran wrote: So far I could understand from reading tons of docs found all over the internet, the socket buffer sizes are crucial for optimizing the network peformance related to fast, high-latency links. Sounds like what the hpn-ssh patches do for ssh: http://www.psc.edu/networking/projects/hpn-ssh/ Zoran: Looks like it would be a mistake to use setsockopt() on Linux = 2.6.17 (released June 2006) ie. RHEL 5+ (ignoring any patches Redhat may have backported): http://www.psc.edu/networking/projects/tcptune/#detailed http://kbase.redhat.com/faq/docs/DOC-3079 -- ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] Socket buffer sizes
On 10.05.2010, at 21:13, Stephen Deasey wrote: Looks like it would be a mistake to use setsockopt() on Linux = I know that. Therefore I said that some OS'es do this automatically. For good or for worse I will (re)add this knobs and leave defaults to zero (== no changes) in both driver code and Tcl API calls. Who needs or wants can play with them. Who doesn't will just use what OS is giving. Reason: we have tons of customers running old OS versions and we need to support them for some years to come. Cheers Zoran -- ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
