Re: raise max value for tcp autosizing buffer [WAS: misc@ network tuning for high bandwidth and high latency]
Hi, On Monday, December 5, 2011 10:53 CET, Sebastian Reitenbach sebas...@l00-bugdead-prods.de wrote: On Sunday, December 4, 2011 21:01 CET, Mark Kettenis mark.kette...@xs4all.nl wrote: Date: Sun, 4 Dec 2011 15:10:56 +0100 From: Claudio Jeker cje...@diehard.n-r-g.com On Sun, Dec 04, 2011 at 01:35:33PM +0100, Sebastian Reitenbach wrote: On Sunday, December 4, 2011 13:24 CET, Camiel Dobbelaar c...@sentia.nl wrote: On 4-12-2011 13:01, Sebastian Reitenbach wrote: the default maximum size of the tcp send and receive buffer used by the autosizing algorithm is way too small, when trying to get maximum speed with high bandwidth and high latency connections. I have tweaked SB_MAX on a system too, but it was for UDP. When running a busy Unbound resolver, the recommendation is too bump the receive buffer to 4M or even 8M. See http://unbound.net/documentation/howto_optimise.html Otherwise a lot of queries are dropped when the cache is cold. I don't think there's a magic value that's right for everyone, so a sysctl would be nice. Maybe separate ones for tcp and udp. I know similar sysctl's have been removed recently, and that they are sometimes abused, but I'd say we have two valid use cases now. So I'd love some more discussion. :-) since they were removed, and there is this keep it simple, and too many knobs are bad attitude, which I think is not too bad, I just bumped the SB_MAX value. If there is consensus that a sysctl would make sense, I'd also look into that approach and send new patch. SB_MAX is there to protect your system. It gives a upperbound on how much memory a socket may allocate. The current value is a compromize. Running with a huge SB_MAX may make one connection faster but it will cause resource starvation issues on busy systems. Sure you can bump it but be aware of the consequneces (and it is why I think we should not bump it at the moment). A proper change needs to include some sort of resource management that ensures that we do not run the kernel out of memory. But 256k simply isn't enough for some use cases. Turning this into a sysctl tunable like FreeBSD and NetBSD would be a good idea if you ask me. Yes, people will use it to shoot themselves in the foot. I don't care. So to be able to shoot myself in the foot without the need to compile the kernel, I'll look into adding a sysctl to tweak the maximum size of the buffer. Well, depending on time and how fast I figure out how to do that, might take some time. here is a first try to add such a sysctl. I called it net.inet.ip.sb-max. A better name, under a different hierarchy maybe? The default value SB_MAX defined in sys/socketvar.h did not changed. I used sysctl_int for the sysctl, but not perfectly sure whether this is right? sb_max is u_long in sys/kern/uipc_socket2.c, so maybe using sysctl_quad? Tested and works for me on i386. Its my first try in kernel land, and I'm no expert with regard to the network stack, so there may be things I should have done better. Please comment and let me know. cheers, Sebastian Index: lib/libc/gen/sysctl.3 === RCS file: /cvs/src/lib/libc/gen/sysctl.3,v retrieving revision 1.210 diff -u -r1.210 sysctl.3 --- lib/libc/gen/sysctl.3 9 Dec 2011 16:14:54 - 1.210 +++ lib/libc/gen/sysctl.3 25 Dec 2011 13:50:15 - @@ -1210,6 +1210,7 @@ .It ip Ta porthilast Ta integer Ta yes .It ip Ta portlast Ta integer Ta yes .It ip Ta redirect Ta integer Ta yes +.It ip Ta sb-max Ta integer Ta yes .It ip Ta sourceroute Ta integer Ta yes .It ip Ta stats Ta structure Ta no .It ip Ta ttl Ta integer Ta yes @@ -1517,6 +1518,9 @@ .Tn IP packets, and should normally be enabled on all systems. +.It Li ip.sb-max +Maximum size of socket buffers. This value is also used by the TCP +send and receive buffer autosizing algorithm. .It Li ip.sourceroute Returns 1 when forwarding of source-routed packets is enabled for the host. Index: sbin/sysctl/sysctl.8 === RCS file: /cvs/src/sbin/sysctl/sysctl.8,v retrieving revision 1.162 diff -u -r1.162 sysctl.8 --- sbin/sysctl/sysctl.83 Sep 2011 22:59:08 - 1.162 +++ sbin/sysctl/sysctl.825 Dec 2011 13:50:54 - @@ -228,6 +228,7 @@ .It net.inet.ip.porthilast Ta integer Ta yes .It net.inet.ip.maxqueue Ta integer Ta yes .It net.inet.ip.encdebug Ta integer Ta yes +.It net.inet.ip.sb-max Ta integer Ta yes .It net.inet.ip.ipsec-expire-acquire Ta integer Ta yes .It net.inet.ip.ipsec-invalid-life Ta integer Ta yes .It net.inet.ip.ipsec-pfs Ta integer Ta yes Index: sys/kern/uipc_socket2.c === RCS file:
Pipe text from mg to external command
This diff allows a user pipe text from current region in mg to
external command. Also adds a command to mark whole buffer. Comments?
Index: def.h
===
RCS file: /home/sunil/cvs/src/usr.bin/mg/def.h,v
retrieving revision 1.118
diff -u -p -r1.118 def.h
--- def.h 10 Dec 2011 14:09:48 - 1.118
+++ def.h 25 Dec 2011 19:18:58 -
@@ -567,6 +567,8 @@ int prefixregion(int, int);
int setprefix(int, int);
int region_get_data(struct region *, char *, int);
voidregion_put_data(const char *, int);
+int markbuffer(int, int);
+int cmdregion(int, int);
/* search.c X */
int forwsearch(int, int);
Index: funmap.c
===
RCS file: /home/sunil/cvs/src/usr.bin/mg/funmap.c,v
retrieving revision 1.35
diff -u -p -r1.35 funmap.c
--- funmap.c28 Nov 2011 04:41:39 - 1.35
+++ funmap.c25 Dec 2011 19:24:00 -
@@ -119,6 +119,7 @@ static struct funmap functnames[] = {
{localbind, local-set-key,},
{localunbind, local-unset-key,},
{makebkfile, make-backup-files,},
+ {markbuffer, mark-whole-buffer,},
{do_meta, meta-key-mode,},/* better name, anyone? */
{negative_argument, negative-argument,},
{newline, newline,},
@@ -172,6 +173,7 @@ static struct funmap functnames[] = {
{setfillcol, set-fill-column,},
{setmark, set-mark-command,},
{setprefix, set-prefix-string,},
+ {cmdregion, shell-command-on-region,},
{shrinkwind, shrink-window,},
#ifdef NOTAB
{space_to_tabstop, space-to-tabstop,},
Index: keymap.c
===
RCS file: /home/sunil/cvs/src/usr.bin/mg/keymap.c,v
retrieving revision 1.46
diff -u -p -r1.46 keymap.c
--- keymap.c28 Nov 2011 04:41:39 - 1.46
+++ keymap.c25 Dec 2011 19:19:06 -
@@ -137,7 +137,7 @@ static PF cXcar[] = {
#endif /* !NO_MACRO */
setfillcol, /* f */
gotoline, /* g */
- rescan, /* h */
+ markbuffer, /* h */
fileinsert, /* i */
rescan, /* j */
killbuffer_cmd, /* k */
@@ -259,7 +259,7 @@ static PF metal[] = {
rescan, /* y */
rescan, /* z */
gotobop,/* { */
- rescan, /* | */
+ cmdregion, /* | */
gotoeop /* } */
};
Index: mg.1
===
RCS file: /home/sunil/cvs/src/usr.bin/mg/mg.1,v
retrieving revision 1.57
diff -u -p -r1.57 mg.1
--- mg.128 Nov 2011 07:58:23 - 1.57
+++ mg.125 Dec 2011 19:53:20 -
@@ -196,6 +196,8 @@ call-last-kbd-macro
set-fill-column
.It C-x g
goto-line
+.It C-x h
+mark-whole-buffer
.It C-x i
insert-file
.It C-x k
@@ -260,6 +262,8 @@ copy-region-as-kill
execute-extended-command
.It M-{
backward-paragraph
+.It M-|
+shell-command-on-region
.It M-}
forward-paragraph
.It M-~
@@ -572,6 +576,9 @@ Bind a key mapping in the local (topmost
Unbind a key mapping in the local (topmost) mode.
.It make-backup-files
Toggle generation of backup files.
+.It mark-whole-buffer
+Marks whole buffer as a region by putting dot at the beginning and mark
+at the end of buffer.
.It meta-key-mode
When disabled, the meta key can be used to insert extended-ascii (8-bit)
characters.
@@ -734,6 +741,8 @@ Used by auto-fill-mode.
Sets the mark in the current window to the current dot location.
.It set-prefix-string
Sets the prefix string to be used by the 'prefix-region' command.
+.It shell-command-on-region
+Provide the text in region to the shell command as input.
.It shrink-window
Shrink current window by one line.
The window immediately below is expanded to pick up the slack.
Index: region.c
===
RCS file: /home/sunil/cvs/src/usr.bin/mg/region.c,v
retrieving revision 1.29
diff -u -p -r1.29 region.c
--- region.c5 Jun 2009 18:02:06 - 1.29
+++ region.c25 Dec 2011 19:38:22 -
@@ -9,6 +9,8 @@
* internal use.
*/
+#include stdio.h
+
#include def.h
static int getregion(struct region *);
@@ -366,4 +368,66 @@ region_put_data(const char *buf, int len
else
linsert(1, buf[i]);
}
+}
+
+/*
+ * Mark whole buffer by first traversing to end-of-buffer
+ * and then to beginning-of-buffer. Mark, dot are implicitly
+ * set to eob, bob respectively during traversal.
+ */
+int
+markbuffer(int f, int n)
+{
+ if (gotoeob(f,n) == FALSE)
+ return (FALSE);
+ if (gotobob(f,n) == FALSE)
+ return (FALSE);
+ return (TRUE);
+}
+
+/*
+ * Pipe text from current region
