----- Original Message -----
From: "Andre Oppermann" <an...@freebsd.org>
To: "Steven Hartland" <kill...@multiplay.co.uk>
Cc: "Alfred Perlstein" <alf...@ixsystems.com>; <sta...@freebsd.org>;
<r...@freebsd.org>; <nones...@longcount.org>
Sent: Thursday, July 11, 2013 1:47 PM
Subject: Re: status of autotuning freebsd for 9.2
On 11.07.2013 11:08, Steven Hartland wrote:
----- Original Message ----- From: "Andre Oppermann" <an...@freebsd.org>
On 08.07.2013 16:37, Andre Oppermann wrote:
On 07.07.2013 20:24, Alfred Perlstein wrote:
On 7/7/13 1:34 AM, Andre Oppermann wrote:
Can you help me with with testing?
Yes. Please give me your proposed changes and I'll stand up a machine and give
feedback.
http://people.freebsd.org/~andre/mfc-autotuning-20130708.diff
Any feedback from testers on this? The MFC window is closing soon.
Few things I've noticed most of which look like issues against the original
patch and not the MFC but worth mentioning.
1. You've introduced a new tunable kern.maxmbufmem which is autosized but
doesnt seem to be exposed via a sysctl so it looks like there is no way
to determine what its actually set to?
Good point. I've made it global and exposed as kern.ipc.maxmbufmem (RDTUN).
2. There's a missmatch between the tuneable kern.ipc.nmbufs in tunable_mbinit
and the sysctl kern.ipc.nmbuf i.e. no 's'.
That's a typo, fixed.
3. Should kern.maxmbufmem be kern.ipc.maxmbufmem to sit along side all of
the other sysctls?
Yes, see above.
4. style issues:
* @@ -178,11 +202,13 @@
...
if (newnmbjumbo9 > nmbjumbo9&&
Thanks. All fixed in r253204.
Finally out of interest what made us arrive at the various defaults for each
type as it looks like the ratios have changed?
Before it was an arbitrary mess. Mbufs were not limited at all and the others
to some random multiple of maxusers with the net limit ending up at some 25,000
mbuf clusters by default.
Now default overall limit is set at 50% of all available min(physical, kmem_map)
memory to prevent mbufs from monopolizing kernel memory and leave some space for
other kernel structures and buffers as well as user-space programs. It can be
raised to 3/4 of available memory by the tunable.
2K and 4K (page size) mbuf clusters can each go up to 25% of this mbuf memory.
The former is dominantly used on the receive path and the latter in the send
path.
9K and 16K jumbo mbuf clusters can each go up to 12.5% of mbuf memory. They are
only used in the receive path if large jumbo MTUs on a network interface are
active.
Both are special in that their memory is contiguous in KVM and physical memory.
This becomes problematic due to memory fragmentation after a short amount of
heavy
system use. I hope to deprecate them for 10.0. Network interfaces should use
4K
clusters instead and chain them together for larger packets. All modern NICs
support that. Only the early and limited DMA engines without scatter-gather
capabilities required contiguous physical memory. They are long gone by now.
The limit for mbufs itselfs is 12.5% of mbuf memory and should be at least as
many as the sum of the cluster types. Each cluster requires an mbuf to which
it is attached.
Two examples on the revised mbuf sizing limits:
1GB KVM:
512MB limit for mbufs
419,430 mbufs
65,536 2K mbuf clusters
32,768 4K mbuf clusters
9,709 9K mbuf clusters
5,461 16K mbuf clusters
16GB RAM:
8GB limit for mbufs
33,554,432 mbufs
1,048,576 2K mbuf clusters
524,288 4K mbuf clusters
155,344 9K mbuf clusters
87,381 16K mbuf clusters
These defaults should be sufficient for even the most demanding network loads.
For additional information see:
http://svnweb.freebsd.org/changeset/base/243631
Thanks for that Andre and thanks for doing this work its something thats been
sorely
needed for a long time.
Am I right is saying there is now the new concept of max mbufs? If so can we get
this added to the output of netstat -m if it already isn't?
I also think this is worth a mention in UPDATING too given the "defaults" are
now becoming sensible ;-)
Regards
Steve
Regards
Steve
================================================
This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it.
In the event of misdirection, illegible or incomplete transmission please
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.
_______________________________________________
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
