Am 04/05/12 20:03, schrieb Arnaud Lacombe: > Hi folks, > > Over the past months, I ran on a couple of unused box the > `hackbench'[HACKBENCH] benchmark used by the Linux folks for tracking > down various kind of regression/improvement. `hackbench' is a > scheduler + IPC test (socket xor pipe). It creates producers/consumers > groups and let a variable quantity of small messages flow happily. > Producers and consumers are either processes xor threads. > > Tested platforms were > - Atom D510, Intel, (incomplete) > - Core 2 Quad Q9560, Intel > - Soekris net5501, AMD (incomplete) > - Xeon E5645, Intel (incomplete) > - Xeon E5620 (dual package), Intel > - Xeon E5-1650 (pending completion) > - Vortex86, DMP > > Tested kernel were: > - FreeBSD 7.4-RELEASE > - FreeBSD 8.2-RELEASE > - FreeBSD 9.0-RC3 and FreeBSD 9.0-RELEASE > - FreeBSD 10-CURRENT as of r231573 > > on the following architecture: > - amd64 (if supported, incomplete) > - i386 > > 1) DISCLAMER > > Let me start by pointing out something important: > > [I] "I am _not_ interested in testing released version FOO with > feature BAR enabled, if enabling BAR require a kernel rebuild." > > All tests for release kernels were made for the kernel shipped > officially, it is the developers responsability to decide whether or > not enable a feature by default, not mine. If option BAR gives a 20% > performance improvement, enable it, don't complain the test to be 20% > slower. > > [II] "I am _not_ interested in altering any hints, tunables or > sysctl, unless they prevent the execution of the test." > > The exception added to the above rule is due to limitation introduced > by `kern.threads.max_threads_per_proc' and `kern.ipc.maxpipekva'. > Those were respectively set to 8192 and between 16M/64M. > > note: rule [I] is alleviated for -CURRENT kernels, which were built > with the same alteration made to GENERIC during the CURRENT->RELEASE > transition (ie. WITNESS and a couple of other option disabled). > > > 2) Tests description > > `hackbench' has the following tunable: > - IPC to use for messaging, either `pipe' or `socket'. > - Threading model, either `thread' or `process' > - Number of iteration to run > - Number of group to create > > The tests covered all of these adjustments more or less heavily > depending on the platform capability. > > > 3) Scripts > > Test scripts are available in the `master' branch of the git repository at: > > https://github.com/lacombar/hackbench > > in the `hackbench/' directory. > > > 4) Results > > Full results are available in the `runs/*' branches of the GitHub repository. > > > 5) Quick results summary > > * UP case > > FreeBSD 9.0 behaves better than FreeBSD 8.2 in process mode, > especially with sockets. Results are comparable with thread. 9.0-RC3 > shows a 10% hits in thread/socket mode on the LX800, this will need > confirmation. > > Linux is stable and scales linearly in all situation. It is only > beaten by FreeBSD 8.2-RELEASE with thread/socket. > > * MP case > > These is a pretty bad regression with FreeBSD 9.0 in thread/pipe mode, > which scale almost in O(N^2), ending up in way worse performance than > FreeBSD 7.4 or 8.2 on the Core 2 Quad. Beside that, it is really > difficult to draw a general trends, ie. whether FreeBSD 9.0 behaves > better than FreeBSD 8.2, or the other way around. Pretty much all > situation arises, FreeBSD 9.0 can beat FreeBSD 8.2 on some workload, > behave the same, or be beaten on others. None really scales regularly > either. Pretty much every runs shows thresholds where scheduling > decision change and/or became erratic. > > 6) Anticipated question and remarks > > Q1: "You should truly enable kick-ass feature BAZ in the kernel." > R1: "I'm lazy. Do your job as a developer to integrate the feature. If > it should be the default, make it the default." > > Q2: "You should set `kern.vm.whatever' to 42, or enable feature BAZ in > the kernel, to get full performance from the Warp engine on > Constellation-class starship." > R2: "Would you ask Lt. Worf to re-aligh plasma injectors or would ask > Lt. Commander La Forge to plan an assault, seriously ?" > > Q3: "You built the Linux kernel, why can't you rebuild FreeBSD's ?" > For a couple of reason: > > - the Linux kernel does not provide binary release per-se. > > - the Linux kernel was not the focus of the tests, but merely a > comparative of what others-can-do. > > - I did not tweak the Linux kernel configuration. The kernels > configuration tested derived from the `defconfig', with very few > amendment[0], mostly about hardware support not enabled by default > > Q3: "Could you post all the graph ?" > R3: I could, but there is really tons of them, so posting a subset of > them would be subjective, all the materials is available on the git > repository. > > Q4: "So, how can I get all the graph ?" > R4: All you need is git, a posix shell, a couple of utility (find, > sort, ...), a recent gnuplot, and a ruby interpreter. > > Comments and suggestions will be greatly appreciated. > > - Arnaud > > [HACKBENCH]: http://people.redhat.com/mingo/cfs-scheduler/tools/hackbench.c > > [0]: the exact list is: > > # CONFIG_KERNEL_GZIP is not set > CONFIG_KERNEL_XZ=y > CONFIG_IKCONFIG=y > CONFIG_IKCONFIG_PROC=y > # CONFIG_MODULES is not set > CONFIG_X86_BIGSMP=y > CONFIG_NR_CPUS=32 > CONFIG_PATA_IT8213=y > CONFIG_PATA_IT821X=y > CONFIG_IGB=y > CONFIG_IGBVF=y > CONFIG_IXGB=y > CONFIG_IXGBE=y > CONFIG_IXGBEVF=y > # CONFIG_EXT3_FS is not set > CONFIG_EXT4_FS=y
Thank you very much for this work!
signature.asc
Description: OpenPGP digital signature
