On 24 June 2014 23:19, Josef 'Jeff' Sipek <[email protected]> wrote: > Does it speed up if you disable apix? (/etc/system: set apix_enable = 0) > > Jeff.
On 24 June 2014 23:41, Dan McDonald <[email protected]> wrote: > On Jun 24, 2014, at 8:59 AM, Richard Elling via illumos-discuss > <[email protected]> wrote: >> Kris, >> are deep C-states enabled? > > If they are, disable them (via the BIOS). > > You mention a CPU difference between your two boxes. Are they the same > socket-type and chipset as well? > > Dan Sorry for the slow response. It's been a hectic couple of days, but I've finally been able to get back to this. There is nothing that explicitly mentions C-states in the description of any of the bios settings, but the power management options are all on the 'screw enenrgy efficiency give me more power' setting. I've set apix_enable = 0 and that has seemed to help the transmit side of the aggregate. I'm still left with receive hovering around 500 Mbit/s. Excerpt from iperf: [ 3] 1270.0-1275.0 sec 275 MBytes 461 Mbits/sec [ 3] 1275.0-1280.0 sec 296 MBytes 496 Mbits/sec [ 3] 1280.0-1285.0 sec 264 MBytes 442 Mbits/sec [ 3] 1285.0-1290.0 sec 292 MBytes 491 Mbits/sec [ 3] 1290.0-1295.0 sec 171 MBytes 286 Mbits/sec [ 3] 1295.0-1300.0 sec 307 MBytes 515 Mbits/sec [ 3] 1300.0-1305.0 sec 278 MBytes 467 Mbits/sec [ 3] 1305.0-1310.0 sec 263 MBytes 441 Mbits/sec [ 3] 1310.0-1315.0 sec 291 MBytes 488 Mbits/sec [ 3] 1315.0-1320.0 sec 272 MBytes 456 Mbits/sec [ 3] 1320.0-1325.0 sec 281 MBytes 471 Mbits/sec I did a quick flamegraph of the kernel stacks (after grepping to remove unix`idle) https://dl.dropboxusercontent.com/content_link/EIqwX5RiX2QJTXiEyj4fP8cCvRvj4HZYktEYqSfE7aYJB9ombZSWhV45wyVhh826 On the box that is working it turns out that recieve is still slower over an aggregate than direct, but still more than ten times better than the problem box. Another excerpt from iperf: [ 3] 1270.0-1275.0 sec 4271 MBytes 7166 Mbits/sec [ 3] 1275.0-1280.0 sec 4121 MBytes 6913 Mbits/sec [ 3] 1280.0-1285.0 sec 3708 MBytes 6221 Mbits/sec [ 3] 1285.0-1290.0 sec 3500 MBytes 5872 Mbits/sec [ 3] 1290.0-1295.0 sec 3494 MBytes 5863 Mbits/sec [ 3] 1295.0-1300.0 sec 3828 MBytes 6423 Mbits/sec [ 3] 1300.0-1305.0 sec 3259 MBytes 5467 Mbits/sec [ 3] 1305.0-1310.0 sec 3856 MBytes 6468 Mbits/sec [ 3] 1310.0-1315.0 sec 3625 MBytes 6082 Mbits/sec [ 3] 1315.0-1320.0 sec 3869 MBytes 6491 Mbits/sec [ 3] 1320.0-1325.0 sec 3713 MBytes 6229 Mbits/sec [ 3] 1325.0-1330.0 sec 4040 MBytes 6779 Mbits/sec A flamegraph of the kernel stacks on this box (I also grepped out zfs`zio_execute and genunix`ioctl because this box is pretty much always running at least one "zfs receive" from somewhere) https://dl.dropboxusercontent.com/content_link/NQl8k7n9TjW9YqwOsV2JHB8TWEh5cauALt44AmrNtkZMcIEZeyyejkmHfDhwLqQ8 Counterintuitively (at least to me) it seems to be going faster using software rings instead of hardware rings. ------------------------------------------- illumos-discuss Archives: https://www.listbox.com/member/archive/182180/=now RSS Feed: https://www.listbox.com/member/archive/rss/182180/21175430-2e6923be Modify Your Subscription: https://www.listbox.com/member/?member_id=21175430&id_secret=21175430-6a77cda4 Powered by Listbox: http://www.listbox.com
