Sao Kiselkov writes: > On 06/12/2012 05:37 PM, Roch Bourbonnais wrote: > > > > So the xcall are necessary part of memory reclaiming, when one needs to > > tear down the TLB entry mapping the physical memory (which can from here > > on be repurposed). > > So the xcall are just part of this. Should not cause trouble, but they do. > > They consume a cpu for some time. > > > > That in turn can cause infrequent latency bubble on the network. A certain > > root cause of these latency bubble is that network thread are bound by > > default and > > if the xcall storm ends up on the CPU that the network thread is bound to, > > it will wait for the storm to pass. > > I understand, but the xcall storm settles only eats up a single core out > of a total of 32, plus it's not a single specific one, it tends to > change, so what are the odds of hitting the same core as the one on > which the mac thread is running? >
That's easy :-) : 1/32 each time it needs to run. So depending on how often it runs (which depends on how much churn there is in the ARC) and how often you see the latency bubbles, that may or may not be it. What is zio_taskq_batch_pct on your system ? That is another storm bit of code which causes bubble. Setting it down to 50 (versus an older default of 100) should help if it's not done already. -r > > So try unbinding the mac threads; it may help you here. > > How do I do that? All I can find on interrupt fencing and the like is to > simply set certain processors to no-intr, which moves all of the > interrupts and it doesn't prevent the xcall storm choosing to affect > these CPUs either... > > -- > Saso _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
