On 24.10.2012 21:49, Jim Harris wrote:
On Wed, Oct 24, 2012 at 12:16 PM, Andre Oppermann <an...@freebsd.org> wrote:
<snip>
See also the discussion on svn-src-all regarding global struct mtx
alignment.
Thank you for proving my point. ;)
Let's go back and see how we can do this the sanest way. These are
the options I see at the moment:
1. sprinkle __aligned(CACHE_LINE_SIZE) all over the place
2. use a macro like MTX_ALIGN that can be SMP/UP aware and in
the future possibly change to a different compiler dependent
align attribute
3. embed __aligned(CACHE_LINE_SIZE) into struct mtx itself so it
automatically gets aligned in all cases, even when dynamically
allocated.
Personally I'm undecided between #2 and #3. #1 is ugly. In favor
of #3 is that there possibly isn't any case where you'd actually
want the mutex to share a cache line with anything else, even a data
structure.
I've run my same tests with #3 as you describe, and I did see further
noticeable improvement. I had a difficult time though quantifying the
effect it would have on all of the different architectures. Putting
it in ULE's tdq gained 60-70% of the overall benefit, and was well
contained.
I just experimented with different specifications of alignment
and couldn't get the globals aligned at all. This seems to be
because of the linker not understanding or not getting passed
the alignment information when linking the kernel.
I agree that sprinkling all over the place isn't pretty. But focused
investigations into specific locks (spin mutexes, default mutexes,
whatever) may find a few key additional ones that would benefit. I
started down this path with the sleepq and turnstile locks, but none
of those specifically showed noticeable improvement (at least in the
tests I was running). There's still some additional ones I want to
look at, but haven't had the time yet.
This runs the very great risk of optimizing for today's available
architectures and then needs rejiggling every five years. Just as
you've noticed the issue with 128B alignment from the Netburst days.
We never know how the next micro-architecture will behave. Micro
optimizing each individual invocation of common building blocks is
the wrong path to go.
I'd very much prefer the alignment *and* padding control to be done
in one place for all of them, either through a magic macro or compiler
__attribute__(whatever).
--
Andre
_______________________________________________
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"
