On Wed, 13 May 2020 at 23:25, Will Deacon <w...@kernel.org> wrote: > > On Wed, May 13, 2020 at 08:54:03PM +0200, Marco Elver wrote: > > On Wed, 13 May 2020 at 19:47, Will Deacon <w...@kernel.org> wrote: > > > On Wed, May 13, 2020 at 07:32:58PM +0200, Marco Elver wrote: > > > > - We do *not* want the call to __read_once_word_nocheck if we have > > > > __no_sanitize_or_inline. AFAIK that's the main problem -- this applies > > > > to both KASAN and KCSAN. > > > > > > Sorry, I should've been more explicit. The code above, with KASAN enabled, > > > compiles to: > > > > > > ffffffff810a2d50 <foo>: > > > ffffffff810a2d50: 48 8b 07 mov (%rdi),%rax > > > ffffffff810a2d53: c3 retq > > > > > > but with KCSAN enabled, compiles to: > > > > > > ffffffff8109ecd0 <foo>: > > > ffffffff8109ecd0: 53 push %rbx > > > ffffffff8109ecd1: 48 89 fb mov %rdi,%rbx > > > ffffffff8109ecd4: 48 8b 7c 24 08 mov 0x8(%rsp),%rdi > > > ffffffff8109ecd9: e8 52 9c 1a 00 callq ffffffff81248930 > > > <__tsan_func_entry> > > > ffffffff8109ecde: 48 89 df mov %rbx,%rdi > > > ffffffff8109ece1: e8 1a 00 00 00 callq ffffffff8109ed00 > > > <__read_once_word_nocheck> > > > ffffffff8109ece6: 48 89 c3 mov %rax,%rbx > > > ffffffff8109ece9: e8 52 9c 1a 00 callq ffffffff81248940 > > > <__tsan_func_exit> > > > ffffffff8109ecee: 48 89 d8 mov %rbx,%rax > > > ffffffff8109ecf1: 5b pop %rbx > > > ffffffff8109ecf2: c3 retq > > > > > > Is that expected? There don't appear to be any more annotations to throw > > > at it. > > > > Right, so this is expected. > > Fair enough, I just found it weird since it's different to the usual > "disable instrumentation/trace" function annotations. > > > We can definitely make __tsan_func_entry/exit disappear with Clang, with > > GCC it's going to be a while if we want to fix it. > > > > If we remove 'noinline' from __no_kcsan_or_inline, we no longer get > > the call to __read_once_word_nocheck above! But... > > > > For KCSAN we force 'noinline' because older compilers still inline and > > then instrument small functions even if we just have the no_sanitize > > attribute (without inline mentioned). The same is actually true for > > KASAN, so KASAN's READ_ONCE_NOCHECK might be broken in a few places, > > but nobody seems to have noticed [1]. KASAN's __no_kasan_or_inline > > should also have a 'noinline' I think. I just tested > > __no_kcsan_or_inline without 'noinline', and yes, GCC 9 still decided > > to inline a small function and then instrument the accesses. > > > > [1] https://gcc.gnu.org/bugzilla/show_bug.cgi?id=59600 > > > > The good news is that Clang does the right thing when removing > > 'noinline' from __no_kcsan_or_inline: > > 1. doesn't inline into functions that are instrumented, and > > 2. your above example doesn't do the call to __read_once_word_nocheck. > > > > The obvious solution to this is: restrict which compiler we want to support? > > I would be in favour of that, but I defer to the x86 folks since this > affects them much more than it does me. On the arm64 side, we've got patches > queued for 5.8 that require GCC 10.0.1 or later, and that thing is only a > week old. I think it's reasonable to require a recent toolchain for optional > features like this that inherently rely on compiler support. > > > > > From what I gather, we want to just compile the function as if the > > > > sanitizer was never enabled. One reason for why this doesn't quite > > > > work is because of the preprocessor. > > > > > > > > Note that the sanitizers won't complain about these accesses, which > > > > unfortunately is all these attributes ever were documented to do. So > > > > the attributes aren't completely useless. Why doesn't > > > > K[AC]SAN_SANITIZE := n work? > > > > > > I just don't get the point in having a function annotation if you then > > > have to > > > pass flags at the per-object level. That also then necessitates either > > > weird > > > refactoring and grouping of code into "noinstrument.c" type files, or > > > blanket > > > disabling of instrumentation for things like arch/x86/ > > > > If you want a solution now, here is one way to get us closer to where > > we want to be: > > > > 1. Peter's patch to add data_race around __READ_ONCE/__WRITE_ONCE. > > 2. Patch to make __tsan_func_entry/exit disappear with Clang. > > 3. Remove 'noinline' from __no_kcsan_or_inline. > > 4. Patch to warn users that KCSAN may have problems with GCC and > > should use Clang >= 7. > > > > But this is probably only half a solution. > > At this point, I think that if READ_ONCE_NOCHECK() works as expected, and > calling __{READ,WRITE}_ONCE from functions tagged with __no_sanitize doesn't > result in instrumentation, then we're good.
Ouch. With the __{READ,WRITE}_ONCE requirement, we're going to need Clang 11 though. Because without the data_race() around __*_ONCE, arch_atomic_{read,set} will be broken for KCSAN, but we can't have data_race() because it would still add kcsan_{enable,disable}_current() calls to __no_sanitize functions (if compilation unit is instrumented). We can't make arch_atomic functions __no_sanitize_or_inline, because even in code that we want to sanitize, they should remain __always_inline (so they work properly in __no_sanitize functions). Therefore, Clang 11 with support for distinguishing volatiles will be the compiler that will satisfy all the constraints. If this is what we want, let me prepare a series on top of -tip/locking/kcsan with all the things I think we need. Thanks, -- Marco