On Tue, Dec 01, 2020 at 06:59:22AM +0100, Sebastien Marie wrote:
> On Mon, Nov 30, 2020 at 11:14:46PM +0000, Stuart Henderson wrote:
> > Thought I'd try a WITNESS kernel to see if that gives any clues about
> > what's going on with my APU crashing all over the place (long shot but
> > I got bored with trying different older kernels..)
> >
> > I see these from time to time (one during netstart, and another 4 in
> > 15 mins uptime), anyone know if they're important?
>
> this check ("lock_object uninitialized") was recently added in witness.
>
> it means that the rwlock was uninitialized (the witness flag
> LO_INITIALIZED isn't present whereas it should) when witness check the
> lock.
>
> it could be:
> - someone omits to call rw_init() or RWLOCK_INITIALIZER() (possible bug if
> memory isn't zero)
> - the struct has been zeroed (possible bug too)
>
> > witness: lock_object uninitialized: 0xffff800000bcf0d8
> > Starting stack trace...
> > witness_checkorder(ffff800000bcf0d8,9,0) at witness_checkorder+0xab
> > rw_enter_write(ffff800000bcf0c8) at rw_enter_write+0x43
> > noise_remote_decrypt(ffff800000bcea48,978dc3d,0,fffffd805e22cb7c,10) at
> > noise_remote_decrypt+0x135
> > wg_decap(ffff80000054a000,fffffd8061835200) at wg_decap+0xda
> > wg_decap_worker(ffff80000054a000) at wg_decap_worker+0x7a
> > taskq_thread(ffff80000012d900) at taskq_thread+0x9f
> > end trace frame: 0x0, count: 251
> > End of stack trace.
>
> from the trace, the sole rw_enter_write() usage in noise_remote_decrypt() is
>
> rw_enter_write(&r->r_keypair_lock)
>
> but I am seeing few rw_init() on r_keypair_lock. I will see if I could
> see the source of the problem.
>
Jason, Matt,
sthen@ told me that the same lock is reported several times (exactly,
two locks are reported several times: lock1, lock2, lock1, lock2...)
witness(4) reports when a lock doesn't have LO_INITIALIZED flag set in
internal part of `struct rwlock'. Next it sets it, and skip futher
analysis for this particular lock.
if the same lock is reported several times, it means the memory is
zeroed (and so the flag removed). it could be intented or not. but in
all cases, a rwlock should be properly initialized with rw_init() or
RWLOCK_INITIALIZER() before use.
I don't have enough experiency with wg(4) stuff to well understand the
issue. in wg_decap(), the peer is get from a `struct wg_tag` (from
wg_tag_get()). If i didn't mess myself, the p_remote could come from
wg_send_initiation() or wg_send_response(). but for both, it comes
from wg_peer_create(), and p_remote is initialized with
noise_remote_init() (and so lock have proper rw_init()).
do you have idea on the cause of the lost of the rwlock flag ?
if you want to test it with witness(4) enabled, you will need to build
a kernel with "option WITNESS" in config file. you could uncomment it
from sys/arch/amd64/conf/GENERIC.MP, and run make config, make clean,
make
Thanks.
--
Sebastien Marie
