Re: Panic [page fault] in _ieee80211_crypto_delkey(): stable/10/amd64 @r286878

2015-08-28 Thread David Wolfskill 
On Wed, Aug 19, 2015 at 01:01:24PM -0700, David Wolfskill wrote:
> On Wed, Aug 19, 2015 at 12:25:38PM -0700, Adrian Chadd wrote:
> > ...  But we definitely ahe enough to put into a PR..
> > ...
> 
> Bug 202494 - Panic [page fault] in _ieee80211_crypto_delkey() 
> 
> 

Caught another one (panic) -- this time, after having run "wlandebug
+crypto".  Above-cited PR updated to reflect updated status & dubugging
info.


Peace,
david
-- 
David H. Wolfskill  da...@catwhisker.org
Those who would murder in the name of God or prophet are blasphemous cowards.

See http://www.catwhisker.org/~david/publickey.gpg for my public key.


pgpik1QWBpJwb.pgp
Description: PGP signature

Re: Panic [page fault] in _ieee80211_crypto_delkey(): stable/10/amd64 @r286878

2015-08-19 Thread David Wolfskill 
On Wed, Aug 19, 2015 at 12:25:38PM -0700, Adrian Chadd wrote:
> ...  But we definitely ahe enough to put into a PR..
> ...

Bug 202494 - Panic [page fault] in _ieee80211_crypto_delkey() 


Peace,
david
-- 
David H. Wolfskill  da...@catwhisker.org
Those who would murder in the name of God or prophet are blasphemous cowards.

See http://www.catwhisker.org/~david/publickey.gpg for my public key.


pgpsCB0osukdb.pgp
Description: PGP signature

Re: Panic [page fault] in _ieee80211_crypto_delkey(): stable/10/amd64 @r286878

2015-08-19 Thread Adrian Chadd 
hi,

you'll have to do some debugging. it looks like it's some kind of odd
race - line 461 is _ieee80211_crypto_delkey(); line 105 is
cipher_detach() and it blows up there.

Try "wlandebug +crypto" during your next boot and let's see what it
logs for the key.

If you can 'print *key' in kgdb on the core at some frame then we
should get some useful information.



-a


On 19 August 2015 at 09:07, David Wolfskill  wrote:
> On Tue, Aug 18, 2015 at 04:20:07PM -0700, David Wolfskill wrote:
>> I was minding my own business in a staff meeting this afternoon, and my
>> laptop rebooted; seems it got a panic.  I've copied the core.txt.0 file
>> to , along with a
>> verbose dmesg.boot from this morning and output of "pciconf -l -v".
>>
>> This was running:
>> FreeBSD localhost 10.2-STABLE FreeBSD 10.2-STABLE #122  
>> r286878M/286880:1002500: Tue Aug 18 04:06:33 PDT 2015 
>> r...@g1-252.catwhisker.org:/common/S1/obj/usr/src/sys/CANARY  amd64
>> 
>
> And this morning (just after I got in to work, and was trying (and
> trying) to get re-associated with the AP at work), I had another one.
>
> I've copied the resulting core.txt.1 over to
> http://www.cawhisker.org:~david/FreeBSD/stable_10/ as well; here are
> excerpts from a unidiff between core.txt.{0,1}:
>
> --- core.txt.0  2015-08-18 15:39:05.232251000 -0700
> +++ core.txt.1  2015-08-19 08:56:37.686238000 -0700
> @@ -1,8 +1,8 @@
> -localhost dumped core - see /var/crash/vmcore.0
> +localhost dumped core - see /var/crash/vmcore.1
>
> -Tue Aug 18 15:39:02 PDT 2015
> +Wed Aug 19 08:56:35 PDT 2015
>
> -FreeBSD localhost 10.2-STABLE FreeBSD 10.2-STABLE #122  
> r286878M/286880:1002500: Tue Aug 18 04:06:33 PDT 2015 
> r...@g1-252.catwhisker.org:/common/S1/obj/usr/src/sys/CANARY  amd64
> +FreeBSD localhost 10.2-STABLE FreeBSD 10.2-STABLE #123  
> r286912M/286918:1002500: Wed Aug 19 04:05:06 PDT 2015 
> r...@g1-252.catwhisker.org:/common/S1/obj/usr/src/sys/CANARY  amd64
>
>  panic: page fault
>
> @@ -16,7 +16,7 @@
>
>  Unread portion of the kernel message buffer:
>  panic: page fault
> -cpuid = 2
> +cpuid = 1
>  KDB: stack backtrace:
>  #0 0x80946e00 at kdb_backtrace+0x60
>  #1 0x8090a9e6 at vpanic+0x126
> @@ -34,8 +34,8 @@
>  #13 0x8095e9f0 at sys_ioctl+0x140
>  #14 0x80c84f97 at amd64_syscall+0x357
>  #15 0x80c6a49b at Xfast_syscall+0xfb
> -Uptime: 9h45m0s
> -Dumping 625 out of 8095 MB:..3%..11%..21%..31%..41%..52%..62%..72%..82%..93%
> +Uptime: 3h16m49s
> +Dumping 584 out of 8095 MB:..3%..11%..22%..31%..42%..53%..61%..72%..83%..91%
>
>  Reading symbols from /boot/kernel/geom_eli.ko.symbols...done.
>  Loaded symbols for /boot/kernel/geom_eli.ko.symbols
> @@ -81,32 +81,32 @@
>  at /usr/src/sys/kern/kern_shutdown.c:687
>  #4  0x80c8467b in trap_fatal (frame=,
>  eva=) at /usr/src/sys/amd64/amd64/trap.c:851
> -#5  0x80c8497d in trap_pfault (frame=0xfe060d88b510,
> +#5  0x80c8497d in trap_pfault (frame=0xfe060d5ea510,
>  usermode=) at /usr/src/sys/amd64/amd64/trap.c:674
> -#6  0x80c8401a in trap (frame=0xfe060d88b510)
> +#6  0x80c8401a in trap (frame=0xfe060d5ea510)
>  at /usr/src/sys/amd64/amd64/trap.c:440
>  #7  0x80c6a1b2 in calltrap ()
>  at /usr/src/sys/amd64/amd64/exception.S:236
>  #8  0x809f003a in _ieee80211_crypto_delkey ()
>  at /usr/src/sys/net80211/ieee80211_crypto.c:105
> -#9  0x809eff5e in ieee80211_crypto_delkey (vap=0xfe03d907,
> -key=0xfe03d9070800) at /usr/src/sys/net80211/ieee80211_crypto.c:461
> -#10 0x80a04d45 in ieee80211_ioctl_delkey (vap=0xfe03d907,
> +#9  0x809eff5e in ieee80211_crypto_delkey (vap=0xfe03dd31a000,
> +key=0xfe03dd31a800) at /usr/src/sys/net80211/ieee80211_crypto.c:461
> +#10 0x80a04d45 in ieee80211_ioctl_delkey (vap=0xfe03dd31a000,
>  ireq=)
>  at /usr/src/sys/net80211/ieee80211_ioctl.c:1252
>  #11 0x80a03bd2 in ieee80211_ioctl_set80211 ()
>  at /usr/src/sys/net80211/ieee80211_ioctl.c:2814
>  #12 0x80a2c323 in in_control (so=,
> -cmd=9214790412651315593, data=0xfe060d88bb80 "", ifp=0x3,
> +cmd=9214790412651315593, data=0xfe060d5eab80 "", ifp=0x3,
>  td=) at /usr/src/sys/netinet/in.c:308
> -#13 0x809cd57b in ifioctl (so=0xfe03d9070800, cmd=2149607914,
> -data=0xfe060d88b8e0 "wlan0", td=0xf80170abb940)
> +#13 0x809cd57b in ifioctl (so=0xfe03dd31a800, cmd=2149607914,
> +data=0xfe060d5ea8e0 "wlan0", td=0xf800098b5940)
>  at /usr/src/sys/net/if.c:2770
> -#14 0x8095ecf5 in kern_ioctl (td=0xf80170abb940,
> -fd=, com=18446741891212314624) at file.h:320
> -#15 0x8095e9f0 in sys_ioctl (td=0xf80170abb940,
> -uap=0xfe060d88ba40) at /usr/src/sys/kern/sys_generic.c:718
> -#16 0x80c84f97 in amd64_syscall (td=0xf80170abb940, traced=0)
>

Re: Panic [page fault] in _ieee80211_crypto_delkey(): stable/10/amd64 @r286878

2015-08-19 Thread David Wolfskill 
On Tue, Aug 18, 2015 at 04:20:07PM -0700, David Wolfskill wrote:
> I was minding my own business in a staff meeting this afternoon, and my
> laptop rebooted; seems it got a panic.  I've copied the core.txt.0 file
> to , along with a
> verbose dmesg.boot from this morning and output of "pciconf -l -v".
> 
> This was running:
> FreeBSD localhost 10.2-STABLE FreeBSD 10.2-STABLE #122  
> r286878M/286880:1002500: Tue Aug 18 04:06:33 PDT 2015 
> r...@g1-252.catwhisker.org:/common/S1/obj/usr/src/sys/CANARY  amd64
> 

And this morning (just after I got in to work, and was trying (and
trying) to get re-associated with the AP at work), I had another one.

I've copied the resulting core.txt.1 over to
http://www.cawhisker.org:~david/FreeBSD/stable_10/ as well; here are
excerpts from a unidiff between core.txt.{0,1}:

--- core.txt.0  2015-08-18 15:39:05.232251000 -0700
+++ core.txt.1  2015-08-19 08:56:37.686238000 -0700
@@ -1,8 +1,8 @@
-localhost dumped core - see /var/crash/vmcore.0
+localhost dumped core - see /var/crash/vmcore.1
 
-Tue Aug 18 15:39:02 PDT 2015
+Wed Aug 19 08:56:35 PDT 2015
 
-FreeBSD localhost 10.2-STABLE FreeBSD 10.2-STABLE #122  
r286878M/286880:1002500: Tue Aug 18 04:06:33 PDT 2015 
r...@g1-252.catwhisker.org:/common/S1/obj/usr/src/sys/CANARY  amd64
+FreeBSD localhost 10.2-STABLE FreeBSD 10.2-STABLE #123  
r286912M/286918:1002500: Wed Aug 19 04:05:06 PDT 2015 
r...@g1-252.catwhisker.org:/common/S1/obj/usr/src/sys/CANARY  amd64
 
 panic: page fault
 
@@ -16,7 +16,7 @@
 
 Unread portion of the kernel message buffer:
 panic: page fault
-cpuid = 2
+cpuid = 1
 KDB: stack backtrace:
 #0 0x80946e00 at kdb_backtrace+0x60
 #1 0x8090a9e6 at vpanic+0x126
@@ -34,8 +34,8 @@
 #13 0x8095e9f0 at sys_ioctl+0x140
 #14 0x80c84f97 at amd64_syscall+0x357
 #15 0x80c6a49b at Xfast_syscall+0xfb
-Uptime: 9h45m0s
-Dumping 625 out of 8095 MB:..3%..11%..21%..31%..41%..52%..62%..72%..82%..93%
+Uptime: 3h16m49s
+Dumping 584 out of 8095 MB:..3%..11%..22%..31%..42%..53%..61%..72%..83%..91%
 
 Reading symbols from /boot/kernel/geom_eli.ko.symbols...done.
 Loaded symbols for /boot/kernel/geom_eli.ko.symbols
@@ -81,32 +81,32 @@
 at /usr/src/sys/kern/kern_shutdown.c:687
 #4  0x80c8467b in trap_fatal (frame=, 
 eva=) at /usr/src/sys/amd64/amd64/trap.c:851
-#5  0x80c8497d in trap_pfault (frame=0xfe060d88b510, 
+#5  0x80c8497d in trap_pfault (frame=0xfe060d5ea510, 
 usermode=) at /usr/src/sys/amd64/amd64/trap.c:674
-#6  0x80c8401a in trap (frame=0xfe060d88b510)
+#6  0x80c8401a in trap (frame=0xfe060d5ea510)
 at /usr/src/sys/amd64/amd64/trap.c:440
 #7  0x80c6a1b2 in calltrap ()
 at /usr/src/sys/amd64/amd64/exception.S:236
 #8  0x809f003a in _ieee80211_crypto_delkey ()
 at /usr/src/sys/net80211/ieee80211_crypto.c:105
-#9  0x809eff5e in ieee80211_crypto_delkey (vap=0xfe03d907, 
-key=0xfe03d9070800) at /usr/src/sys/net80211/ieee80211_crypto.c:461
-#10 0x80a04d45 in ieee80211_ioctl_delkey (vap=0xfe03d907, 
+#9  0x809eff5e in ieee80211_crypto_delkey (vap=0xfe03dd31a000, 
+key=0xfe03dd31a800) at /usr/src/sys/net80211/ieee80211_crypto.c:461
+#10 0x80a04d45 in ieee80211_ioctl_delkey (vap=0xfe03dd31a000, 
 ireq=)
 at /usr/src/sys/net80211/ieee80211_ioctl.c:1252
 #11 0x80a03bd2 in ieee80211_ioctl_set80211 ()
 at /usr/src/sys/net80211/ieee80211_ioctl.c:2814
 #12 0x80a2c323 in in_control (so=, 
-cmd=9214790412651315593, data=0xfe060d88bb80 "", ifp=0x3, 
+cmd=9214790412651315593, data=0xfe060d5eab80 "", ifp=0x3, 
 td=) at /usr/src/sys/netinet/in.c:308
-#13 0x809cd57b in ifioctl (so=0xfe03d9070800, cmd=2149607914, 
-data=0xfe060d88b8e0 "wlan0", td=0xf80170abb940)
+#13 0x809cd57b in ifioctl (so=0xfe03dd31a800, cmd=2149607914, 
+data=0xfe060d5ea8e0 "wlan0", td=0xf800098b5940)
 at /usr/src/sys/net/if.c:2770
-#14 0x8095ecf5 in kern_ioctl (td=0xf80170abb940, 
-fd=, com=18446741891212314624) at file.h:320
-#15 0x8095e9f0 in sys_ioctl (td=0xf80170abb940, 
-uap=0xfe060d88ba40) at /usr/src/sys/kern/sys_generic.c:718
-#16 0x80c84f97 in amd64_syscall (td=0xf80170abb940, traced=0)
+#14 0x8095ecf5 in kern_ioctl (td=0xf800098b5940, 
+fd=, com=18446741891282216960) at file.h:320
+#15 0x8095e9f0 in sys_ioctl (td=0xf800098b5940, 
+uap=0xfe060d5eaa40) at /usr/src/sys/kern/sys_generic.c:718
+#16 0x80c84f97 in amd64_syscall (td=0xf800098b5940, traced=0)
 at subr_syscall.c:134
 #17 0x80c6a49b in Xfast_syscall ()
 at /usr/src/sys/amd64/amd64/exception.S:396
@@ -118,305 +118,301 @@
 



So it looks to me to be quite s

Panic [page fault] in _ieee80211_crypto_delkey(): stable/10/amd64 @r286878

2015-08-18 Thread David Wolfskill 
I was minding my own business in a staff meeting this afternoon, and my
laptop rebooted; seems it got a panic.  I've copied the core.txt.0 file
to , along with a
verbose dmesg.boot from this morning and output of "pciconf -l -v".

This was running:
FreeBSD localhost 10.2-STABLE FreeBSD 10.2-STABLE #122  
r286878M/286880:1002500: Tue Aug 18 04:06:33 PDT 2015 
r...@g1-252.catwhisker.org:/common/S1/obj/usr/src/sys/CANARY  amd64

Excerpts from core.txt.0:

panic: page fault
...
Unread portion of the kernel message buffer:
panic: page fault
cpuid = 2
KDB: stack backtrace:
#0 0x80946e00 at kdb_backtrace+0x60
#1 0x8090a9e6 at vpanic+0x126
#2 0x8090a8b3 at panic+0x43
#3 0x80c8467b at trap_fatal+0x36b
#4 0x80c8497d at trap_pfault+0x2ed
#5 0x80c8401a at trap+0x47a
#6 0x80c6a1b2 at calltrap+0x8
#7 0x809eff5e at ieee80211_crypto_delkey+0x1e
#8 0x80a04d45 at ieee80211_ioctl_delkey+0x65
#9 0x80a03bd2 at ieee80211_ioctl_set80211+0x572
#10 0x80a2c323 at in_control+0x203
#11 0x809cd57b at ifioctl+0x15eb
#12 0x8095ecf5 at kern_ioctl+0x255
#13 0x8095e9f0 at sys_ioctl+0x140
#14 0x80c84f97 at amd64_syscall+0x357
#15 0x80c6a49b at Xfast_syscall+0xfb
Uptime: 9h45m0s
...
Loaded symbols for /usr/local/modules/rtc.ko
#0  doadump (textdump=) at pcpu.h:219
219 pcpu.h: No such file or directory.
in pcpu.h
(kgdb) #0  doadump (textdump=) at pcpu.h:219
#1  0x8090a642 in kern_reboot (howto=260)
at /usr/src/sys/kern/kern_shutdown.c:451
#2  0x8090aa25 in vpanic (fmt=, 
ap=) at /usr/src/sys/kern/kern_shutdown.c:758
#3  0x8090a8b3 in panic (fmt=0x0)
at /usr/src/sys/kern/kern_shutdown.c:687
#4  0x80c8467b in trap_fatal (frame=, 
eva=) at /usr/src/sys/amd64/amd64/trap.c:851
#5  0x80c8497d in trap_pfault (frame=0xfe060d88b510, 
usermode=) at /usr/src/sys/amd64/amd64/trap.c:674
#6  0x80c8401a in trap (frame=0xfe060d88b510)
at /usr/src/sys/amd64/amd64/trap.c:440
#7  0x80c6a1b2 in calltrap ()
at /usr/src/sys/amd64/amd64/exception.S:236
#8  0x809f003a in _ieee80211_crypto_delkey ()
at /usr/src/sys/net80211/ieee80211_crypto.c:105
#9  0x809eff5e in ieee80211_crypto_delkey (vap=0xfe03d907, 
key=0xfe03d9070800) at /usr/src/sys/net80211/ieee80211_crypto.c:461
#10 0x80a04d45 in ieee80211_ioctl_delkey (vap=0xfe03d907, 
ireq=)
at /usr/src/sys/net80211/ieee80211_ioctl.c:1252
#11 0x80a03bd2 in ieee80211_ioctl_set80211 ()
at /usr/src/sys/net80211/ieee80211_ioctl.c:2814
#12 0x80a2c323 in in_control (so=, 
cmd=9214790412651315593, data=0xfe060d88bb80 "", ifp=0x3, 
td=) at /usr/src/sys/netinet/in.c:308
#13 0x809cd57b in ifioctl (so=0xfe03d9070800, cmd=2149607914, 
data=0xfe060d88b8e0 "wlan0", td=0xf80170abb940)
at /usr/src/sys/net/if.c:2770
#14 0x8095ecf5 in kern_ioctl (td=0xf80170abb940, 
fd=, com=18446741891212314624) at file.h:320
#15 0x8095e9f0 in sys_ioctl (td=0xf80170abb940, 
uap=0xfe060d88ba40) at /usr/src/sys/kern/sys_generic.c:718
#16 0x80c84f97 in amd64_syscall (td=0xf80170abb940, traced=0)
at subr_syscall.c:134
#17 0x80c6a49b in Xfast_syscall ()
at /usr/src/sys/amd64/amd64/exception.S:396
#18 0x0008012a2f9a in ?? ()
Previous frame inner to this frame (corrupt stack?)
Current language:  auto; currently minimal
(kgdb) 


Physical 802.11 hardware is iwn(4).

I can copy the vmcore.0 file itself after I get home -- it's ~625MB,
and I'd rather not try to get that through over a WAN before I need
to catch the shuttle to get home. :-}

Peace,
david
-- 
David H. Wolfskill  da...@catwhisker.org
Those who would murder in the name of God or prophet are blasphemous cowards.

See http://www.catwhisker.org/~david/publickey.gpg for my public key.


pgpWb4UItbSf2.pgp
Description: PGP signature