Re: OpenBSD crash with pfsync
Hi Martin,
i'm using IPv6 but not on the syncdev interface, on all my other
interfaces (CARP & OSPF).
Here is the syncdev config:
inet 10.XX.XX.2 255.255.255.224 NONE
vlan 30 vlandev trunk1
and the pfsync config:
up
defer
syncdev vlan30
To precise how the bug happens, i have removed some other interfaces
before restarting it (a vlan57 and carp57 interfaces, childs of trunk1),
and also reloaded (without removing) the trunk1 interface.
Before do the "ifconfig pfsync0 up" i also saw that pfsync0 was down and
there is no syncdev (strange because the interface wasn't modified since
boot).
--
Best regards,
Loïc BLOT, Engineering
UNIX Systems, Security and Network Engineer
http://www.unix-experience.fr
Le jeudi 22 mai 2014 à 12:49 +0200, Martin Pieuchot a écrit :
> Hello Loïc,
>
> On 22/05/14(Thu) 11:11, Loïc Blot wrote:
> > Hi,
> > today i upgraded my primary router from OpenBSD 5.4 to OpenBSD 5.5 (i
> > follow the process described here:
> > http://www.openbsd.org/faq/upgrade55.html and this is my 5th upgrade
> > from 5.4 to 5.5 since the release).
> >
> > After rebooting and doing the sysmerge without network copper cables, i
> > rebooted and set my carp & pfsync interfaces to down before plugging the
> > cables. At this time, the router was in a CARP INIT mode, no problem.
> >
> > Note: all the traffic was redirected to my second OpenBSD router (which
> > was upgraded at release time) 3 days ago, this routers hasn't any
> > problem and have exactly the same hardware and software configuration
> > (except some IPs).
> >
> > After finishing the upgrading process, i have incremented the carpdemote
> > counter to force CARP to be in BACKUP mode and then i have set all my
> > carp interface to up. Then ifaces were in BACKUP mode. No problem since
> > there all was right, all services works fine, etc.
> >
> > The last step i do is to do a "ifconfig pfsync0 up" and then a OpenBSD
> > crash.
>
> Thanks for the report. How is your pfsync0 interface configured? Same
> thing for its syncdev interface.
>
> > ddb{1}> trace
> > ip_output() at ip_output+0xdd9
>
> This is a use after free in IFP_TO_IA(), somehow one of the ifa pointer
> on the list of your syncdev is now pointing to memory that has been freed:
>
> > rcx 0xdeadbeefdeadbeef
>
> This should be pointing to `ifa_addr` which makes me believe that you
> are hitting a reference counting bug because it should not be possible
> to free an ifa which is still on the list of an ifp.
>
> Can you easily reproduce this bug? Do you use IPv6? If yes does
> setting "-inet6" on the syncdev interface helps?
>
> Martin
Re: OpenBSD crash with pfsync
Hello Loïc,
On 22/05/14(Thu) 11:11, Loïc Blot wrote:
> Hi,
> today i upgraded my primary router from OpenBSD 5.4 to OpenBSD 5.5 (i
> follow the process described here:
> http://www.openbsd.org/faq/upgrade55.html and this is my 5th upgrade
> from 5.4 to 5.5 since the release).
>
> After rebooting and doing the sysmerge without network copper cables, i
> rebooted and set my carp & pfsync interfaces to down before plugging the
> cables. At this time, the router was in a CARP INIT mode, no problem.
>
> Note: all the traffic was redirected to my second OpenBSD router (which
> was upgraded at release time) 3 days ago, this routers hasn't any
> problem and have exactly the same hardware and software configuration
> (except some IPs).
>
> After finishing the upgrading process, i have incremented the carpdemote
> counter to force CARP to be in BACKUP mode and then i have set all my
> carp interface to up. Then ifaces were in BACKUP mode. No problem since
> there all was right, all services works fine, etc.
>
> The last step i do is to do a "ifconfig pfsync0 up" and then a OpenBSD
> crash.
Thanks for the report. How is your pfsync0 interface configured? Same
thing for its syncdev interface.
> ddb{1}> trace
> ip_output() at ip_output+0xdd9
This is a use after free in IFP_TO_IA(), somehow one of the ifa pointer
on the list of your syncdev is now pointing to memory that has been freed:
> rcx 0xdeadbeefdeadbeef
This should be pointing to `ifa_addr` which makes me believe that you
are hitting a reference counting bug because it should not be possible
to free an ifa which is still on the list of an ifp.
Can you easily reproduce this bug? Do you use IPv6? If yes does
setting "-inet6" on the syncdev interface helps?
Martin
OpenBSD crash with pfsync
Hi,
today i upgraded my primary router from OpenBSD 5.4 to OpenBSD 5.5 (i
follow the process described here:
http://www.openbsd.org/faq/upgrade55.html and this is my 5th upgrade
from 5.4 to 5.5 since the release).
After rebooting and doing the sysmerge without network copper cables, i
rebooted and set my carp & pfsync interfaces to down before plugging the
cables. At this time, the router was in a CARP INIT mode, no problem.
Note: all the traffic was redirected to my second OpenBSD router (which
was upgraded at release time) 3 days ago, this routers hasn't any
problem and have exactly the same hardware and software configuration
(except some IPs).
After finishing the upgrading process, i have incremented the carpdemote
counter to force CARP to be in BACKUP mode and then i have set all my
carp interface to up. Then ifaces were in BACKUP mode. No problem since
there all was right, all services works fine, etc.
The last step i do is to do a "ifconfig pfsync0 up" and then a OpenBSD
crash.
Following you can found the trace, the ps and the registers.
I haven't seen any errata for pfsync, then i didn't patched the kernel,
it's the GENERIC BSD.MP.
Is there any workaround ? A kernel patch to apply ?
Thanks in advance
=
ddb{1}> trace
ip_output() at ip_output+0xdd9
pfsync_sendout() at pfsync_sendout+0x489
netintr() at netintr+0xed
softintr_dispatch() at softintr_dispatch+0x5d
Xsoftnet() at Xsoftnet+0x2d
--- interrupt ---
end of kernel
end trace frame: 0x80206910, count: -5
0x80943000:
PID PPID PGRPUID S FLAGS WAIT COMMAND
*18270 3703 18270 0 7 0x3ifconfig
3769 19350 19350 70 30x90 selectnamed
19350 1 19350 0 30x90 netio named
20234617 19369515 30x82 netio squidGuard
14044617 19369515 30x82 netio squidGuard
16252617 19369515 30x82 netio squidGuard
7350617 19369515 30x82 netio squidGuard
7207617 19369515 30x82 netio squidGuard
4662617 19369515 30x82 netio squidGuard
20849617 19369515 30x82 netio squidGuard
15847617 19369515 30x82 netio squidGuard
24271617 19369515 30x82 netio squidGuard
9469617 19369515 30x82 netio squidGuard
26465617 19369515 30x82 netio squidGuard
25889617 19369515 30x82 netio squidGuard
18394617 19369515 30x82 netio squidGuard
24590617 19369515 30x82 netio squidGuard
26389617 19369515 30x82 netio squidGuard
9456617 19369515 30x82 netio squidGuard
11447617 19369515 30x82 netio squidGuard
13398617 19369515 30x82 netio squidGuard
23571617 19369515 30x82 netio squidGuard
13704617 19369515 30x82 netio squidGuard
22507617 19369515 30x82 netio squidGuard
4039617 19369515 30x82 netio squidGuard
22683617 19369515 30x82 netio squidGuard
9501617 19369515 30x82 netio squidGuard
23298617 19369515 30x82 netio squidGuard
966617 19369515 30x82 netio squidGuard
31585617 19369515 30x82 netio squidGuard
3979617 19369515 30x82 netio squidGuard
23135617 19369515 30x82 netio squidGuard
2705617 19369515 30x82 netio squidGuard
414617 19369515 30x82 netio squidGuard
26606617 19369515 30x82 netio squidGuard
31755617 19369515 30x82 netio squidGuard
315617 19369515 30x82 netio squidGuard
649617 19369515 30x82 netio squidGuard
26506617 19369515 30x82 netio squidGuard
7278617 19369515 30x82 netio squidGuard
21806617 19369515 30x82 netio squidGuard
32275617 19369515 30x82 netio squidGuard
28676617 19369515 30x82 netio squidGuard
2160617 19369515 30x82 netio squidGuard
30698617 19369515 30x82 netio squidGuard
3057617 19369515 30x82 netio squidGuard
17056617 19369515 30x82 netio squidGuard
17734617 19369515 30x82 netio squidGuard
22101617 19369515 30x82
