Re: Weird nd6_debug:nd6_ns_input logging on neighbor sol

2024-08-12 Thread Rémi Laurent

We had another set of issues, this time with OpenBSD 7.5, fully patched

Same weird logging about nd6_ns_input: NS packet from non-neighbor

Again with an extra few bits (16-31) in the logged source link-local 
address - we've discovered those bits are actually the ifindex of the 
receiving interface



It seems this is totally bugged (maybe also linked to ipv6 on CARP 
interfaces), so we just ended up with a hack, forcing the first unicast 
address to be used as source for any "interface bound" multicast trafic, 
instead of the link-local address


For example with something like

$ route change -iface -inet6 -net ff02::%vio2/32 -ifp vio2 
2a02:6f00::1234:4321::1


The "generic" way of applying this hack on interface and via hostname.if 
config file could be the following


!ifconfig \$if | awk -viface=\$if '$1 == "inet6" && $2 !~ /^fe80::/ { 
system("route change -iface -inet6 -net ff02::%"iface"/32 -ifp "iface" 
"$2); exit(0); }'


Note that this hack doesn't need to be applied on carp interfaces as 
they don't have either link-local address nor this ff02::%ifname/32 
route created



So far we applied this on two systems (OpenBSD 6.8 and 7.5) that were 
consistently exhibiting the issue and it was fixed immediately after


It looks like this is not the first time this issue was discussed and 
this CVE-2008-2476 mitigation code was checked


https://misc.openbsd.narkive.com/3KdNDcEM/openbsd-ignoring-rfc-compliant-ipv6-neighbor-solicitation

FreeBSD also, but at least they implemented a sysctl flag to disable it

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=263288


Hope this helps other in the same situation


On 16/05/2024 15:33, Rémi Laurent wrote:

Greetings,

we're having at least two different weird ndp/icmp6 related behaviours 
we would

like to share

The setup is quite simple, we're trying to ping6 from one OpenBSD 6.8 to
another, sometimes, without any clear reason, the host sending the 
request will

start to use its link-local address in the icmp neighbor solicitation

See below a NOK scenario with the solicitation dropped by the 
requested system
The OK scenario may come back after a while, where the unicast address 
is again
in use in the the neighbor solicitation source address and is properly 
answered


A simple way we found to reproduce this on demand is to run a ndp 
delete and

then ping in a loop
  $ while (:); do ndp -d 2a02:6f00:c0:30::10; ping6 -c2 
2a02:6f00:c0:30::10; done

Usually after 5 to 10 attempts the issue shows up

NOK pcap on requested host (trafic goes through but ignored by 
target/requested host)
# 16:50:06.340898 00:15:17:dd:60:fa 33:33:ff:00:00:10 86dd 86: 
fe80::215:17ff:fedd:60fa > ff02::1:ff00:10: icmp6: neighbor sol: who 
has 2a02:6f00:c0:30::10
# 16:50:07.356429 00:15:17:dd:60:fa 33:33:ff:00:00:10 86dd 86: 
fe80::215:17ff:fedd:60fa > ff02::1:ff00:10: icmp6: neighbor sol: who 
has 2a02:6f00:c0:30::10


OK same pcap with unicast address of source instead of link local, 
properly answered
# 16:57:10.308265 00:15:17:dd:60:fa 33:33:ff:00:00:10 86dd 86: 
2a02:6f00:c0:30::3 > ff02::1:ff00:10: icmp6: neighbor sol: who has 
2a02:6f00:c0:30::10
# 16:57:10.308512 00:1e:67:15:18:ba 00:15:17:dd:60:fa 86dd 86: 
2a02:6f00:c0:30::11 > 2a02:6f00:c0:30::3: icmp6: neighbor adv: tgt is 
2a02:6f00:c0:30::10


We then enabled nd6_debug with sysctl net.inet6.icmp6.nd6_debug=1, ran 
the test

again and below are the pcap and dmesg output
Notice how the src and dst address in nd6_ns_input are altered

We started to wonder what is at fault, the 'NS packet from 
non-neighbor' check

or the ipv6 fields being altered higher up in the chain

On the live system - OpenBSD 6.8 (yes, we plan to update)
# pcap - correct source link-local address
# 13:42:54.252709 00:15:17:dd:60:fa 33:33:ff:00:00:10 86dd 86: 
fe80::215:17ff:fedd:60fa > ff02::1:ff00:10: icmp6: neighbor sol: who 
has 2a02:6f00:c0:30::10


# dmesg with altered src and dst addresses (extra 9 on bits 16-31)
# May 16 13:42:54 vpn1 /bsd: nd6_ns_input: NS packet from non-neighbor
# May 16 13:42:54 vpn1 /bsd: nd6_ns_input: src=fe80:9::215:17ff:fedd:60fa
# May 16 13:42:54 vpn1 /bsd: nd6_ns_input: dst=ff02:9::1:ff00:10
# May 16 13:42:54 vpn1 /bsd: nd6_ns_input: tgt=2a02:6f00:c0:30::10


The same attempt on a 7.3 system in a "lab" are working correctly:

1. With a "default" configuration, the src address in the neighbor 
solicitation

is the unicast address and not the link local address, 'NS packet from
non-neighbor' is not triggered and ndp works correctly, icmp6 
request/reply

then goes through without issues

2. When explicitly crafting a packet with the link local address as 
source,
the remote system properly reply to neighbor solicitation and icmp 
req/rep goes
as well too which would indicate that 'NS packet from non-neighbor' 
allows link

local address as source

3. The interesting part is when we explicitly craft a neighbor 
solicitation

packet with a "invalid" source link local address, the 'NS packet from
non-neigh

Weird nd6_debug:nd6_ns_input logging on neighbor sol

2024-05-16 Thread Rémi Laurent

Greetings,

we're having at least two different weird ndp/icmp6 related behaviours we would
like to share

The setup is quite simple, we're trying to ping6 from one OpenBSD 6.8 to
another, sometimes, without any clear reason, the host sending the request will
start to use its link-local address in the icmp neighbor solicitation

See below a NOK scenario with the solicitation dropped by the requested system
The OK scenario may come back after a while, where the unicast address is again
in use in the the neighbor solicitation source address and is properly answered

A simple way we found to reproduce this on demand is to run a ndp delete and
then ping in a loop
  $ while (:); do ndp -d 2a02:6f00:c0:30::10; ping6 -c2 2a02:6f00:c0:30::10; 
done
Usually after 5 to 10 attempts the issue shows up

NOK pcap on requested host (trafic goes through but ignored by target/requested 
host)
# 16:50:06.340898 00:15:17:dd:60:fa 33:33:ff:00:00:10 86dd 86: 
fe80::215:17ff:fedd:60fa > ff02::1:ff00:10: icmp6: neighbor sol: who has 
2a02:6f00:c0:30::10
# 16:50:07.356429 00:15:17:dd:60:fa 33:33:ff:00:00:10 86dd 86: 
fe80::215:17ff:fedd:60fa > ff02::1:ff00:10: icmp6: neighbor sol: who has 
2a02:6f00:c0:30::10

OK same pcap with unicast address of source instead of link local, properly 
answered
# 16:57:10.308265 00:15:17:dd:60:fa 33:33:ff:00:00:10 86dd 86: 2a02:6f00:c0:30::3 
> ff02::1:ff00:10: icmp6: neighbor sol: who has 2a02:6f00:c0:30::10
# 16:57:10.308512 00:1e:67:15:18:ba 00:15:17:dd:60:fa 86dd 86: 2a02:6f00:c0:30::11 
> 2a02:6f00:c0:30::3: icmp6: neighbor adv: tgt is 2a02:6f00:c0:30::10

We then enabled nd6_debug with sysctl net.inet6.icmp6.nd6_debug=1, ran the test
again and below are the pcap and dmesg output
Notice how the src and dst address in nd6_ns_input are altered

We started to wonder what is at fault, the 'NS packet from non-neighbor' check
or the ipv6 fields being altered higher up in the chain

On the live system - OpenBSD 6.8 (yes, we plan to update)
# pcap - correct source link-local address
# 13:42:54.252709 00:15:17:dd:60:fa 33:33:ff:00:00:10 86dd 86: 
fe80::215:17ff:fedd:60fa > ff02::1:ff00:10: icmp6: neighbor sol: who has 
2a02:6f00:c0:30::10

# dmesg with altered src and dst addresses (extra 9 on bits 16-31)
# May 16 13:42:54 vpn1 /bsd: nd6_ns_input: NS packet from non-neighbor
# May 16 13:42:54 vpn1 /bsd: nd6_ns_input: src=fe80:9::215:17ff:fedd:60fa
# May 16 13:42:54 vpn1 /bsd: nd6_ns_input: dst=ff02:9::1:ff00:10
# May 16 13:42:54 vpn1 /bsd: nd6_ns_input: tgt=2a02:6f00:c0:30::10


The same attempt on a 7.3 system in a "lab" are working correctly:

1. With a "default" configuration, the src address in the neighbor solicitation
is the unicast address and not the link local address, 'NS packet from
non-neighbor' is not triggered and ndp works correctly, icmp6 request/reply
then goes through without issues

2. When explicitly crafting a packet with the link local address as source,
the remote system properly reply to neighbor solicitation and icmp req/rep goes
as well too which would indicate that 'NS packet from non-neighbor' allows link
local address as source

3. The interesting part is when we explicitly craft a neighbor solicitation
packet with a "invalid" source link local address, the 'NS packet from
non-neighbor' triggers and shows up in dmesg, but again, with an altered bit

- "lab" testing - 7.3 
# pcap - crafted incorrect source link-local address
# 13:58:42.675849 c8:8a:9a:a3:ea:26 33:33:ff:00:00:01 86dd 86: 
fe80::9:5054:ff:fe28:e372 > ff02::1:ff00:1: icmp6: neighbor sol: who has 
2a02:6f00:::1

# dmesg with altered src and dst address (extra 1 on bits 16-31 on top of 
crafted 9)
# May 16 13:58:42 obsd68 /bsd: nd6_ns_input: NS packet from non-neighbor
# May 16 13:58:42 obsd68 /bsd: nd6_ns_input: src=fe80:1:0:9:5054:ff:fe28:e372
# May 16 13:58:42 obsd68 /bsd: nd6_ns_input: dst=ff02:1::1:ff00:1
# May 16 13:58:42 obsd68 /bsd: nd6_ns_input: tgt=2a02:6f00:::1

Questions are then:
- why would a system change between link-local and unicast address in the
  source address of neighbor solicitations it sends (although I'm painfully
  aware 6.8 is quite old)
- is the nd6_ns_input logging having issue with display of dst/src addresses ?
- and/or the 'NS packet from non-neighbor' mechanism known to sometime discard
  perfectly valid neighbor solicitation with link-local address as source ?

Thank you,
and let me know if you would need more details on this