Changing the mtu wasn't enough. To make it works i had to disable vxlan offload
features:
```
ifconfig mce0 -vxlanhwcsum -vxlanhwtso
```
Then suddenly all started to work. The card is the following:
```
mlx5_core1@pci0:17:0:1: class=0x020000 rev=0x00 hdr=0x00 vendor=0x15b3
device=0x1015 subvendor=0x1590 subdevice=0x00d4
vendor = 'Mellanox Technologies'
device = 'MT27710 Family [ConnectX-4 Lx]'
class = network
subclass = ethernet
PCI-e errors = Correctable Error Detected
Unsupported Request Detected
Corrected = Advisory Non-Fatal Error
```
Which is odd. Wonder why it's needed. Thoughts?
Benoît
On Tuesday, December 5th, 2023 at 13:11, Kristof Provost <k...@freebsd.org>
wrote:
> On 4 Dec 2023, at 11:25, Benoit Chesneau wrote:
>
> > Hi,
> >
> > Is IPv6 underlay fully supported with FreebBSD ? I have created the a
> > tunnel and associated an Ipv6 address to each side. I'm able to ping
> > between each devicesl. But when I want to curl from the remote side it
> > timeout. Locally on the remote side it is OK. Is this expected ? Should I
> > rather create a bridge with vxlan as a member and bind nginx to it ?
> >
> > ```
> > $ ifconfig vxlan0 create vxlanid 108 vxlanlocal AAAA:BBBB:110b:102::100
> > vxlanremote AAAA:BBBB:110b:102::12
> > $ ifconfig vxlan0
> > vxlan0: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP>
> > metric 0 mtu 1430
> > options=80020<JUMBO_MTU,LINKSTATE>
> > ether 58:9c:fc:10:ff:eb
> > groups: vxlan
> > vxlan vni 108 local [AAAA:BBBB:102::100]:4789 remote
> > [AAAA:BBBB:110b:102::12]:4789
> > media: Ethernet autoselect (autoselect <full-duplex>)
> > status: active nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
> > $ ifconfig vxlan0 inet6 AAAA:BBBB:110b:300::1/64
> > ````
> >
> > Ping from remote is ok:
> > `ifconfig vxlan0 vxlan0:
> > flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu
> > 1430
> > options=680323<RXCSUM,TXCSUM,JUMBO_MTU,TSO4,TSO6,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
> > ether 58:9c:fc:10:df:1f inet6 fe80::5a9c:fcff:fe10:df1f%vxlan0 prefixlen
> > 64 scopeid 0xf inet6 AAAA:BBBB:110b:300::2 prefixlen 64 groups: vxlan vxlan
> > vni 108 local [AAAA:BBBB110b:102::12]:4789 remote
> > [AAAA:BBBB:110b:102::100]:4789 media: Ethernet autoselect (autoselect
> > <full-duplex>) status: active nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> $
> > ping6 AAAA:BBBB:110b:300::1 PING6(56=40+8+8 bytes) AAAA:BBBB:110b:300::2
> > --> AAAA:BBBB::110b:300::1 16 bytes from 2a0e:e701:110b:300::1, icmp_seq=0
> > hlim=64 time=0.071 ms 16 bytes from 2a0e:e701:110b:300::1, icmp_seq=1
> > hlim=64 time=0.078 ms 16 bytes from 2a0e:e701:110b:300::1, icmp_seq=2
> > hlim=64 time=0.076 ms 16 bytes from 2a0e:e701:110b:300::1, icmp_seq=3
> > hlim=64 time=0.104 ms 16 bytes from 2a0e:e701:110b:300::1, icmp_seq=4
> > hlim=64 time=0.077 ms^C`
> >
> > But when I run `curl -6 -v 'http://[AAAA:BBBB:110b:300::1]'` it timeout.
>
> That smells like an MTU issue. Make sure you’re not blocking ICMPv6, and
> check with larger ping messages.
>
> Best regards,
> Kristof
