Bug#1054642: Failing ARP relay from external -> Linux bridge -> veth port --> NS veth port
Hi Daniel,
hope you are good, had peaceful Christmas time, entering yet better NY 2024
hope so... sorry for overlooking this, even wanted to respond early December,
then got delayed again.. Now I do so as its still interesting to me!
1) yes, my sole quick method was "ip nei" command to confirm the ARP passthrough
2) no firewall at all, plain Debian installation
3) you will not believe --> but before Xmas and now, it all works and MAC is
passed e2e. That's so pitty. Only change I made was my underlay change of
vSwitch uplink to another port... because I re-considered my overall lab setup,
yet it hardly could improve this as the external MAC made it to external (VLAN)
iface of the bridge, before/. Anyhow, possibly I understand the "bridge fbd"
only shows learned MACs on given interface (my VLAN199) and is not supposed to
attribute it to all others all way up to NS, like I attempted to guess..
Finally, either this of MACVLAN setup (where I found this), I have new finding
which I don’t like as it creates a hell of duplicate traffic into network. The
problem is, that either VETH or MACVLAN-configured IP host's VM duplicates
incoming packets on its receiving port, connected to vSphere vSwitch, which in
turn just dully floods it to uplinks, where my Wireshark sniffer sees it. This
is how I discovered that.
I prepared this diagram for you to see and tell.
https://docs.google.com/document/d/1mNkZswDSG_OjLnsgXJvIX2tUGSEebcZf720eS29eFCA/edit?usp=sharing
BR, all the best wishes in NY2024!
Peter
-Original Message-
From: Daniel Gröber
Sent: nedeľa 3. decembra 2023 11:09
To: GASPAROVIC Peter OBS/MKT
Cc: 1054...@bugs.debian.org
Subject: Re: Bug#1054642: Failing ARP relay from external -> Linux bridge ->
veth port --> NS veth port
Hi Peter,
> So now we move to VLAN level?
Yeah, but I'm still waiting for the answers to my questions from two emails
ago:
> I'd be happy to still track this bug down but I need you to
> investigate the behaviour in your environment. If you've torn down the
> lab already we can also just call it quits.
>
> If you do want to continue some questions are still pending, see quoted below.
>
> > On Mon, Oct 30, 2023 at 07:25:38PM +, peter.gasparo...@orange.com wrote:
> > > 1) As was reported, foreign external world MAC@ does not pass into
> > > network namespace, just external border point "vlan199"
> >
> > How did you check this?
> >
> > > 2) now collecting data for you, honestly I don’t see external mac
> > > address on "inet-br" object, so my previous statement was incorrect..
> > > {ossibly I might mixed this up with another "labinet-br" (working
> > > in its limited
> > > scope) which is IP-defined, while "inet-br" in question is not.
> > >
> > > 3) so question is, if the MACs learnt via vlan199 are supposed to
> > > be paired (displayed) with "inet-br" object and all way up into NS
> >
> > I want to make sure we're on the same page, how do you check if the
> > MAC is reaching into the NS? I assume using `ip neigh`? I'd have a
> > look at tcpdump this will tell you whether ARP is even reaching the NS or
> > not.
> >
> > Something simple like
> >
> > $ tcpdump -enli $IFACE 'arp or icmp'
> >
> > optionally you can filter by MAC (`ether host` in pcap-filter speak):
> >
> > $ tcpdump -enli $IFACE ('arp or icmp) and ether host
> > aa:00:00:00:00:01
> >
> > Oh and one last thing: please double and tripple check that a firewall
> > isn't interfering.
--Daniel
Ce message et ses pieces jointes peuvent contenir des informations
confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce
message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages
electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou
falsifie. Merci.
This message and its attachments may contain confidential or privileged
information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete
this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been
modified, changed or falsified.
Thank you.
Bug#1054642: Failing ARP relay from external -> Linux bridge -> veth port --> NS veth port
Hi Peter,
> So now we move to VLAN level?
Yeah, but I'm still waiting for the answers to my questions from two emails
ago:
> I'd be happy to still track this bug down but I need you to investigate
> the behaviour in your environment. If you've torn down the lab already we
> can also just call it quits.
>
> If you do want to continue some questions are still pending, see quoted below.
>
> > On Mon, Oct 30, 2023 at 07:25:38PM +, peter.gasparo...@orange.com wrote:
> > > 1) As was reported, foreign external world MAC@ does not pass into
> > > network namespace, just external border point "vlan199"
> >
> > How did you check this?
> >
> > > 2) now collecting data for you, honestly I don’t see external mac
> > > address on "inet-br" object, so my previous statement was incorrect..
> > > {ossibly I might mixed this up with another "labinet-br" (working in
> > > its limited
> > > scope) which is IP-defined, while "inet-br" in question is not.
> > >
> > > 3) so question is, if the MACs learnt via vlan199 are supposed to be
> > > paired (displayed) with "inet-br" object and all way up into NS
> >
> > I want to make sure we're on the same page, how do you check if the MAC
> > is reaching into the NS? I assume using `ip neigh`? I'd have a look at
> > tcpdump this will tell you whether ARP is even reaching the NS or not.
> >
> > Something simple like
> >
> > $ tcpdump -enli $IFACE 'arp or icmp'
> >
> > optionally you can filter by MAC (`ether host` in pcap-filter speak):
> >
> > $ tcpdump -enli $IFACE ('arp or icmp) and ether host
> > aa:00:00:00:00:01
> >
> > Oh and one last thing: please double and tripple check that a firewall
> > isn't interfering.
--Daniel
signature.asc
Description: PGP signature
Bug#1054642: Failing ARP relay from external -> Linux bridge -> veth port --> NS veth port
Hi Daniel,
of course we can steadily move on, no problem. So now we move to VLAN level?
BR
Peter
-Original Message-
From: Daniel Gröber
Sent: sobota 18. novembra 2023 3:43
To: GASPAROVIC Peter OBS/MKT
Cc: 1054...@bugs.debian.org
Subject: Re: Bug#1054642: Failing ARP relay from external -> Linux bridge ->
veth port --> NS veth port
Hi Peter,
On Mon, Nov 13, 2023 at 09:40:46AM +, peter.gasparo...@orange.com wrote:
> In the meantime, I was stubborn to find a solution to what I need in
> order to progress and MACVLAN tech actually delivered it (private mode
> enough),
I used to love macvlan too but now I do L3 instead ;P
> something newer than VETH tech what I could read about, and it's just
> perfect avoiding bridge itself. So no problem to cooperate on this
> fix, I will be glad, just it can get lower priority on your side if
> you even attributed it some
I'd be happy to still track this bug down but I need you to investigate the
behaviour in your environment. If you've torn down the lab already we can also
just call it quits.
If you do want to continue some questions are still pending, see quoted below.
> On Mon, Oct 30, 2023 at 07:25:38PM +, peter.gasparo...@orange.com wrote:
> > 1) As was reported, foreign external world MAC@ does not pass into
> > network namespace, just external border point "vlan199"
>
> How did you check this?
>
> > 2) now collecting data for you, honestly I don’t see external mac
> > address on "inet-br" object, so my previous statement was incorrect..
> > {ossibly I might mixed this up with another "labinet-br" (working in
> > its limited
> > scope) which is IP-defined, while "inet-br" in question is not.
> >
> > 3) so question is, if the MACs learnt via vlan199 are supposed to be
> > paired (displayed) with "inet-br" object and all way up into NS
>
> I want to make sure we're on the same page, how do you check if the MAC is
> reaching into the NS? I assume using `ip neigh`? I'd have a look at tcpdump
> this will tell you whether ARP is even reaching the NS or not.
>
> Something simple like
>
> $ tcpdump -enli $IFACE 'arp or icmp'
>
> optionally you can filter by MAC (`ether host` in pcap-filter speak):
>
> $ tcpdump -enli $IFACE ('arp or icmp) and ether host
> aa:00:00:00:00:01
>
> Oh and one last thing: please double and tripple check that a firewall isn't
> interfering.
--Daniel
Ce message et ses pieces jointes peuvent contenir des informations
confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce
message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages
electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou
falsifie. Merci.
This message and its attachments may contain confidential or privileged
information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete
this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been
modified, changed or falsified.
Thank you.
Bug#1054642: Failing ARP relay from external -> Linux bridge -> veth port --> NS veth port
Hi Peter,
On Mon, Nov 13, 2023 at 09:40:46AM +, peter.gasparo...@orange.com wrote:
> In the meantime, I was stubborn to find a solution to what I need in
> order to progress and MACVLAN tech actually delivered it (private mode
> enough),
I used to love macvlan too but now I do L3 instead ;P
> something newer than VETH tech what I could read about, and it's
> just perfect avoiding bridge itself. So no problem to cooperate on this
> fix, I will be glad, just it can get lower priority on your side if you
> even attributed it some
I'd be happy to still track this bug down but I need you to investigate the
behaviour in your environment. If you've torn down the lab already we can
also just call it quits.
If you do want to continue some questions are still pending, see quoted below.
> On Mon, Oct 30, 2023 at 07:25:38PM +, peter.gasparo...@orange.com wrote:
> > 1) As was reported, foreign external world MAC@ does not pass into
> > network namespace, just external border point "vlan199"
>
> How did you check this?
>
> > 2) now collecting data for you, honestly I don’t see external mac
> > address on "inet-br" object, so my previous statement was incorrect..
> > {ossibly I might mixed this up with another "labinet-br" (working in
> > its limited
> > scope) which is IP-defined, while "inet-br" in question is not.
> >
> > 3) so question is, if the MACs learnt via vlan199 are supposed to be
> > paired (displayed) with "inet-br" object and all way up into NS
>
> I want to make sure we're on the same page, how do you check if the MAC is
> reaching into the NS? I assume using `ip neigh`? I'd have a look at tcpdump
> this will tell you whether ARP is even reaching the NS or not.
>
> Something simple like
>
> $ tcpdump -enli $IFACE 'arp or icmp'
>
> optionally you can filter by MAC (`ether host` in pcap-filter speak):
>
> $ tcpdump -enli $IFACE ('arp or icmp) and ether host aa:00:00:00:00:01
>
> Oh and one last thing: please double and tripple check that a firewall isn't
> interfering.
--Daniel
signature.asc
Description: PGP signature
Bug#1054642: Failing ARP relay from external -> Linux bridge -> veth port --> NS veth port
Hi Daniel,
Thank you honestly for you time to look at this and cooperation.. Good decision
to supply my directly with scripts like you want to deal with this. So this one
was success, it worked:
peterg@debian:~/Downloads$ ./repro.sh
Cannot remove namespace file "/var/run/netns/ns0": No such file or directory
Cannot remove namespace file "/var/run/netns/ns1": No such file or directory
Cannot find device "br0"
PING 10.0.0.101 (10.0.0.101): 56 data bytes
64 bytes from 10.0.0.101: icmp_seq=0 ttl=64 time=0.057 ms
64 bytes from 10.0.0.101: icmp_seq=1 ttl=64 time=0.072 ms
64 bytes from 10.0.0.101: icmp_seq=2 ttl=64 time=0.037 ms
64 bytes from 10.0.0.101: icmp_seq=3 ttl=64 time=0.055 ms
--- 10.0.0.101 ping statistics ---
4 packets transmitted, 4 packets received, 0% packet loss
round-trip min/avg/max/stddev = 0.037/0.055/0.072/0.000 ms
In the meantime, I was stubborn to find a solution to what I need in order to
progress and MACVLAN tech actually delivered it (private mode enough),
something newer than VETH tech what I could read about, and it's just perfect
avoiding bridge itself. So no problem to cooperate on this fix, I will be
glad, just it can get lower priority on your side if you even attributed it
some
Thanks, wishing successful new week.
Peter
-Original Message-
From: Daniel Gröber
Sent: sobota 11. novembra 2023 1:35
To: GASPAROVIC Peter OBS/MKT ;
1054...@bugs.debian.org
Subject: Re: Bug#1054642: Failing ARP relay from external -> Linux bridge ->
veth port --> NS veth port
Hi Peter,
looking at the ip/bridge dumps I don't see anything obviously broken so I
started by building a reproducer using two netns'en and a bridge on the host to
simulate your setup, leaving out the vlan stuff for now.
I setup two namespaces ns0/ns1 with a veth pair each connected to br0 on the
host. I assign MAC addresses statically to make looking at `bridge fdb` easier
(grep ^aa:). The script looks like this (trimmed, full version
attached):
ip netns add ns0
ip link add veth0 type veth \
peer name veth0 address aa:00:00:00:00:00 netns ns0
ip netns add ns1
ip link add veth1 type veth \
peer name veth1 address aa:00:00:00:00:01 netns ns1
ip link add br0 address aa:bb:bb:bb:bb:bb type bridge forward_delay 0
#^ forward_delay=0 to disable STP as this delays interfaces coming up
ip link set dev veth0 master br0
ip link set dev veth1 master br0
ip -n ns0 addr add 10.0.0.100/24 dev veth0
ip -n ns1 addr add 10.0.0.101/24 dev veth1
iplink set br0 up
iplink set dev veth0 up
ip -n ns0 link set dev veth0 up
iplink set dev veth1 up
ip -n ns1 link set dev veth1 up
ip -n ns0 link set dev lo up
ip -n ns1 link set dev lo up
ip netns exec ns0 ping -c4 10.0.0.101
Seems to work fine. So we can establish the basic functionality does work and
we need to go deeper.
Peter, can you confirm this script works on your system? If so the next step is
introducing vlans.
On Mon, Oct 30, 2023 at 07:25:38PM +, peter.gasparo...@orange.com wrote:
> 1) As was reported, foreign external world MAC@ does not pass into
> network namespace, just external border point "vlan199"
How did you check this?
> 2) now collecting data for you, honestly I don’t see external mac
> address on "inet-br" object, so my previous statement was incorrect..
> {ossibly I might mixed this up with another "labinet-br" (working in
> its limited
> scope) which is IP-defined, while "inet-br" in question is not.
>
> 3) so question is, if the MACs learnt via vlan199 are supposed to be
> paired (displayed) with "inet-br" object and all way up into NS
I want to make sure we're on the same page, how do you check if the MAC is
reaching into the NS? I assume using `ip neigh`? I'd have a look at tcpdump
this will tell you whether ARP is even reaching the NS or not.
Something simple like
$ tcpdump -enli $IFACE 'arp or icmp'
optionally you can filter by MAC (`ether host` in pcap-filter speak):
$ tcpdump -enli $IFACE ('arp or icmp) and ether host aa:00:00:00:00:01
Oh and one last thing: please double and tripple check that a firewall isn't
interfering.
--Daniel
Ce message et ses pieces jointes peuvent contenir des informations
confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce
message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages
electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou
falsifie. Merci.
This message and its attachments may contain confidential or pri
Bug#1054642: Failing ARP relay from external -> Linux bridge -> veth port --> NS veth port
Hi Peter,
looking at the ip/bridge dumps I don't see anything obviously broken so I
started by building a reproducer using two netns'en and a bridge on the
host to simulate your setup, leaving out the vlan stuff for now.
I setup two namespaces ns0/ns1 with a veth pair each connected to br0 on
the host. I assign MAC addresses statically to make looking at `bridge fdb`
easier (grep ^aa:). The script looks like this (trimmed, full version
attached):
ip netns add ns0
ip link add veth0 type veth \
peer name veth0 address aa:00:00:00:00:00 netns ns0
ip netns add ns1
ip link add veth1 type veth \
peer name veth1 address aa:00:00:00:00:01 netns ns1
ip link add br0 address aa:bb:bb:bb:bb:bb type bridge forward_delay 0
#^ forward_delay=0 to disable STP as this delays interfaces coming up
ip link set dev veth0 master br0
ip link set dev veth1 master br0
ip -n ns0 addr add 10.0.0.100/24 dev veth0
ip -n ns1 addr add 10.0.0.101/24 dev veth1
iplink set br0 up
iplink set dev veth0 up
ip -n ns0 link set dev veth0 up
iplink set dev veth1 up
ip -n ns1 link set dev veth1 up
ip -n ns0 link set dev lo up
ip -n ns1 link set dev lo up
ip netns exec ns0 ping -c4 10.0.0.101
Seems to work fine. So we can establish the basic functionality does work
and we need to go deeper.
Peter, can you confirm this script works on your system? If so the next
step is introducing vlans.
On Mon, Oct 30, 2023 at 07:25:38PM +, peter.gasparo...@orange.com wrote:
> 1) As was reported, foreign external world MAC@ does not pass into
> network namespace, just external border point "vlan199"
How did you check this?
> 2) now collecting data for you, honestly I don’t see external mac address
> on "inet-br" object, so my previous statement was incorrect.. {ossibly I
> might mixed this up with another "labinet-br" (working in its limited
> scope) which is IP-defined, while "inet-br" in question is not.
>
> 3) so question is, if the MACs learnt via vlan199 are supposed to be
> paired (displayed) with "inet-br" object and all way up into NS
I want to make sure we're on the same page, how do you check if the MAC is
reaching into the NS? I assume using `ip neigh`? I'd have a look at tcpdump
this will tell you whether ARP is even reaching the NS or not.
Something simple like
$ tcpdump -enli $IFACE 'arp or icmp'
optionally you can filter by MAC (`ether host` in pcap-filter speak):
$ tcpdump -enli $IFACE ('arp or icmp) and ether host aa:00:00:00:00:01
Oh and one last thing: please double and tripple check that a firewall
isn't interfering.
--Daniel
repro.sh
Description: Bourne shell script
signature.asc
Description: PGP signature
Bug#1054642: Failing ARP relay from external -> Linux bridge -> veth port --> NS veth port
Hi Daniel,
Hope you are good. What is the outlook after a week here? Thanks.
BR
Peter
-Original Message-
From: GASPAROVIC Peter OBS/MKT
Sent: pondelok 30. októbra 2023 20:26
To: Daniel Gröber
Cc: 1054...@bugs.debian.org
Subject: RE: Bug#1054642: Failing ARP relay from external -> Linux bridge ->
veth port --> NS veth port
Hi Daniel,
Definitely I can't do any script at the moment, so manual steps could be enough
I hope so.
1) As was reported, foreign external world MAC@ does not pass into network
namespace, just external border point "vlan199"
2) now collecting data for you, honestly I don’t see external mac address on
"inet-br" object, so my previous statement was incorrect.. {ossibly I might
mixed this up with another "labinet-br" (working in its limited scope) which is
IP-defined, while "inet-br" in question is not.
3) so question is, if the MACs learnt via vlan199 are supposed to be paired
(displayed) with "inet-br" object and all way up into NS
4) I collected all into text file. If this is problem, then I paste it here.
Thanks, BR
Peter
-Original Message-
From: Daniel Gröber
Sent: pondelok 30. októbra 2023 13:04
To: GASPAROVIC Peter OBS/MKT
Cc: 1054...@bugs.debian.org
Subject: Re: Bug#1054642: Failing ARP relay from external -> Linux bridge ->
veth port --> NS veth port
Hi Peter,
On Mon, Oct 30, 2023 at 10:43:39AM +, peter.gasparo...@orange.com wrote:
> Would it be possible to join a Webex session setup by me to check this
> out quickly? It's all lab environment.
I don't think that would help with reproducing your environment in this case,
besides I only offer synchronous debugging sessions for paid consulting
engagements.
> If not I will proceed per your instructions
Please do.
--Daniel
Ce message et ses pieces jointes peuvent contenir des informations
confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce
message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages
electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou
falsifie. Merci.
This message and its attachments may contain confidential or privileged
information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete
this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been
modified, changed or falsified.
Thank you.
Bug#1054642: Failing ARP relay from external -> Linux bridge -> veth port --> NS veth port
Hi Daniel,
Definitely I can't do any script at the moment, so manual steps could be enough
I hope so.
1) As was reported, foreign external world MAC@ does not pass into network
namespace, just external border point "vlan199"
2) now collecting data for you, honestly I don’t see external mac address on
"inet-br" object, so my previous statement was incorrect.. {ossibly I might
mixed this up with another "labinet-br" (working in its limited scope) which is
IP-defined, while "inet-br" in question is not.
3) so question is, if the MACs learnt via vlan199 are supposed to be paired
(displayed) with "inet-br" object and all way up into NS
4) I collected all into text file. If this is problem, then I paste it here.
Thanks, BR
Peter
-Original Message-
From: Daniel Gröber
Sent: pondelok 30. októbra 2023 13:04
To: GASPAROVIC Peter OBS/MKT
Cc: 1054...@bugs.debian.org
Subject: Re: Bug#1054642: Failing ARP relay from external -> Linux bridge ->
veth port --> NS veth port
Hi Peter,
On Mon, Oct 30, 2023 at 10:43:39AM +, peter.gasparo...@orange.com wrote:
> Would it be possible to join a Webex session setup by me to check this
> out quickly? It's all lab environment.
I don't think that would help with reproducing your environment in this case,
besides I only offer synchronous debugging sessions for paid consulting
engagements.
> If not I will proceed per your instructions
Please do.
--Daniel
Ce message et ses pieces jointes peuvent contenir des informations
confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce
message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages
electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou
falsifie. Merci.
This message and its attachments may contain confidential or privileged
information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete
this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been
modified, changed or falsified.
Thank you.
peterg@debian:~$
peterg@debian:~$
peterg@debian:~$
peterg@debian:~$ ip -d addr show
1: lo: mtu 65536 qdisc noqueue state UNKNOWN group
default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 promiscuity 0 minmtu
0 maxmtu 0 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max
_segs 65535
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens161: mtu 9000 qdisc mq state UP group
default qlen 1000
link/ether 00:50:56:01:01:04 brd ff:ff:ff:ff:ff:ff promiscuity 0 minmtu 60
maxmtu 9000 numtxqueues 2 numrxqueues 2 gso_max_size 65536 gso_ma
x_segs 65535
inet6 fe80::250:56ff:fe01:104/64 scope link
valid_lft forever preferred_lft forever
3: ens192: mtu 1500 qdisc mq state UP group
default qlen 1000
link/ether 00:50:56:01:01:01 brd ff:ff:ff:ff:ff:ff promiscuity 0 minmtu 60
maxmtu 9000 numtxqueues 2 numrxqueues 2 gso_max_size 65536 gso_ma
x_segs 65535
inet 172.31.254.50/28 scope global ens192
valid_lft forever preferred_lft forever
inet6 fe80::250:56ff:fe01:101/64 scope link
valid_lft forever preferred_lft forever
4: ens224: mtu 1500 qdisc mq state UP group
default qlen 1000
link/ether 00:50:56:01:01:02 brd ff:ff:ff:ff:ff:ff promiscuity 2 minmtu 60
maxmtu 9000 numtxqueues 2 numrxqueues 2 gso_max_size 65536 gso_ma
x_segs 65535
inet6 fe80::250:56ff:fe01:102/64 scope link
valid_lft forever preferred_lft forever
5: ens256: mtu 1500 qdisc mq state UP group
default qlen 1000
link/ether 00:50:56:01:01:03 brd ff:ff:ff:ff:ff:ff promiscuity 0 minmtu 60
maxmtu 9000 numtxqueues 2 numrxqueues 2 gso_max_size 65536 gso_ma
x_segs 65535
inet6 fe80::250:56ff:fe01:103/64 scope link
valid_lft forever preferred_lft forever
6: vlan11@ens224: mtu 1500 qdisc noqueue
state UP group default qlen 1000
link/ether 00:50:56:01:01:02 brd ff:ff:ff:ff:ff:ff promiscuity 0 minmtu 0
maxmtu 65535
vlan protocol 802.1Q id 11 numtxqueues 1 numrxqueues 1
gso_max_size 65536 gso_max_segs 65535
inet 192.168.255.254/24 brd 192.168.255.255 scope global vlan11
valid_lft forever preferred_lft forever
inet6 fe80::250:56ff:fe01:102/64 scope link
valid_lft forever preferred_lft forever
20: vlan77@ens224: mtu 1500 qdisc noqueue
master labinet-br state UP group default qlen 1000
link/ether 00:50:56:01:01:02 brd ff:ff:ff:ff:ff:ff promiscuity 1 minmtu 0
maxmtu 65535
vlan
Bug#1054642: Failing ARP relay from external -> Linux bridge -> veth port --> NS veth port
Hi Peter, On Mon, Oct 30, 2023 at 10:43:39AM +, peter.gasparo...@orange.com wrote: > Would it be possible to join a Webex session setup by me to check this > out quickly? It's all lab environment. I don't think that would help with reproducing your environment in this case, besides I only offer synchronous debugging sessions for paid consulting engagements. > If not I will proceed per your instructions Please do. --Daniel
Bug#1054642: Failing ARP relay from external -> Linux bridge -> veth port --> NS veth port
Hi Daniel, Thank you for this yet hope even my joy to help fix something in this amazing Linux world you dive so deep in in contrast to me. Would it be possible to join a Webex session setup by me to check this out quickly? It's all lab environment. If not I will proceed per your instructions BR Peter -Original Message- From: Daniel Gröber Sent: nedeľa 29. októbra 2023 11:47 To: GASPAROVIC Peter OBS/MKT Cc: Luca Boccassi ; 1054...@bugs.debian.org Subject: Re: Bug#1054642: Failing ARP relay from external -> Linux bridge -> veth port --> NS veth port Hi Peter, On Fri, Oct 27, 2023 at 09:29:25AM +, peter.gasparo...@orange.com wrote: > No attempt at all? Then it's against your own rules I've read before > submitting this. I think Luca was a bit harsh here, I'd be happy to help debug this. From a first look it seems unlikely this is related to iproute2, smells more like a kernel issue to me, but either way we need a reproducer. So first step to move this forward is to put together a self contained set of instructions to reproduce the problem. Your original report is a bit sparse on context and details. If you don't feel up to compiling the reproducer script yourself you could start by showing us your system state using $ ip -d addr show # on the host and inside the NS if you could $ bridge -d link; bridge fdb snippets from /etc/network/interfaces or similar relevant config would help too. --Daniel Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci. This message and its attachments may contain confidential or privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. If you have received this email in error, please notify the sender and delete this message and its attachments. As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified. Thank you.
Bug#1054642: Failing ARP relay from external -> Linux bridge -> veth port --> NS veth port
Hi Peter, On Fri, Oct 27, 2023 at 09:29:25AM +, peter.gasparo...@orange.com wrote: > No attempt at all? Then it's against your own rules I've read before > submitting this. I think Luca was a bit harsh here, I'd be happy to help debug this. From a first look it seems unlikely this is related to iproute2, smells more like a kernel issue to me, but either way we need a reproducer. So first step to move this forward is to put together a self contained set of instructions to reproduce the problem. Your original report is a bit sparse on context and details. If you don't feel up to compiling the reproducer script yourself you could start by showing us your system state using $ ip -d addr show # on the host and inside the NS if you could $ bridge -d link; bridge fdb snippets from /etc/network/interfaces or similar relevant config would help too. --Daniel signature.asc Description: PGP signature
Bug#1054642: Failing ARP relay from external -> Linux bridge -> veth port --> NS veth port
No attempt at all? Then it's against your own rules I've read before submitting this. -Original Message- From: Luca Boccassi Sent: piatok 27. októbra 2023 11:17 To: GASPAROVIC Peter OBS/MKT ; 1054...@bugs.debian.org Subject: Re: Bug#1054642: Failing ARP relay from external -> Linux bridge -> veth port --> NS veth port Control: tags -1 upstream On Fri, 27 Oct 2023 at 09:18, wrote: > > Package: iproute2 > Version: 4.20.0-2+deb10u1 > Problem: External MAC@ reaches max Linux bridge, but not net namespace > linked via veth pair to it, based on very minimal config > > Hello Debian team, > > I would like to report problem which possibly has to do with IPROUTE2 > package, I’m experiencing it both Debian 10 (this) and 12 (6.1.0-3). I really > did my best reviewing at least 7 stack-exchange (and like) stories and I’m at > my wit’s end, wondering why this is possibly not fixed in 2023 seeing debates > go back into like 2014.. > > So it’s plain simple to want to make multiple namespaces able to communicate > via common host bridge to external network. VETH tech is all time documented > as solution to this. The problem on given path in subject is this: > > NS veth IP@ = .251 , 0e:61:72:97:aa:ff > (Bridge) veth IP@ = .44 , ce:18:16:4b:0c:c2 Bridge IP@ = .254 , > 00:50:56:01:01:02 External IP@ = .other > > 1) When I initially set up plain “veth port --> NS veth port”, with IP@ at > each end, it’s all seamless, ARP and pings.. > >== 2) Once I enslave veth port to bridge, I can’t reach external > >network <=== > 3) Veth also does not work on IP level anymore, all the time with ICMP echo > from NS space it runs ARP first, though both “ip nei” are populated with > mutual MAC records. The following goes in loop.. > peterg@debian:~$ sudo tcpdump -ni vinet-brp > tcpdump: verbose output suppressed, use -v or -vv for full protocol > decode listening on vinet-brp, link-type EN10MB (Ethernet), capture > size 262144 bytes > 11:18:12.966955 IP 70.0.0.251 > 70.0.0.44: ICMP echo request, id 2333, > seq 0, length 64 > 11:18:12.966984 ARP, Request who-has 70.0.0.251 tell 70.0.0.44, length > 28 > 11:18:12.966989 ARP, Reply 70.0.0.251 is-at 0e:61:72:97:aa:ff, length > 28 > 11:18:13.967994 IP 70.0.0.251 > 70.0.0.44: ICMP echo request, id 2333, > seq 1, length > 4) Once I configure bridge iface with some IP address of same subnet > /24 like veth and NS veth (also externals) use → the NS nei can show > changing MAC address for bridge veth iface > 11:30:27.860907 ARP, Reply 70.0.0.251 is-at 0e:61:72:97:aa:ff, length > 28 > 11:30:28.848251 IP 70.0.0.251 > 70.0.0.44: ICMP echo request, id 2352, > seq 14, length 64 > 11:30:28.884892 ARP, Request who-has 70.0.0.251 tell 70.0.0.44, length > 28 > 11:30:28.884908 ARP, Reply 70.0.0.251 is-at 0e:61:72:97:aa:ff, length > 28 > 11:30:28.980890 ARP, Request who-has 70.0.0.44 tell 70.0.0.251, length > 28 > 11:30:28.980909 ARP, Reply 70.0.0.44 is-at 00:50:56:01:01:02, length > 28 <--- > > inet_bash >> ip nei > 70.0.0.1 dev vinet FAILED > 70.0.0.44 dev vinet lladdr ce:18:16:4b:0c:c2 DELAY <--- > > 5) The bridge vs NS veth pinging works > 6) The bridge relays ARP into external network and back (checked on Cisco > switch), learns of external MAC@s > ===> 7) External MAC@ does not make it to NS space by ARP<=== > 8) I don’t aim to deploy IP@ with bridge and bridge veth ifaces → this > is just to check how it works > 9) This blog was quite surprising stating that bridge without IP@ can > affect routing in global namespace, few /sys kernel tweaks → no help > https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Funix > .stackexchange.com%2Fquestions%2F655602%2Fwhy-two-bridged-veth-cannot- > ping-each-other%2F674621%23674621=05%7C01%7Cpeter.gasparovic%40or > ange.com%7C5d4b8a4dcdf140886f7608dbd6cd941d%7C90c7a20af34b40bfbc48b925 > 3b6f5d20%7C0%7C0%7C638339950657397266%7CUnknown%7CTWFpbGZsb3d8eyJWIjoi > MC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7 > C%7C=mgDKlgwu7NBcoSLZwC0GooZDiBEiz6GVRL02UaMF40E%3D=0 > 10) Even tried to stop default MAC learning on bridge veth iface by “ip link > set dev vinet-brp type bridge_slave learning off” ⇒ did not work, neigh > flushed and pinging .251 vs .254 just worked. > > So I believe that bridge veth iface is broken in its essential functionality > using default “learning/flooding on” settings. > > Thanks for your time to look at this and give hope or deny this so I need to > create extra ports in my host to get what I want to. > BR > Peter You need to report this upstream, nobody here is going to look at something like this __
Bug#1054642: Failing ARP relay from external -> Linux bridge -> veth port --> NS veth port
Control: tags -1 upstream On Fri, 27 Oct 2023 at 09:18, wrote: > > Package: iproute2 > Version: 4.20.0-2+deb10u1 > Problem: External MAC@ reaches max Linux bridge, but not net namespace linked > via veth pair to it, based on very minimal config > > Hello Debian team, > > I would like to report problem which possibly has to do with IPROUTE2 > package, I’m experiencing it both Debian 10 (this) and 12 (6.1.0-3). I really > did my best reviewing at least 7 stack-exchange (and like) stories and I’m at > my wit’s end, wondering why this is possibly not fixed in 2023 seeing debates > go back into like 2014.. > > So it’s plain simple to want to make multiple namespaces able to communicate > via common host bridge to external network. VETH tech is all time documented > as solution to this. The problem on given path in subject is this: > > NS veth IP@ = .251 , 0e:61:72:97:aa:ff > (Bridge) veth IP@ = .44 , ce:18:16:4b:0c:c2 > Bridge IP@ = .254 , 00:50:56:01:01:02 > External IP@ = .other > > 1) When I initially set up plain “veth port --> NS veth port”, with IP@ at > each end, it’s all seamless, ARP and pings.. > >== 2) Once I enslave veth port to bridge, I can’t reach external network <=== > 3) Veth also does not work on IP level anymore, all the time with ICMP echo > from NS space it runs ARP first, though both “ip nei” are populated with > mutual MAC records. The following goes in loop.. > peterg@debian:~$ sudo tcpdump -ni vinet-brp > tcpdump: verbose output suppressed, use -v or -vv for full protocol decode > listening on vinet-brp, link-type EN10MB (Ethernet), capture size 262144 bytes > 11:18:12.966955 IP 70.0.0.251 > 70.0.0.44: ICMP echo request, id 2333, seq 0, > length 64 > 11:18:12.966984 ARP, Request who-has 70.0.0.251 tell 70.0.0.44, length 28 > 11:18:12.966989 ARP, Reply 70.0.0.251 is-at 0e:61:72:97:aa:ff, length 28 > 11:18:13.967994 IP 70.0.0.251 > 70.0.0.44: ICMP echo request, id 2333, seq 1, > length > 4) Once I configure bridge iface with some IP address of same subnet /24 like > veth and NS veth (also externals) use → the NS nei can show changing MAC > address for bridge veth iface > 11:30:27.860907 ARP, Reply 70.0.0.251 is-at 0e:61:72:97:aa:ff, length 28 > 11:30:28.848251 IP 70.0.0.251 > 70.0.0.44: ICMP echo request, id 2352, seq > 14, length 64 > 11:30:28.884892 ARP, Request who-has 70.0.0.251 tell 70.0.0.44, length 28 > 11:30:28.884908 ARP, Reply 70.0.0.251 is-at 0e:61:72:97:aa:ff, length 28 > 11:30:28.980890 ARP, Request who-has 70.0.0.44 tell 70.0.0.251, length 28 > 11:30:28.980909 ARP, Reply 70.0.0.44 is-at 00:50:56:01:01:02, length 28 <--- > > inet_bash >> ip nei > 70.0.0.1 dev vinet FAILED > 70.0.0.44 dev vinet lladdr ce:18:16:4b:0c:c2 DELAY <--- > > 5) The bridge vs NS veth pinging works > 6) The bridge relays ARP into external network and back (checked on Cisco > switch), learns of external MAC@s > ===> 7) External MAC@ does not make it to NS space by ARP<=== > 8) I don’t aim to deploy IP@ with bridge and bridge veth ifaces → this is > just to check how it works > 9) This blog was quite surprising stating that bridge without IP@ can affect > routing in global namespace, few /sys kernel tweaks → no help > https://unix.stackexchange.com/questions/655602/why-two-bridged-veth-cannot-ping-each-other/674621#674621 > 10) Even tried to stop default MAC learning on bridge veth iface by “ip link > set dev vinet-brp type bridge_slave learning off” ⇒ did not work, neigh > flushed and pinging .251 vs .254 just worked. > > So I believe that bridge veth iface is broken in its essential functionality > using default “learning/flooding on” settings. > > Thanks for your time to look at this and give hope or deny this so I need to > create extra ports in my host to get what I want to. > BR > Peter You need to report this upstream, nobody here is going to look at something like this
Bug#1054642: Failing ARP relay from external -> Linux bridge -> veth port --> NS veth port
Package: iproute2 Version: 4.20.0-2+deb10u1 Problem: External MAC@ reaches max Linux bridge, but not net namespace linked via veth pair to it, based on very minimal config Hello Debian team, I would like to report problem which possibly has to do with IPROUTE2 package, I’m experiencing it both Debian 10 (this) and 12 (6.1.0-3). I really did my best reviewing at least 7 stack-exchange (and like) stories and I’m at my wit’s end, wondering why this is possibly not fixed in 2023 seeing debates go back into like 2014.. So it’s plain simple to want to make multiple namespaces able to communicate via common host bridge to external network. VETH tech is all time documented as solution to this. The problem on given path in subject is this: NS veth IP@ = .251 , 0e:61:72:97:aa:ff (Bridge) veth IP@ = .44 , ce:18:16:4b:0c:c2 Bridge IP@ = .254 , 00:50:56:01:01:02 External IP@ = .other 1) When I initially set up plain “veth port --> NS veth port”, with IP@ at each end, it’s all seamless, ARP and pings.. >== 2) Once I enslave veth port to bridge, I can’t reach external network <=== 3) Veth also does not work on IP level anymore, all the time with ICMP echo from NS space it runs ARP first, though both “ip nei” are populated with mutual MAC records. The following goes in loop.. peterg@debian:~$ sudo tcpdump -ni vinet-brp tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on vinet-brp, link-type EN10MB (Ethernet), capture size 262144 bytes 11:18:12.966955 IP 70.0.0.251 > 70.0.0.44: ICMP echo request, id 2333, seq 0, length 64 11:18:12.966984 ARP, Request who-has 70.0.0.251 tell 70.0.0.44, length 28 11:18:12.966989 ARP, Reply 70.0.0.251 is-at 0e:61:72:97:aa:ff, length 28 11:18:13.967994 IP 70.0.0.251 > 70.0.0.44: ICMP echo request, id 2333, seq 1, length 4) Once I configure bridge iface with some IP address of same subnet /24 like veth and NS veth (also externals) use → the NS nei can show changing MAC address for bridge veth iface 11:30:27.860907 ARP, Reply 70.0.0.251 is-at 0e:61:72:97:aa:ff, length 28 11:30:28.848251 IP 70.0.0.251 > 70.0.0.44: ICMP echo request, id 2352, seq 14, length 64 11:30:28.884892 ARP, Request who-has 70.0.0.251 tell 70.0.0.44, length 28 11:30:28.884908 ARP, Reply 70.0.0.251 is-at 0e:61:72:97:aa:ff, length 28 11:30:28.980890 ARP, Request who-has 70.0.0.44 tell 70.0.0.251, length 28 11:30:28.980909 ARP, Reply 70.0.0.44 is-at 00:50:56:01:01:02, length 28 <--- inet_bash >> ip nei 70.0.0.1 dev vinet FAILED 70.0.0.44 dev vinet lladdr ce:18:16:4b:0c:c2 DELAY <--- 5) The bridge vs NS veth pinging works 6) The bridge relays ARP into external network and back (checked on Cisco switch), learns of external MAC@s ===> 7) External MAC@ does not make it to NS space by ARP<=== 8) I don’t aim to deploy IP@ with bridge and bridge veth ifaces → this is just to check how it works 9) This blog was quite surprising stating that bridge without IP@ can affect routing in global namespace, few /sys kernel tweaks → no help https://unix.stackexchange.com/questions/655602/why-two-bridged-veth-cannot-ping-each-other/674621#674621 10) Even tried to stop default MAC learning on bridge veth iface by “ip link set dev vinet-brp type bridge_slave learning off” ⇒ did not work, neigh flushed and pinging .251 vs .254 just worked. So I believe that bridge veth iface is broken in its essential functionality using default “learning/flooding on” settings. Thanks for your time to look at this and give hope or deny this so I need to create extra ports in my host to get what I want to. BR Peter Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci. This message and its attachments may contain confidential or privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. If you have received this email in error, please notify the sender and delete this message and its attachments. As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified. Thank you.
