Hi,
for a few days now I had problems with my setup with the new veb in 7.9.
I solved it today and just want to leave it here to prevent other people
scratching their heads.
I think this topic here is
related:https://marc.info/?l=openbsd-misc&m=177349148821654&w=2
My setup:
I have a machine behind a gateway which hosts two VMs, one for got/git
and one as a mail server. I want to access the host as well as both VMs
via ssh. All of them are in the same VLAN (id 40). This VLAN is only
accessible by ssh'ing through the gateway (as a jumphost).
Additionally, I use sftp to transfer files between my cell phone and the
host. This is directly forwarded through the gateway to a VLAN with id
50. This is because I haven't found a decent app which supports
jumphosts (never mind public/private keys), and I wanted to have some
logical barrier between my "trusted" VLAN (id 40) and the VLAN which is
accessible by "untrusted" devices.
The host has one physical interface, em0.
All interfaces get their IPs per DHCP from the gateway (IPv4 only).
In the 7.8 days, I had two vlan interfaces (vlan40 and vlan50, you guess
their ids) with em0 as parent. The VMs had interfaces which connected to
the bridge veb0 with their tapX interfaces, inside each VM there was an
interface also called vlan40 with vio0 as parent. em0 was also added to
veb0. In graphical form (at least I thought it would operate like this):
host-vlan40 -> em0 -> veb0
host-vlan50 -> em0 -> veb0
git-vlan40 -> vio0 -> tap0 -> veb0
mail-vlan40 -> vio0 -> tap0 -> veb0
veb0 had the link0 flag set. That setup worked in 7.8, but hell did it
bite me after upgrading to 7.9! The host vlans still got IPs and had
contact to the outside world, but not the VMs. I saw DHCP requests going
out on em0, and DHCP offers coming in on em0, but these offers were
never forwarded to the VMs.
The reason, and I hope I am right here because I have no clue about the
internals, was that em0 forwarded all packets with id XY immediately to
the respective vlanXY, bypassing the veb and therefore these packets did
not reach the tap interfaces. And this took me a few days figuring this
out.
After that, I changed the parent of the host's vlanXY interfaces to a
vport0, and this worked for these two. I also removed the link0 flag,
but I am not sure if this is important, and don't want to test it right
now. All veb0 members were then correctly tagged (40,50). So this path
looked like this:
host-vlanXY -> vport0 -> veb0
But this did not work for the VMs. And then it hit me: that's the
"untagged" option for! So the solution is to bin the vlanXY interface
inside the VM and set, e.g.,
ifconfig veb0 untagged tap0 40
Then this path looks like this:
vio0 -> tap0 -> veb0
So now every packet going out of the VM through tha tap get its id set
to 40.
It is important to create the tap interfaces before starting the VM, so
I simply added hostname.tapX files.
And then I thought if this is also possible with the host's vlan
interfaces, and yes, I simply added two vports (vport40 an vport50), set
the untagged traffic to their respective ids and it works. So then this
path looks like this:
vport40 -> veb0
vport50 -> veb0
Here are the resultant hostname.if files. I added the tagged option to
each interface so they deal only with the traffic they are supposed to,
em0, tap0 and tap1 are only up'ped, no further configuration, omitted
here. Also the vport50 is the same as vport40, and the vio's of each VM
are identical.
$ cat /etc/hostname.veb0
add em0
-untagged em0
tagged em0 40,50
add vport40
untagged vport40 40
tagged vport40 40
add vport50
add tap0
untagged tap0 40
tagged tap0 40
add tap1
untagged tap1 40
tagged tap1 40
untagged vport50 50
tagged vport50 50
up
$ cat /etc/hostname.vport40
lladdr 90:fb:a6:46:db:e1
inet autoconf
up
$ cat /etc/hostname.vio0
inet autoconf
up
I hope this helps, and if anything is wrong or can be done better,
please correct me.
Cheers
Jochen
