Re: [lxc-users] Connecting container to tagged VLAN

[Joshua Schaeffer]

On Wed, Jan 27, 2016 at 6:09 PM, Fajar A. Nugraha  wrote:

>
>
>> eth2 already works. I set it up for testing outside of all containers
>> (i.e. on the host only). From the host:
>>
>>
> That doesn't match what you said earlier.
>

It actually does. Remember that this LXC host is a virtual machine running
off of VMware, which makes this whole situation more complex. I'll try to
clarify.

VLAN10, the native vlan, is 192.168.54.0/25. It's my management vlan
VLAN500 is 10.240.78.0/24.

eth1 and eth2 are setup to connect to vlan500 because they were setup that
way through VMware. Normarlly you would be correct, on a physical server
eth2 would only be able to contact the native vlan, because no tagging
information is provided. However VMware allows you to tag a NIC (its
actually called a port group, but it is essentially VMware's way of saying
a NIC) from outside the VM guest. If you do this (as I have) then you don't
(and shouldn't) need to tag anything on the VM guest itself. So by just
looking at the guest it can look incorrect/confusing.

My original problem was I was tagging the port group (a.k.a. VMware's NIC)
and I was tagging eth1 inside the VM guest (a.k.a. the LXC host). Clearly
this causes problems. Because I was tagging eth1 but not eth2 that is where
the problem resided. I was trying to mimic a setup I have in my home lab
where I tag an Ethernet device, add it to a bridge, then use that bridge in
a container, but my home lab uses a physical LXC host. Hopefully I've
explained it in a way that clears this up.

Either way I have that problem resolved. Now I'm just wondering why the
container is not adding the gateway's MAC address when it ARP's for it (as
I explained in my last email).


>
> What I meant, check that ETH1 works on the host. If eth2 is on the same
> network, it might interfere with settings. So disable eth2 first, then test
> eth1 on the host. Without bridging.
>

Okay that makes sense.

Thanks,
Joshua
___
lxc-users mailing list
lxc-users@lists.linuxcontainers.org
http://lists.linuxcontainers.org/listinfo/lxc-users

Re: [lxc-users] Connecting container to tagged VLAN

[Fajar A. Nugraha]

On Thu, Jan 28, 2016 at 5:19 AM, Joshua Schaeffer 
wrote:

>
> On Wed, Jan 27, 2016 at 2:39 PM, Fajar A. Nugraha  wrote:
>>
>>
>> Is eth1 connected to your switch as trunk? If no (e.g. you have the same
>> settings for eth1 and eth2 on the switch side),
>>
>
> Both ports are connected as trunk. As far as the switch side goes each
> ports is configured the same. Trunked for VLAN 10, 500 and 501. Native VLAN
> is 10.
>
> eth2 already works. I set it up for testing outside of all containers
> (i.e. on the host only). From the host:
>
>
That doesn't match what you said earlier.

"two NIC's (eth1 and eth2) are setup to connect to this VLAN (vlan id 500)"

"
Native VLAN is 10.
"

"
iface eth2 inet static
address 10.240.78.4/24
gateway 10.240.78.1
"

So 10.240.78.0/24 with gateway 10.240.78.1 is VLAN 10? It must be, since
you use eth2 directly.

Yet on lxc config file, you use
"
lxc.network.link = br0-500
lxc.network.ipv4 = 10.240.78.3/24
lxc.network.ipv4.gateway = 10.240.78.1
"

So vlan 10 and vlan 500 is using the same network?



> Kernel IP routing table
> Destination Gateway Genmask Flags Metric RefUse
> Iface
> 0.0.0.0 192.168.54.10.0.0.0 UG0  00
> eth0
> 10.0.3.00.0.0.0 255.255.255.0   U 0  00
> lxcbr0
> 10.240.78.0 0.0.0.0 255.255.255.0   U 0  00
> eth2
> 192.168.54.00.0.0.0 255.255.255.128 U 0  00
> eth0
>
> PING 10.240.78.1 (10.240.78.1) 56(84) bytes of data.
> 64 bytes from 10.240.78.1: icmp_seq=1 ttl=255 time=1.76 ms
>


That should be vlan10 (native vlan for eth2).

You haven't tested it in vlan500.

then you can't tag it inside your host.
>>
>
> I did have that idea and tried it without success:
>
> # The second network interface
> auto eth1
> iface eth1 inet manual
>
> #commenting out dot1q
> #iface eth1.500 inet manual
> #   vlan-raw-device eth1
>
> [...]
>
> auto br0-500
> iface br0-500 inet manual
> bridge_ports eth1
> bridge_stp off
> bridge_fd 0
> bridge_maxwait 0
>

If the settings are the same, then br0-500 in this configuration SHOULD be
able to access vlan10, its native vlan. If it DOESN't work, check your
switch.




>
>
>>
>> To put it another way:
>> - start with known-good configuration, THEN make incremental changes
>> - in yout case, start by testing whether it works on the HOST side when
>> you assign an IP address to eth1.500, WITHOUT br0-500 bridge
>>
>
> Okay thanks, I will try different configurations out.
>
>
>> , and WITHOUT any ip address assigned to eth2.
>>
>
> I'm not sure what you mean by not assigning an IP address to eth2. Eth2 is
> already working from the host, and I don't plan on using it inside any
> container (I may have failed to mention that before). Also how would the
> NIC work without an IP address? I feel I'm missing something obvious here.
>
>
What I meant, check that ETH1 works on the host. If eth2 is on the same
network, it might interfere with settings. So disable eth2 first, then test
eth1 on the host. Without bridging.

-- 
Fajar
___
lxc-users mailing list
lxc-users@lists.linuxcontainers.org
http://lists.linuxcontainers.org/listinfo/lxc-users

Re: [lxc-users] Connecting container to tagged VLAN

[Fajar A. Nugraha]

On Thu, Jan 28, 2016 at 1:43 AM, Joshua Schaeffer 
wrote:

> I'm trying to setup a container on a new VLAN that only allows tagged
> traffic and I'm getting varied success.
>

the other two NIC's (eth1 and eth2) are setup to connect to this VLAN (vlan
> id 500).
>
>



>
> # The third network interface
> auto eth2
> iface eth2 inet static
> address 10.240.78.4/24
> gateway 10.240.78.1
>
> iface eth1.500 inet manual
> vlan-raw-device eth1
>
>
Is eth1 connected to your switch as trunk? If no (e.g. you have the same
settings for eth1 and eth2 on the switch side), then you can't tag it
inside your host.

To put it another way:
- start with known-good configuration, THEN make incremental changes
- in yout case, start by testing whether it works on the HOST side when you
assign an IP address to eth1.500, WITHOUT br0-500 bridge, and WITHOUT any
ip address assigned to eth2.

-- 
Fajar
___
lxc-users mailing list
lxc-users@lists.linuxcontainers.org
http://lists.linuxcontainers.org/listinfo/lxc-users

Re: [lxc-users] Connecting container to tagged VLAN

[Guido Jäkel]

Dear Joshua,

you wrote, that there's a trunk on eth1 and eth2. But for eth2, i can't see any 
VLAN (501 ?) detrunking as with eth1 & eth1.500. In the other hand you wrote, 
that eth2 is working. Are you shure, that you realy receive this trunk of 3 
VLANs on your both eth's?

I'm using a (working) comparable setup: On the host, eth0 is used for host 
management on a detrunked port. On eth1, there's a trunk with the needed VLANs 
for different network for a staged environment. On eth1, there is a VLAN 
decoder for each of the needed VLANs. And this is attached to a seperate 
software bridge for each VLAN. A container's outside veth is attached for the 
appropriate bridge - this is done in a start script by a calculated 
configuration statement based on the container's name. But the lxc host is 
located on plain hardware, not in a VM.

On 27.01.2016 23:19, Joshua Schaeffer wrote:
> On Wed, Jan 27, 2016 at 2:39 PM, Fajar A. Nugraha  wrote:
>>
>>
>> Is eth1 connected to your switch as trunk? If no (e.g. you have the same
>> settings for eth1 and eth2 on the switch side),
>>
> 
> Both ports are connected as trunk. As far as the switch side goes each
> ports is configured the same. Trunked for VLAN 10, 500 and 501. Native VLAN
> is 10.
> 
> eth2 already works. I set it up for testing outside of all containers (i.e.
> on the host only). From the host:
> 

___
lxc-users mailing list
lxc-users@lists.linuxcontainers.org
http://lists.linuxcontainers.org/listinfo/lxc-users

Re: [lxc-users] Connecting container to tagged VLAN

[Joshua Schaeffer]

On Wed, Jan 27, 2016 at 4:38 PM, Guido Jäkel  wrote:

> Dear Joshua,
>
> you wrote, that there's a trunk on eth1 and eth2. But for eth2, i can't
> see any VLAN (501 ?) detrunking as with eth1 & eth1.500. In the other hand
> you wrote, that eth2 is working. Are you shure, that you realy receive this
> trunk of 3 VLANs on your both eth's?
>

I started to think about this as well and I've found the reason. VMware
allows you to tag NICs from the hypervisor level. Eth1 and eth2 were both
setup under VLAN 500 so that is why no tagging on the LXC host was
required, hence why eth2 worked. So the lesson there is don't mix dot1q,
either set it on the hypervisor and leave it completely out of the LXC host
and container or vice versa.

I've completely removed VLAN tagging from my LXC host and making progress,
but still running into odd situations:

lxcuser@prvlxc01:~$ sudo ip -d link show
1: lo:  mtu 65536 qdisc noqueue state UNKNOWN mode
DEFAULT group default
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 promiscuity 0
2: eth0:  mtu 1500 qdisc pfifo_fast state
UP mode DEFAULT group default qlen 1000
link/ether 00:50:56:be:13:94 brd ff:ff:ff:ff:ff:ff promiscuity 0
3: eth1:  mtu 1500 qdisc pfifo_fast master
br0-500 state UP mode DEFAULT group default qlen 1000
link/ether 00:50:56:be:46:c5 brd ff:ff:ff:ff:ff:ff promiscuity 1
bridge_slave
4: eth2:  mtu 1500 qdisc noop state DOWN mode DEFAULT
group default qlen 1000
link/ether 00:50:56:be:26:4f brd ff:ff:ff:ff:ff:ff promiscuity 0
5: eth3:  mtu 1500 qdisc noop state DOWN mode DEFAULT
group default qlen 1000
link/ether 00:50:56:be:01:d8 brd ff:ff:ff:ff:ff:ff promiscuity 0
6: br0-500:  mtu 1500 qdisc noqueue state
UP mode DEFAULT group default
link/ether 00:50:56:be:46:c5 brd ff:ff:ff:ff:ff:ff promiscuity 0
bridge
7: lxcbr0:  mtu 1500 qdisc noqueue state
UNKNOWN mode DEFAULT group default
link/ether de:ef:8c:53:01:0b brd ff:ff:ff:ff:ff:ff promiscuity 0
bridge
9: vethKAG02C:  mtu 1500 qdisc pfifo_fast
master br0-500 state UP mode DEFAULT group default qlen 1000
link/ether fe:bf:b5:cf:f0:83 brd ff:ff:ff:ff:ff:ff promiscuity 1
veth
bridge_slave

*Scenario 1*: When assigning an IP directly to eth1 on the host, no
bridging involved, no containers involved (Success):

/etc/network/interfaces
auto eth1
iface eth1 inet static
address 10.240.78.3/24

route -n
10.240.78.0 0.0.0.0 255.255.255.0   U 0  00 eth1

PING 10.240.78.1 (10.240.78.1) 56(84) bytes of data.
64 bytes from 10.240.78.1: icmp_seq=1 ttl=255 time=8.25 ms
64 bytes from 10.240.78.1: icmp_seq=2 ttl=255 time=2.59 ms
^C
--- 10.240.78.1 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 2.597/5.425/8.254/2.829 ms

*Scenario 2*: When assigning an IP to a bridge and making eth1 a slave to
the bridge, no containers involved (Success):

/etc/network/interfaces
auto eth1
iface eth1 inet manual

auto br0-500
iface br0-500 inet static
address 10.240.78.3/24
bridge_ports eth1
bridge_stp off
bridge_fd 0
bridge_maxwait 0


route -n
10.240.78.0 0.0.0.0 255.255.255.0   U 0  00
br0-500

PING 10.240.78.1 (10.240.78.1) 56(84) bytes of data.
64 bytes from 10.240.78.1: icmp_seq=1 ttl=255 time=3.26 ms
64 bytes from 10.240.78.1: icmp_seq=2 ttl=255 time=1.51 ms
64 bytes from 10.240.78.1: icmp_seq=3 ttl=255 time=2.30 ms
^C
--- 10.240.78.1 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 1.514/2.360/3.262/0.715 ms

*Scenario 3*: Same scenario as above, expect the bridge is not assigned and
IP and a container is created and connects to the same bridge (Failure):

/etc/network/interfaces
auto eth1
iface eth1 inet manual

auto br0-500
iface br0-500 inet static
bridge_ports eth1
bridge_stp off
bridge_fd 0
bridge_maxwait 0

~/.local/share/lxc/c4/config
# Network configuration
lxc.network.type = veth
lxc.network.link = br0-500
lxc.network.ipv4 = 10.240.78.3/24
lxc.network.ipv4.gateway = 10.240.78.1
lxc.network.flags = up
lxc.network.hwaddr = 00:16:3e:f7:0a:83

route -n (on host)
10.240.78.0 0.0.0.0 255.255.255.0   U 0  00
br0-500

route -n (inside container)
10.240.78.0 0.0.0.0 255.255.255.0   U 0  00 eth0

ping (on host)
PING 10.240.78.1 (10.240.78.1) 56(84) bytes of data.
64 bytes from 10.240.78.1: icmp_seq=1 ttl=255 time=1.12 ms
64 bytes from 10.240.78.1: icmp_seq=2 ttl=255 time=1.17 ms
64 bytes from 10.240.78.1: icmp_seq=3 ttl=255 time=6.54 ms
^C
--- 10.240.78.1 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt 

[lxc-users] Connecting container to tagged VLAN

[Joshua Schaeffer]

I'm trying to setup a container on a new VLAN that only allows tagged
traffic and I'm getting varied success. Maybe somebody can point me in the
right direction. I can ping the gateway from the host but not from the
container and I can't see what I'm missing. I'm using LXC 1.1.5 on Debian
Jessie. The container is unprivileged. The host itself is a VM running off
of VMware. The VM has 3 NIC's. eth0 is for my management network and the
other two NIC's (eth1 and eth2) are setup to connect to this VLAN (vlan id
500).

/etc/network/interfaces
# The second network interface
auto eth1
iface eth1 inet manual

# The third network interface
auto eth2
iface eth2 inet static
address 10.240.78.4/24
gateway 10.240.78.1

iface eth1.500 inet manual
vlan-raw-device eth1

auto br0-500
iface br0-500 inet manual
bridge_ports eth1.500
bridge_stp off
bridge_fd 0
bridge_maxwait 0

I've setup br0-500 to use with my container:

# Network configuration
lxc.network.type = veth
lxc.network.link = br0-500
lxc.network.ipv4 = 10.240.78.3/24
lxc.network.ipv4.gateway = 10.240.78.1
lxc.network.flags = up
lxc.network.hwaddr = 00:16:3e:3d:51:af

When I start the container everything seems to be in order:

eth0  Link encap:Ethernet  HWaddr 00:16:3e:3d:51:af
  inet addr:10.240.78.3  Bcast:10.240.78.255  Mask:255.255.255.0
  inet6 addr: fe80::216:3eff:fe3d:51af/64 Scope:Link
  UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
  RX packets:8 errors:0 dropped:0 overruns:0 frame:0
  TX packets:11 errors:0 dropped:0 overruns:0 carrier:0
  collisions:0 txqueuelen:1000
  RX bytes:648 (648.0 B)  TX bytes:774 (774.0 B)

Kernel IP routing table
Destination Gateway Genmask Flags Metric RefUse
Iface
0.0.0.0 10.240.78.1 0.0.0.0 UG0  00 eth0
10.240.78.0 0.0.0.0 255.255.255.0   U 0  00 eth0

But when I try to ping the gateway I get no response:

PING 10.240.78.1 (10.240.78.1) 56(84) bytes of data.
>From 10.240.78.3 icmp_seq=1 Destination Host Unreachable
>From 10.240.78.3 icmp_seq=2 Destination Host Unreachable
>From 10.240.78.3 icmp_seq=3 Destination Host Unreachable
>From 10.240.78.3 icmp_seq=4 Destination Host Unreachable
>From 10.240.78.3 icmp_seq=5 Destination Host Unreachable
>From 10.240.78.3 icmp_seq=6 Destination Host Unreachable
^C
--- 10.240.78.1 ping statistics ---
7 packets transmitted, 0 received, +6 errors, 100% packet loss, time 6030ms

Address  HWtype  HWaddress   Flags Mask
 Iface
10.240.78.1  (incomplete)
 eth0

Running tcpdump on eth1 on the host, I can see the arp requests coming
through the host but there is no reply from the gateway.

lxcuser@prvlxc01:~$ su root -c "tcpdump -i eth1 -Uw - | tcpdump -en -r -
vlan 500"
Password:
tcpdump: listening on eth1, link-type EN10MB (Ethernet), capture size
262144 bytes
reading from file -, link-type EN10MB (Ethernet)
11:35:34.589795 00:16:3e:3d:51:af > ff:ff:ff:ff:ff:ff, ethertype 802.1Q
(0x8100), length 46: vlan 500, p 0, ethertype ARP, Request who-has
10.240.78.1 tell 10.240.78.3, length 28
11:35:35.587647 00:16:3e:3d:51:af > ff:ff:ff:ff:ff:ff, ethertype 802.1Q
(0x8100), length 46: vlan 500, p 0, ethertype ARP, Request who-has
10.240.78.1 tell 10.240.78.3, length 28
11:35:36.587413 00:16:3e:3d:51:af > ff:ff:ff:ff:ff:ff, ethertype 802.1Q
(0x8100), length 46: vlan 500, p 0, ethertype ARP, Request who-has
10.240.78.1 tell 10.240.78.3, length 28
11:35:37.604816 00:16:3e:3d:51:af > ff:ff:ff:ff:ff:ff, ethertype 802.1Q
(0x8100), length 46: vlan 500, p 0, ethertype ARP, Request who-has
10.240.78.1 tell 10.240.78.3, length 28
11:35:38.603408 00:16:3e:3d:51:af > ff:ff:ff:ff:ff:ff, ethertype 802.1Q
(0x8100), length 46: vlan 500, p 0, ethertype ARP, Request who-has
10.240.78.1 tell 10.240.78.3, length 28
11:35:39.603387 00:16:3e:3d:51:af > ff:ff:ff:ff:ff:ff, ethertype 802.1Q
(0x8100), length 46: vlan 500, p 0, ethertype ARP, Request who-has
10.240.78.1 tell 10.240.78.3, length 28
11:35:40.620677 00:16:3e:3d:51:af > ff:ff:ff:ff:ff:ff, ethertype 802.1Q
(0x8100), length 46: vlan 500, p 0, ethertype ARP, Request who-has
10.240.78.1 tell 10.240.78.3, length 28
11:35:41.619399 00:16:3e:3d:51:af > ff:ff:ff:ff:ff:ff, ethertype 802.1Q
(0x8100), length 46: vlan 500, p 0, ethertype ARP, Request who-has
10.240.78.1 tell 10.240.78.3, length 28
^C
Session terminated, terminating shell...tcpdump: pcap_loop: error reading
dump file: Interrupted system call
16 packets captured
17 packets received by filter
0 packets dropped by kernel

I feel that this is a setup problem with the router, but I'm not getting
much help from my networking team so I'm kind of asking all around to see
if anybody has any good ideas. The only other source of the problem I can
think of is with VMware. Maybe somebody more familiar with the hypervisor
has seen this issue before? I have every port group on