Re: ezjails, systat -ifstat, and multiple network cards
On Thu, Feb 13, 2014 at 2:22 PM, Allan Jude wrote: > On 2014-02-13 13:59, Preston Hagar wrote: > > I have a server setup with FreeBSD-10.0-RELEASE. It has 3 Intel gigabit > > network cards in it, em0, em1, and em2. I have multiple ezjails setup > that > > run various things. > > > > One jail, called db, runs a postgresql database. It was my intention to > > give it em0 all to itself. The other jails and host machine should be > > going through em2. em1 currently isn't being used. > > > > If I do an ifconfig, I see that em0 has the alias IP for my db jail and > em2 > > has the alias IP for all other jails. All the jails respond to network > > traffic as expected and seemingly work fine. > > > > The weird thing is when I do a systat -ifstat from the host, it should > > essentially all traffic going through em0. Some of the jails that run > off > > of em2 (as defined in their jail config files and seen in ifconfig) have > > large data transfers and/or are web servers with lots of photos. I have > > even tried to manually scp a large file out of a jail setup through em2 > and > > the numbers don't seem to budge. > > > > If I do netstat -i -b -n -I and check em0 and em2, it seems to support > the > > numbers shown by systat -ifstat. However, if I use trafshow or iftop > (both > > of which require choosing one interface at a time), they both seem to > > indicate the traffic flowing through the interfaces as I would expect. > > > > So I was curious if anyone had seen something like this before or had any > > ideas of what is going on. I have net.fibs=2 set in /boot/loader.conf, > but > > in all the jails I current have jail_name_fib="" as I haven't got around > to > > fullying setting up fibs. Is that perhaps the issue? Is there any way > to > > determine with certainty which jail is using which interface short of > > physically pulling a network cable and seeing what stops working? > > > > Here are the relevant lines from my db (the one that should be on em0) > > config: > > > > export jail_db_hostname="db" > > export jail_db_ip="em0|10.1.10.2" > > > > From another jail on em2 called www: > > > > export jail_www_hostname="www" > > export jail_www_ip="em2|10.1.10.7" > > > > from ifconfig > > > > em0: flags=8843 metric 0 mtu 1500 > > > options=4219b > > ether 08:60:6e:13:94:06 > > inet 10.1.1.4 netmask 0x broadcast 10.1.255.255 > > inet6 fe80::a60:6eff:fe13:9406%em0 prefixlen 64 scopeid 0x1 > > inet 10.1.10.2 netmask 0x broadcast 10.1.10.2 > > nd6 options=29 > > media: Ethernet autoselect (1000baseT ) > > status: active > > > > em2: flags=8843 metric 0 mtu 1500 > > > options=4219b > > ether 68:05:ca:13:74:2a > > inet 10.1.1.2 netmask 0x broadcast 10.1.255.255 > > inet6 fe80::6a05:caff:fe13:742a%em2 prefixlen 64 scopeid 0x3 > > inet 10.1.10.3 netmask 0x broadcast 10.1.10.3 > > inet 10.1.10.1 netmask 0x broadcast 10.1.10.1 > > inet 10.1.10.8 netmask 0x broadcast 10.1.10.8 > > inet 10.1.10.10 netmask 0x broadcast 10.1.10.10 > > inet 10.1.10.4 netmask 0x broadcast 10.1.10.4 > > inet 10.1.10.9 netmask 0x broadcast 10.1.10.9 > > inet 10.1.10.7 netmask 0x broadcast 10.1.10.7 > > nd6 options=29 > > media: Ethernet autoselect (1000baseT ) > > status: active > > > > > > Let me know if any more detail would be helpful or if you have any ideas > of > > things to check. > > > > Thanks, > > > > Preston > > ___ > > freebsd-current@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-current > > To unsubscribe, send any mail to " > freebsd-current-unsubscr...@freebsd.org" > > > > All traffic going out from the jails will using the routing table from > the host system. The routing table will use the network card that is in > the same subnet as your default gateway to route the traffic to the > internet. > > In your case, I would imagine this is 10.1.1.4/16 (and 10.1.1.2/16). > > 'netstat -rn' will tell the tale, but I imagine it is whichever was > added first. > > If you want to have separate routing tables per jail, you'd have to > either use FIBs, and set the jails to use the different FIBs, or use > VNET jails and have a routing table in each jail. > > -- > Allan Jude > > Makes sense, thank you. I'll setup the FIBs. Preston ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: ezjails, systat -ifstat, and multiple network cards
On 2014-02-13 13:59, Preston Hagar wrote: > I have a server setup with FreeBSD-10.0-RELEASE. It has 3 Intel gigabit > network cards in it, em0, em1, and em2. I have multiple ezjails setup that > run various things. > > One jail, called db, runs a postgresql database. It was my intention to > give it em0 all to itself. The other jails and host machine should be > going through em2. em1 currently isn't being used. > > If I do an ifconfig, I see that em0 has the alias IP for my db jail and em2 > has the alias IP for all other jails. All the jails respond to network > traffic as expected and seemingly work fine. > > The weird thing is when I do a systat -ifstat from the host, it should > essentially all traffic going through em0. Some of the jails that run off > of em2 (as defined in their jail config files and seen in ifconfig) have > large data transfers and/or are web servers with lots of photos. I have > even tried to manually scp a large file out of a jail setup through em2 and > the numbers don't seem to budge. > > If I do netstat -i -b -n -I and check em0 and em2, it seems to support the > numbers shown by systat -ifstat. However, if I use trafshow or iftop (both > of which require choosing one interface at a time), they both seem to > indicate the traffic flowing through the interfaces as I would expect. > > So I was curious if anyone had seen something like this before or had any > ideas of what is going on. I have net.fibs=2 set in /boot/loader.conf, but > in all the jails I current have jail_name_fib="" as I haven't got around to > fullying setting up fibs. Is that perhaps the issue? Is there any way to > determine with certainty which jail is using which interface short of > physically pulling a network cable and seeing what stops working? > > Here are the relevant lines from my db (the one that should be on em0) > config: > > export jail_db_hostname="db" > export jail_db_ip="em0|10.1.10.2" > > From another jail on em2 called www: > > export jail_www_hostname="www" > export jail_www_ip="em2|10.1.10.7" > > from ifconfig > > em0: flags=8843 metric 0 mtu 1500 > options=4219b > ether 08:60:6e:13:94:06 > inet 10.1.1.4 netmask 0x broadcast 10.1.255.255 > inet6 fe80::a60:6eff:fe13:9406%em0 prefixlen 64 scopeid 0x1 > inet 10.1.10.2 netmask 0x broadcast 10.1.10.2 > nd6 options=29 > media: Ethernet autoselect (1000baseT ) > status: active > > em2: flags=8843 metric 0 mtu 1500 > options=4219b > ether 68:05:ca:13:74:2a > inet 10.1.1.2 netmask 0x broadcast 10.1.255.255 > inet6 fe80::6a05:caff:fe13:742a%em2 prefixlen 64 scopeid 0x3 > inet 10.1.10.3 netmask 0x broadcast 10.1.10.3 > inet 10.1.10.1 netmask 0x broadcast 10.1.10.1 > inet 10.1.10.8 netmask 0x broadcast 10.1.10.8 > inet 10.1.10.10 netmask 0x broadcast 10.1.10.10 > inet 10.1.10.4 netmask 0x broadcast 10.1.10.4 > inet 10.1.10.9 netmask 0x broadcast 10.1.10.9 > inet 10.1.10.7 netmask 0x broadcast 10.1.10.7 > nd6 options=29 > media: Ethernet autoselect (1000baseT ) > status: active > > > Let me know if any more detail would be helpful or if you have any ideas of > things to check. > > Thanks, > > Preston > ___ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org" > All traffic going out from the jails will using the routing table from the host system. The routing table will use the network card that is in the same subnet as your default gateway to route the traffic to the internet. In your case, I would imagine this is 10.1.1.4/16 (and 10.1.1.2/16). 'netstat -rn' will tell the tale, but I imagine it is whichever was added first. If you want to have separate routing tables per jail, you'd have to either use FIBs, and set the jails to use the different FIBs, or use VNET jails and have a routing table in each jail. -- Allan Jude signature.asc Description: OpenPGP digital signature
ezjails, systat -ifstat, and multiple network cards
I have a server setup with FreeBSD-10.0-RELEASE. It has 3 Intel gigabit network cards in it, em0, em1, and em2. I have multiple ezjails setup that run various things. One jail, called db, runs a postgresql database. It was my intention to give it em0 all to itself. The other jails and host machine should be going through em2. em1 currently isn't being used. If I do an ifconfig, I see that em0 has the alias IP for my db jail and em2 has the alias IP for all other jails. All the jails respond to network traffic as expected and seemingly work fine. The weird thing is when I do a systat -ifstat from the host, it should essentially all traffic going through em0. Some of the jails that run off of em2 (as defined in their jail config files and seen in ifconfig) have large data transfers and/or are web servers with lots of photos. I have even tried to manually scp a large file out of a jail setup through em2 and the numbers don't seem to budge. If I do netstat -i -b -n -I and check em0 and em2, it seems to support the numbers shown by systat -ifstat. However, if I use trafshow or iftop (both of which require choosing one interface at a time), they both seem to indicate the traffic flowing through the interfaces as I would expect. So I was curious if anyone had seen something like this before or had any ideas of what is going on. I have net.fibs=2 set in /boot/loader.conf, but in all the jails I current have jail_name_fib="" as I haven't got around to fullying setting up fibs. Is that perhaps the issue? Is there any way to determine with certainty which jail is using which interface short of physically pulling a network cable and seeing what stops working? Here are the relevant lines from my db (the one that should be on em0) config: export jail_db_hostname="db" export jail_db_ip="em0|10.1.10.2" >From another jail on em2 called www: export jail_www_hostname="www" export jail_www_ip="em2|10.1.10.7" from ifconfig em0: flags=8843 metric 0 mtu 1500 options=4219b ether 08:60:6e:13:94:06 inet 10.1.1.4 netmask 0x broadcast 10.1.255.255 inet6 fe80::a60:6eff:fe13:9406%em0 prefixlen 64 scopeid 0x1 inet 10.1.10.2 netmask 0x broadcast 10.1.10.2 nd6 options=29 media: Ethernet autoselect (1000baseT ) status: active em2: flags=8843 metric 0 mtu 1500 options=4219b ether 68:05:ca:13:74:2a inet 10.1.1.2 netmask 0x broadcast 10.1.255.255 inet6 fe80::6a05:caff:fe13:742a%em2 prefixlen 64 scopeid 0x3 inet 10.1.10.3 netmask 0x broadcast 10.1.10.3 inet 10.1.10.1 netmask 0x broadcast 10.1.10.1 inet 10.1.10.8 netmask 0x broadcast 10.1.10.8 inet 10.1.10.10 netmask 0x broadcast 10.1.10.10 inet 10.1.10.4 netmask 0x broadcast 10.1.10.4 inet 10.1.10.9 netmask 0x broadcast 10.1.10.9 inet 10.1.10.7 netmask 0x broadcast 10.1.10.7 nd6 options=29 media: Ethernet autoselect (1000baseT ) status: active Let me know if any more detail would be helpful or if you have any ideas of things to check. Thanks, Preston ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
