Re: Problem with ipfw, in-kernel NAT and port redirection to jails
08.02.2016 12:30, Kiryanov Vassily пишет: Hello Alexey, Thank you for this information, I have thoughts about using pf nat as an alternative way and your example will be useful for me. But Eugene Grosbein adviced me to turn off tso4 on network card underlaying my VLANs and it was enough to solve problem with port redirection. Without turning tso4 off ipfw + in-kernel NAT works fine but port redirection fails. Thank you. It's my mistake - was confused by home gateway, where redirect_port worked perfectly (NIC without TSO support), and there is a notice in section BUGS of ipfw(8) about incompatible libalias and TSO. ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re: Problem with ipfw, in-kernel NAT and port redirection to jails
On 8/02/2016 9:27 PM, Alexey Roslyakov via freebsd-net wrote: 08.02.2016 12:30, Kiryanov Vassily пишет: Hello Alexey, Thank you for this information, I have thoughts about using pf nat as an alternative way and your example will be useful for me. But Eugene Grosbein adviced me to turn off tso4 on network card underlaying my VLANs and it was enough to solve problem with port redirection. Without turning tso4 off ipfw + in-kernel NAT works fine but port redirection fails. Thank you. It's my mistake - was confused by home gateway, where redirect_port worked perfectly (NIC without TSO support), and there is a notice in section BUGS of ipfw(8) about incompatible libalias and TSO. so why are you using libalias? I may have misread what you are doing but IP masquerading might work better. (ipfw fwd rule with local destination) ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org" ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re[2]: Problem with ipfw, in-kernel NAT and port redirection to jails
Hello Alexey,
Thank you for this information, I have thoughts about using pf nat as
an alternative way and your example will be useful for me.
But Eugene Grosbein adviced me to turn off tso4 on network card
underlaying my VLANs and it was enough to solve problem with port
redirection. Without turning tso4 off ipfw + in-kernel NAT works
fine but port redirection fails.
Saturday, February 6, 2016, 1:57:47 PM, you wrote:
ARvfn> Hello.
ARvfn> I have same problem when I'm trying redirect incoming traffic into the
ARvfn> jailed web server.
ARvfn> I repeated my installation few times on different releases - problem
ARvfn> with redirected ports was here all time (except 9.3 - there was random
ARvfn> result).
ARvfn> As a temporary solution am using pf nat for redirect ports.
ARvfn> My test configuration:
ARvfn> /etc/rc.conf:
ARvfn> ifconfig_vtnet0="inet 192.168.1.18/24"
ARvfn> defaultrouter="192.168.1.1"
ARvfn> cloned_interfaces="lo1"
ARvfn> /etc/jail.conf:
ARvfn> exec.start = "/bin/sh /etc/rc";
ARvfn> exec.stop = "/bin/sh /etc/rc.shutdown";
ARvfn> exec.clean;
ARvfn> j1 {
ARvfn> path = /home/jail1;
ARvfn> mount.devfs;
ARvfn> host.hostname = j1;
ARvfn> interface = "lo1";
ARvfn> ip4.addr = 10.8.0.1;
ARvfn> persist;
ARvfn> }
ARvfn> rc.firewall:
ARvfn> ipfw nat 1 config if vtnet0 redirect_port tcp 10.8.0.1:80 80
ARvfn> ipfw add 500 nat 1 ip from any to 192.168.1.18 in via vtnet0
ARvfn> ipfw add 600 nat 1 ip from 10.8.0.1 to any out via vtnet0
ARvfn> ipfw add allow ip from any to any
ARvfn> pf.conf:
ARvfn> ext_if = "vtnet0"
ARvfn> int_if = "lo1"
ARvfn> jail_net = $int_if:network
ARvfn> nat on $ext_if from $jail_net to any -> ($ext_if)
ARvfn> rdr pass on $ext_if inet proto tcp from any to ($ext_if:0) port 80 ->
ARvfn> 10.8.0.1 port 80
ARvfn> In jail I'm try nginx, apache24 and nc as source for redirection. Test
ARvfn> file was generated: dd if/dev/random of=tmp.raw bs=1M count=2
ARvfn> On 10.1 and 10.2 there is no big differences, when using ipfw nat we can
ARvfn> get only part of file (I'm using curl on different machine: curl
ARvfn> http://192.168.1.18/tmp.raw > /dev/null):
ARvfn> with nginx: Received = 33045
ARvfn> with apache: Received = 33092
ARvfn> with nc: Received = 16384
ARvfn> and result seems to be very stable in numbers.
ARvfn> On 9.3:
ARvfn> nginx: random bytes received, has no successful downloads
ARvfn> apache: random bytes received, sometimes download entire file
ARvfn> nc: entire file received
ARvfn> My virtual environment is proxmox 3.
ARvfn> Maybe it's related to
ARvfn> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=137346 or just not
ARvfn> properly configured ipfw nat?
ARvfn> ___
ARvfn> freebsd-net@freebsd.org mailing list
ARvfn> https://lists.freebsd.org/mailman/listinfo/freebsd-net
ARvfn> To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
--
Best regards,
Kiryanovmailto:k...@bf.pstu.ru
___
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re: Problem with ipfw, in-kernel NAT and port redirection to jails
Hello.
I have same problem when I'm trying redirect incoming traffic into the
jailed web server.
I repeated my installation few times on different releases - problem
with redirected ports was here all time (except 9.3 - there was random
result).
As a temporary solution am using pf nat for redirect ports.
My test configuration:
/etc/rc.conf:
ifconfig_vtnet0="inet 192.168.1.18/24"
defaultrouter="192.168.1.1"
cloned_interfaces="lo1"
/etc/jail.conf:
exec.start = "/bin/sh /etc/rc";
exec.stop = "/bin/sh /etc/rc.shutdown";
exec.clean;
j1 {
path = /home/jail1;
mount.devfs;
host.hostname = j1;
interface = "lo1";
ip4.addr = 10.8.0.1;
persist;
}
rc.firewall:
ipfw nat 1 config if vtnet0 redirect_port tcp 10.8.0.1:80 80
ipfw add 500 nat 1 ip from any to 192.168.1.18 in via vtnet0
ipfw add 600 nat 1 ip from 10.8.0.1 to any out via vtnet0
ipfw add allow ip from any to any
pf.conf:
ext_if = "vtnet0"
int_if = "lo1"
jail_net = $int_if:network
nat on $ext_if from $jail_net to any -> ($ext_if)
rdr pass on $ext_if inet proto tcp from any to ($ext_if:0) port 80 ->
10.8.0.1 port 80
In jail I'm try nginx, apache24 and nc as source for redirection. Test
file was generated: dd if/dev/random of=tmp.raw bs=1M count=2
On 10.1 and 10.2 there is no big differences, when using ipfw nat we can
get only part of file (I'm using curl on different machine: curl
http://192.168.1.18/tmp.raw > /dev/null):
with nginx: Received = 33045
with apache: Received = 33092
with nc: Received = 16384
and result seems to be very stable in numbers.
On 9.3:
nginx: random bytes received, has no successful downloads
apache: random bytes received, sometimes download entire file
nc: entire file received
My virtual environment is proxmox 3.
Maybe it's related to
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=137346 or just not
properly configured ipfw nat?
___
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Problem with ipfw, in-kernel NAT and port redirection to jails
Hello, I have a vps on vultr.com http://vultr.com/ running FreeBSD 10.1-p9 and a generic kernel: % uname -a FreeBSD tzar 10.1-RELEASE-p9 FreeBSD 10.1-RELEASE-p9 #0: Tue Apr 7 01:09:46 UTC 2015 r...@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64 My goal is to run multiple services in jails (hopefully using ezjail or other convenient manager) and make them accessible from the Internet only on arbitrary ports (like 80 for http(s) server). So far my approach is as follows: I clone the lo0 interface and assign IP from 127.0.0.0/8 space to the jail and redirect port in ipfw nat definition to given address (example in configs below, I tried also with other addresses on vtnet0, which is my base network interface, with similar issues). Unfortunately this configuration doesn’t work for me. I tested this for znc (IRC bouncer) and nginx. If I run them on main host (without NAT in front of them) everything works fine. However, if I run them in jail, behind NAT and send a HTTP(S) request to get some file, connections get dropped (znc has a web admin module, which is broken because of that). It works fine for small files but breaks for larger (I haven’t check the threshold but can do this if this is necessary). For example given curl command and znc service: % curl http://$my_ip:6697/pub/jquery-1.11.2.min.js /dev/null # (stats cut out) curl: (18) transfer closed with 58648 bytes remaining to read if I tcpdump connection, transfer looks fine for some time and then ends with a following sequence (run on main host, jail master): % sudo tcpdump port 6697 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on vtnet0, link-type EN10MB (Ethernet), capture size 65535 bytes 23:37:28.621409 IP $my_home_host.51256 $my_vultr_host.vultr.com.6697: Flags [S], seq 3967654146, win 65535, options [mss 1460,nop,wscale 5,nop,nop,TS val 601353780 ecr 0,sackOK,eol], length 0 23:37:28.621468 IP $my_vultr_host.vultr.com.6697 $my_home_host.51256: Flags [S.], seq 517055725, ack 3967654147, win 65535, options [mss 1460,nop,wscale 6,sackOK,TS val 2553669008 ecr 601353780], length 0 23:37:28.635788 IP $my_home_host.51256 $my_vultr_host.vultr.com.6697: Flags [.], ack 1, win 4117, options [nop,nop,TS val 601353791 ecr 2553669008], length 0 23:37:28.635865 IP $my_home_host.51256 $my_vultr_host.vultr.com.6697: Flags [P.], seq 1:109, ack 1, win 4117, options [nop,nop,TS val 601353791 ecr 2553669008], length 108 23:37:28.636122 IP $my_vultr_host.vultr.com.6697 $my_home_host.51256: Flags [P.], seq 1:18, ack 109, win 1040, options [nop,nop,TS val 2553669022 ecr 601353791], length 17 23:37:28.650153 IP $my_home_host.51256 $my_vultr_host.vultr.com.6697: Flags [.], ack 18, win 4117, options [nop,nop,TS val 601353805 ecr 2553669022], length 0 23:37:29.123244 IP $my_vultr_host.vultr.com.6697 $my_home_host.51256: Flags [.], seq 18:1466, ack 109, win 1040, options [nop,nop,TS val 2553669510 ecr 601353805], length 1448 (transfer goes normally) 23:37:35.519163 IP $my_vultr_host.vultr.com.6697 $my_home_host.51256: Flags [F.], seq 37666, ack 109, win 1040, options [nop,nop,TS val 2553675906 ecr 601360615], length 0 23:37:35.531004 IP $my_home_host.51256 $my_vultr_host.vultr.com.6697: Flags [.], ack 33322, win 4096, options [nop,nop,TS val 601360640 ecr 2553675880], length 0 23:37:36.165352 IP $my_vultr_host.vultr.com.6697 $my_home_host.51256: Flags [.], seq 33322:34770, ack 109, win 1040, options [nop,nop,TS val 2553676552 ecr 601360640], length 1448 23:37:36.184582 IP $my_home_host.51256 $my_vultr_host.vultr.com.6697: Flags [.], ack 34770, win 4050, options [nop,nop,TS val 601361283 ecr 2553676552], length 0 23:37:36.801437 IP $my_vultr_host.vultr.com.6697 $my_home_host.51256: Flags [.], seq 34770:36218, ack 109, win 1040, options [nop,nop,TS val 2553677188 ecr 601361283], length 1448 23:37:36.910742 IP $my_home_host.51256 $my_vultr_host.vultr.com.6697: Flags [.], ack 36218, win 4050, options [nop,nop,TS val 601362012 ecr 2553677188], length 0 23:37:36.910796 IP $my_vultr_host.vultr.com.6697 $my_home_host.51256: Flags [FP.], seq 36218:37666, ack 109, win 1040, options [nop,nop,TS val 2553677297 ecr 601362012], length 1448 23:37:36.922685 IP $my_home_host.51256 $my_vultr_host.vultr.com.6697: Flags [F.], seq 109, ack 37667, win 4096, options [nop,nop,TS val 601362025 ecr 2553677297], length 0 23:37:36.922742 IP $my_vultr_host.vultr.com.6697 $my_home_host.51256: Flags [.], ack 110, win 1040, options [nop,nop,TS val 2553677309 ecr 601362025], length 0 My ipfw log doesn’t show any rejected packages in this case. For comparison when I run service on the main host (without NAT and port redirection) sending transfer is longer and ending sequence looks like follows: % sudo tcpdump port 6696 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on vtnet0, link-type EN10MB (Ethernet), capture size 65535 bytes
