I've made a handful of changes based up on Chris and Willy's suggestions, which I've included below. This avoids the nf_conntrack errors in the logs.
I would like to skip nf_conntrack altogether. I've been digging around to try to learn how to do that, but I now admit I don't know how. I can't just drop the module, as it's currently in use. [mmar...@w1 w1]$ sudo modprobe -n -r nf_conntrack FATAL: Module nf_conntrack is in use. What do I need to change in my iptables rules to pave the way for removing this module. Once I've got that straight, how do I then disable the module. I'm happy to get an RTFM response if I'm just being stupid. Point me at the right M ;) Michael Marano ---- iptables rules script --------------- #!/bin/sh sudo /sbin/iptables -F sudo /sbin/iptables -A INPUT -i lo -j ACCEPT sudo /sbin/iptables -A INPUT -i ! lo -d 127.0.0.0/8 -j REJECT sudo /sbin/iptables -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT sudo /sbin/iptables -A OUTPUT -j ACCEPT # don't track incoming or outgoing port 80 sudo /sbin/iptables -t raw -A PREROUTING -p tcp --dport 80 -j NOTRACK sudo /sbin/iptables -t raw -A PREROUTING -p tcp --dport 8080 -j NOTRACK sudo /sbin/iptables -t raw -A PREROUTING -p tcp --dport 81 -j NOTRACK # don't track traffic starting from the private ip sudo /sbin/iptables -t raw -A PREROUTING -p tcp -s 10.176.45.165 -j NOTRACK # these may not actually be useful, but I'm leaving them in. sudo /sbin/iptables -t raw -A OUTPUT -p tcp --sport 80 -j NOTRACK sudo /sbin/iptables -t raw -A OUTPUT -p tcp --sport 8080 -j NOTRACK sudo /sbin/iptables -t raw -A OUTPUT -p tcp --sport 81 -j NOTRACK sudo /sbin/iptables -A INPUT -p tcp -m state --state NEW --dport 22 -j ACCEPT sudo /sbin/iptables -A INPUT -p icmp -m icmp --icmp-type 8 -j ACCEPT sudo /sbin/iptables -A INPUT -j REJECT sudo /sbin/iptables -A FORWARD -j REJECT ---- iptables rules script --------------- ---- additions to sysctl.conf --------------- # # TCP tuning # # from http://agiletesting.blogspot.com/2009/03/haproxy-and-apache-performance-tuni ng.html net.ipv4.tcp_tw_reuse = 1 net.ipv4.ip_local_port_range = 1024 65023 net.ipv4.tcp_max_syn_backlog = 10240 net.ipv4.tcp_max_tw_buckets = 400000 net.ipv4.tcp_max_orphans = 60000 net.ipv4.tcp_synack_retries = 3 net.core.somaxconn = 40000 # from http://serverfault.com/questions/11106/best-linux-network-tuning-tips net.ipv4.route.max_size = 262144 net.ipv4.netfilter.ip_conntrack_tcp_timeout_established = 18000 net.ipv4.neigh.default.gc_thresh1 = 1024 net.ipv4.neigh.default.gc_thresh2 = 2048 net.ipv4.neigh.default.gc_thresh3 = 4096 net.netfilter.nf_conntrack_max = 128000 net.netfilter.nf_conntrack_expect_max = 4096 # additions based on questions to the haproxy mailing list # http://www.mail-archive.com/haproxy@formilux.org/msg01321.html net.ipv4.tcp_timestamps = 1 net.core.netdev_max_backlog = 40000 # these were all lower than the default values already set, so I left them out #net.ipv4.tcp_rmem = 4096 8192 16384 #net.ipv4.tcp_wmem = 4096 8192 16384 #net.ipv4.tcp_mem = 65536 98304 131072 ---- additions to sysctl.conf --------------- > From: <ch...@sargy.co.uk> > Date: Wed, 07 Oct 2009 11:24:23 +0100 > To: Michael Marano <mmar...@futureus.com> > Cc: <haproxy@formilux.org> > Subject: Re: Kernel tuning recommendations > > Here is the adjusted IPv4 settings I use on my haproxy box - I picked > these up from around the web, and they seem to work for me, not that > they are in use on a particularly high volume site currently. > > Chris > > net.ipv4.tcp_tw_reuse = 1 > net.ipv4.ip_local_port_range = 1024 65023 > net.ipv4.tcp_max_syn_backlog = 10240 > net.ipv4.tcp_max_tw_buckets = 400000 > net.ipv4.tcp_max_orphans = 60000 > net.ipv4.tcp_synack_retries = 3 > net.ipv4.tcp_max_syn_backlog = 45000 > net.ipv4.tcp_timestamps = 1 > net.ipv4.tcp_rmem = 4096 8192 16384 > net.ipv4.tcp_wmem = 4096 8192 16384 > net.ipv4.tcp_mem = 65536 98304 131072 > net.core.somaxconn = 40000 > net.core.netdev_max_backlog = 40000 > > > > Quoting Michael Marano <mmar...@futureus.com>: > >> Subsequent load tests proved me wrong. I¹m still getting the nf_conntrack >> messages. Perhaps I¹ve misconfigigured my iptables rules? >> >> >> # bits of /var/log/messages >> >> Oct 6 21:58:40 w1 kernel: [3718555.091684] printk: 2 messages suppressed. >> Oct 6 21:58:40 w1 kernel: [3718555.091705] nf_conntrack: table full, >> dropping packet. >> Oct 6 21:58:41 w1 kernel: [3718290.353966] device eth0 entered promiscuous >> mode >> Oct 6 21:58:43 w1 kernel: [3718558.070993] nf_conntrack: table full, >> dropping packet. >> Oct 6 21:58:44 w1 kernel: [3718559.097679] nf_conntrack: table full, >> dropping packet. >> >> >> I¹ve got this in a shell script: >> >> >> ---- >> #!/bin/sh >> >> sudo /sbin/iptables -F >> sudo /sbin/iptables -A INPUT -i lo -j ACCEPT >> sudo /sbin/iptables -A INPUT -i ! lo -d 127.0.0.0/8 -j REJECT >> sudo /sbin/iptables -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT >> sudo /sbin/iptables -A OUTPUT -j ACCEPT >> >> # tell iptables to skip tracking on ports haproxy is monitoring >> sudo /sbin/iptables -t raw -A PREROUTING -p tcp --dport 80 -j NOTRACK >> sudo /sbin/iptables -t raw -A PREROUTING -p tcp --dport 8080 -j NOTRACK >> >> # ... Rules to allow stuff... >> >> sudo /sbin/iptables -A INPUT -j REJECT >> sudo /sbin/iptables -A FORWARD -j REJECT >> ------ >> >> But then when I list my tables, I¹m not seeing anything about the NOTRACK >> rules. >> >> ----- >> Chain INPUT (policy ACCEPT) >> target prot opt source destination >> ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 >> REJECT all -- 0.0.0.0/0 127.0.0.0/8 reject-with >> icmp-port-unreachable >> ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 state >> RELATED,ESTABLISHED >> ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:81 >> ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:80 >> ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:443 >> ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:8080 >> >> ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 state NEW tcp >> dpt:22 >> ACCEPT icmp -- 0.0.0.0/0 0.0.0.0/0 icmp type 8 >> REJECT all -- 0.0.0.0/0 0.0.0.0/0 reject-with >> icmp-port-unreachable >> >> Chain FORWARD (policy ACCEPT) >> target prot opt source destination >> REJECT all -- 0.0.0.0/0 0.0.0.0/0 reject-with >> icmp-port-unreachable >> >> Chain OUTPUT (policy ACCEPT) >> target prot opt source destination >> ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 >> >> Chain RH-Firewall-1-INPUT (0 references) >> target prot opt source destination >> >> ----- >> >> >> >> >> Michael Marano >> >> >> From: Michael Marano <mmar...@futureus.com> >> Date: Tue, 06 Oct 2009 13:49:02 -0700 >> To: Stefan Johansson <phu...@hotmail.com>, <haproxy@formilux.org> >> Conversation: Kernel tuning recommendations >> Subject: Re: Kernel tuning recommendations >> >> Stefan, >> >> That seems to have eliminated any log messages in my staging environment >> under a load test. I think that will do the trick. Thanks for your help. >> >> Any general recommendations for sysctl settings would still be appreciated. >> This is the first time I¹ve had to tune the kernel settings so any guidance >> will help. >> >> Michael Marano >> >> >> From: Stefan Johansson <phu...@hotmail.com> >> Date: Tue, 6 Oct 2009 22:27:49 +0200 >> To: Michael Marano <mmar...@futureus.com>, <haproxy@formilux.org> >> Subject: RE: Kernel tuning recommendations >> >> iptables -t raw -A PREROUTING -p tcp --dport 80 -j NOTRACK >> >> >> > > > > ---------------------------------------------------------------- > This message was sent using IMP, the Internet Messaging Program. >
