We use LVS/Keepalived using DR (Direct Routing) running stock IPVS with the size=4096, we have about 35 servers in a single cluster using LVS to answer for both port 80 and port 443 traffic. We are pushing around 100 Mbps out, and 80 Mbps in currently through our firewalls which is all serving this cluster.
I know with using the DR (Direct Routing) method of balancing that the packet comes into the load balancer initially but doesn't actually see the packet going back out, as it comes directly from the web server. However sometimes we have clients who send us crazy amounts of traffic and we end up having to block the traffic at the load balancer/keepalived server using IPTABLES. We basically configure IPTABLES to send a TCP RESET packet back to the user's browser so that the browser will give up trying to connect to our cluster. This is the command we run on Linux to do the block. /sbin/iptables -A INPUT -p tcp -m tcp --dport 80 -m string --string "sid=3225" --algo bm --to 65535 -j REJECT --reject-with tcp-reset So I know sometimes this messes with the ActiveConn count, and I am not sure if this is affecting LVS in any way as we have to sometimes block a user for an entire day. So the problem I notice with this is that the “Activeconn” count just keeps building up as we are rejecting the traffic for a while but does seem to cap off at 32,000 connections per server right now. I wasn’t sure if there is a way to better let LVS know to consider the connection InActiveConn so it doesn’t use up memory. Because if I am correct in assuming that LVS is consuming some memory for the connection that it see’s as an incoming connection, But because we are doing a TCP RST, LVS doesn’t clear out that connection. Please correct me if I am wrong in my assumption and shouldn’t need to worry?? IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP pixel.host.com:http wrr -> box70.domain:http Route 2 31787 9127 -> box71.domain:http Route 2 32233 9681 -> box48.domain:http Route 2 32266 9658 -> box73.domain:http Route 2 32062 9658 -> box83.domain:http Route 2 32430 9672 .... total of 35 servers listed here. Any ideas would be greatly appreciated here. Would there be any improvement to change the size of the 4096 to something higher? The load balancer server we are using has 4GB of Ram, and Quad Core 2.5Ghz processor, we are not using the box for anything else but to run the "keepalived" process and IPVS. Thanks so much. Devin Acosta _______________________________________________ Please read the documentation before posting - it's available at: http://www.linuxvirtualserver.org/ LinuxVirtualServer.org mailing list - [email protected] Send requests to [email protected] or go to http://lists.graemef.net/mailman/listinfo/lvs-users
