Stefan, could you send the piece of zen logs where the iptables command is launched and failed? It seems that the command is malformed.
Thanks On Thu, Aug 14, 2014 at 12:39 PM, Laura Garcia <[email protected]> wrote: > Good one, let me check it out in our lab. > > Thanks. > > > On Thu, Aug 14, 2014 at 12:24 PM, Stefan <[email protected]> wrote: > >> Hi all, >> >> I have recently been running into the problem that some of my l4xnat >> farms become unreachable every few days until I restart the problem >> farm(s). In the farmguardian logs I found messages like: >> >> iptables: Resource temporarily unavailable. >> iptables: Index of deletion too big. >> >> The above messages appeared at the time of backend status changes when >> farmguardian restarts the farm. >> >> Sometimes the problem farms were then actually shown as down in the >> web interface. >> >> In zenloadbalancer.log I found messages like: >> some date - - - - running 'Start write false' >> ... >> some date - - - - running /sbin/iptables .... >> some date - - - - last command failed! >> ... >> >> The more farms and thus farmguardian instances are running the more >> likely and frequent the problem occurs. And even more likely if >> multiple farms target the same backend server. >> >> So what I figured is that if the backends of multiple farms change >> status at the same time multiple farmguardian instances will restart >> multiple farms i.e. running many iptables commands at the same time >> which will cause some of those commands to fail. >> >> As a quick workaround I modified the farmguardian script to prevent >> multiple farmguardian instances restarting farms at the same time >> (using file locking). The problem seems to have disappeared now. >> >> ... >> use Fcntl qw(:flock); >> ... >> if ($type eq "l4xnat"){ >> if (open(FGA,">/var/run/farmguardian_lock")){ >> my $count = 1; >> while (!flock(FGA,LOCK_EX|LOCK_NB) && ($count < 11) ) { >> print("Restart farm $farmname blocked, waiting... >> $count\n"); >> sleep(1); >> $count++; >> } >> } >> &_runFarmStop($farmname,"false"); >> &setFarmBackendStatus($farmname,$j,"up"); >> &_runFarmStart($farmname,"false"); >> close(FGA); >> } >> ... >> if ($type eq "l4xnat"){ >> if (open(FGA,">/var/run/farmguardian_lock")){ >> my $count = 1; >> while (!flock(FGA,LOCK_EX|LOCK_NB) && ($count < 11) ) { >> print("Restart farm $farmname blocked, waiting... >> $count\n"); >> sleep(1); >> $count++; >> } >> } >> &_runFarmStop($farmname,"false"); >> &setFarmBackendStatus($farmname,$j,"fgDOWN"); >> &_runFarmStart($farmname,"false"); >> close(FGA); >> } >> >> I guess a better place to do something like this would be in >> farms_functions.cgi... Maybe the developers can have a look into this? >> >> Kind Regards, >> Stefan >> >> >> ------------------------------------------------------------------------------ >> _______________________________________________ >> Zenloadbalancer-support mailing list >> [email protected] >> https://lists.sourceforge.net/lists/listinfo/zenloadbalancer-support >> > >
------------------------------------------------------------------------------
_______________________________________________ Zenloadbalancer-support mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/zenloadbalancer-support
