Hello,
I've a L4 farm with persistancy enabled. On the backend servers, we can
collect errors when a client arrives thinking its authenticated while
it's not. (so when it has been switched from one server to another one)
We see almost none of these error for a while, and then sometimes, we
have plenty at the same time, just like sometimes all persistancy is lost.
I've received the info that yesterday between 10h19 and 11h19 there were
a lot of errors. I've checked at the logs on ZLB, and I see in
zenloadbalancer.log that at at 10h29 there were some action on this farm.
These actions are "running 'Stop write false' for ZLB-ULG farm l4xnat"
and "running 'Start write false' for ZLB-ULG farm l4xnat" (see attached
file).
I've seen that farmguardian as detected a backend being down, and being
back up again afterward. Though this is great and i'll check with the
owner of the backend to fix this, i'm concerned about loosing all
persistancy when farmguardian remove/add a backend.
I've 2 backends and when farmguardian remove one of them, it in fact
deletes all iptables entry for this farm, and re-added only the ones for
the alive backend. This is fine, having only 2 backend, I can live with
that (if I would have more, that would be the same problem as below =>
all persistancy is lost)
When the backend comes back alive, again all iptables rules are deleted
and re-added for both backend. This is bad, because while running with 1
backend, persistancy has attached all users to that backend, but when
the 2nd backend joins back, all these persistancy is lost and all users
are splitted on both backend. Which in our case means a disconnection
for half of them.
I was expecting that only new connections would be associated with the
new joining backend, and all other remains on the first backend. This
way, there would be no disruption.
I can imagine that it's probably easier to remove and re-add everything,
but is there any way to keep the persistancy? Maybe before you re-add a
backend coming back alive, you could dump the /proc/file/xt_recent/ file
associated with the running backend to re-inject the associations back
while you re-add the iptables entry?
If not, what other way could you suggest?
This make me thing about an enhancement for farmguardian for next
version, which would be "consider the backend as down only if X
consecutives checks fails and not only one".
Thanks
tibz
Thu Jun 5 10:29:16 2014 - * - 172.26.0.210 - admin - running 'Stop write
false' for ZLB-ULG farm l4xnat
Thu Jun 5 10:29:16 2014 - * - 172.26.0.210 - admin - deleteIptRules:: running
'/sbin/iptables -t mangle -D PREROUTING 4'
Thu Jun 5 10:29:16 2014 - * - 172.26.0.210 - admin - deleteIptRules:: delete
netfilter rule '4 5566K 2435M MARK tcp -- * * 0.0.0.0/0
192.168.135.18 recent: CHECK seconds: 14400 name:
_ZLB-ULG_0x203_sessions side: source multiport dports 0:65535 /*
FARM_ZLB-ULG_1_ */ MARK set 0x203
'
Thu Jun 5 10:29:16 2014 - * - 172.26.0.210 - admin - deleteIptRules:: running
'/sbin/iptables -t mangle -D PREROUTING 3'
Thu Jun 5 10:29:16 2014 - * - 172.26.0.210 - admin - deleteIptRules:: delete
netfilter rule '3 5589K 2443M MARK tcp -- * * 0.0.0.0/0
192.168.135.18 recent: CHECK seconds: 14400 name:
_ZLB-ULG_0x202_sessions side: source multiport dports 0:65535 /*
FARM_ZLB-ULG_0_ */ MARK set 0x202
'
Thu Jun 5 10:29:16 2014 - * - 172.26.0.210 - admin - deleteIptRules:: running
'/sbin/iptables -t mangle -D PREROUTING 2'
Thu Jun 5 10:29:16 2014 - * - 172.26.0.210 - admin - deleteIptRules:: delete
netfilter rule '2 5740K 2558M MARK tcp -- * * 0.0.0.0/0
192.168.135.18 statistic mode random probability 0.500000
multiport dports 0:65535 /* FARM_ZLB-ULG_0_ */ MARK set 0x202
'
Thu Jun 5 10:29:16 2014 - * - 172.26.0.210 - admin - deleteIptRules:: running
'/sbin/iptables -t mangle -D PREROUTING 1'
Thu Jun 5 10:29:16 2014 - * - 172.26.0.210 - admin - deleteIptRules:: delete
netfilter rule '1 11M 5119M MARK tcp -- * * 0.0.0.0/0
192.168.135.18 statistic mode random probability 1.000000
multiport dports 0:65535 /* FARM_ZLB-ULG_1_ */ MARK set 0x203
'
Thu Jun 5 10:29:16 2014 - * - 172.26.0.210 - admin - deleteIptRules:: running
'/sbin/iptables -t nat -D PREROUTING 2'
Thu Jun 5 10:29:16 2014 - * - 172.26.0.210 - admin - deleteIptRules:: delete
netfilter rule '2 201K 9736K DNAT tcp -- * * 0.0.0.0/0
0.0.0.0/0 mark match 0x203 recent: SET name:
_ZLB-ULG_0x203_sessions side: source /* FARM_ZLB-ULG_1_ */ to:192.168.118.13
'
Thu Jun 5 10:29:16 2014 - * - 172.26.0.210 - admin - deleteIptRules:: running
'/sbin/iptables -t nat -D PREROUTING 1'
Thu Jun 5 10:29:16 2014 - * - 172.26.0.210 - admin - deleteIptRules:: delete
netfilter rule '1 204K 9899K DNAT tcp -- * * 0.0.0.0/0
0.0.0.0/0 mark match 0x202 recent: SET name:
_ZLB-ULG_0x202_sessions side: source /* FARM_ZLB-ULG_0_ */ to:192.168.118.11
'
Thu Jun 5 10:29:16 2014 - * - 172.26.0.210 - admin - running 'Start write
false' for ZLB-ULG farm l4xnat
Thu Jun 5 10:29:16 2014 - * - 172.26.0.210 - admin - running /sbin/iptables -t
mangle -A PREROUTING -m statistic --mode random --probability 1 -d
192.168.135.18 -p tcp -m multiport --dports 0:65535 -j MARK --set-mark 0x202 -m
comment --comment ' FARM_ZLB-ULG_0_ '
Thu Jun 5 10:29:16 2014 - * - 172.26.0.210 - admin - running /sbin/iptables -t
mangle -A PREROUTING -m recent --name "_ZLB-ULG_0x202_sessions" --rcheck
--seconds 14400 -d 192.168.135.18 -p tcp -m multiport --dports 0:65535 -j MARK
--set-mark 0x202 -m comment --comment ' FARM_ZLB-ULG_0_ '
Thu Jun 5 10:29:16 2014 - * - 172.26.0.210 - admin - running /sbin/iptables -t
nat -A PREROUTING -m mark --mark 0x202 -j DNAT -p tcp --to-destination
192.168.118.11 -m recent --name "_ZLB-ULG_0x202_sessions" --set -m comment
--comment ' FARM_ZLB-ULG_0_ '
Thu Jun 5 10:29:16 2014 - * - 172.26.0.210 - admin - setting true to IP
forwarding
Thu Jun 5 10:29:26 2014 - * - 172.26.0.210 - admin - running 'Stop write
false' for ZLB-ULG farm l4xnat
Thu Jun 5 10:29:26 2014 - * - 172.26.0.210 - admin - deleteIptRules:: running
'/sbin/iptables -t mangle -D PREROUTING 4'
Thu Jun 5 10:29:26 2014 - * - 172.26.0.210 - admin - deleteIptRules:: delete
netfilter rule '4 557 236K MARK tcp -- * * 0.0.0.0/0
192.168.135.18 recent: CHECK seconds: 14400 name:
_ZLB-ULG_0x202_sessions side: source multiport dports 0:65535 /*
FARM_ZLB-ULG_0_ */ MARK set 0x202
'
Thu Jun 5 10:29:26 2014 - * - 172.26.0.210 - admin - deleteIptRules:: running
'/sbin/iptables -t mangle -D PREROUTING 3'
Thu Jun 5 10:29:26 2014 - * - 172.26.0.210 - admin - deleteIptRules:: delete
netfilter rule '3 1240 542K MARK tcp -- * * 0.0.0.0/0
192.168.135.18 statistic mode random probability 1.000000
multiport dports 0:65535 /* FARM_ZLB-ULG_0_ */ MARK set 0x202
'
Thu Jun 5 10:29:26 2014 - * - 172.26.0.210 - admin - deleteIptRules:: running
'/sbin/iptables -t nat -D PREROUTING 3'
Thu Jun 5 10:29:26 2014 - * - 172.26.0.210 - admin - deleteIptRules:: delete
netfilter rule '3 58 2788 DNAT tcp -- * * 0.0.0.0/0
0.0.0.0/0 mark match 0x202 recent: SET name:
_ZLB-ULG_0x202_sessions side: source /* FARM_ZLB-ULG_0_ */ to:192.168.118.11
'
Thu Jun 5 10:29:26 2014 - * - 172.26.0.210 - admin - running 'Start write
false' for ZLB-ULG farm l4xnat
Thu Jun 5 10:29:26 2014 - * - 172.26.0.210 - admin - running /sbin/iptables -t
mangle -A PREROUTING -m statistic --mode random --probability 1 -d
192.168.135.18 -p tcp -m multiport --dports 0:65535 -j MARK --set-mark 0x203 -m
comment --comment ' FARM_ZLB-ULG_1_ '
Thu Jun 5 10:29:26 2014 - * - 172.26.0.210 - admin - running /sbin/iptables -t
mangle -A PREROUTING -m statistic --mode random --probability 0.5 -d
192.168.135.18 -p tcp -m multiport --dports 0:65535 -j MARK --set-mark 0x202 -m
comment --comment ' FARM_ZLB-ULG_0_ '
Thu Jun 5 10:29:26 2014 - * - 172.26.0.210 - admin - running /sbin/iptables -t
mangle -A PREROUTING -m recent --name "_ZLB-ULG_0x202_sessions" --rcheck
--seconds 14400 -d 192.168.135.18 -p tcp -m multiport --dports 0:65535 -j MARK
--set-mark 0x202 -m comment --comment ' FARM_ZLB-ULG_0_ '
Thu Jun 5 10:29:26 2014 - * - 172.26.0.210 - admin - running /sbin/iptables -t
mangle -A PREROUTING -m recent --name "_ZLB-ULG_0x203_sessions" --rcheck
--seconds 14400 -d 192.168.135.18 -p tcp -m multiport --dports 0:65535 -j MARK
--set-mark 0x203 -m comment --comment ' FARM_ZLB-ULG_1_ '
Thu Jun 5 10:29:26 2014 - * - 172.26.0.210 - admin - running /sbin/iptables -t
nat -A PREROUTING -m mark --mark 0x202 -j DNAT -p tcp --to-destination
192.168.118.11 -m recent --name "_ZLB-ULG_0x202_sessions" --set -m comment
--comment ' FARM_ZLB-ULG_0_ '
Thu Jun 5 10:29:26 2014 - * - 172.26.0.210 - admin - running /sbin/iptables -t
nat -A PREROUTING -m mark --mark 0x203 -j DNAT -p tcp --to-destination
192.168.118.13 -m recent --name "_ZLB-ULG_0x203_sessions" --set -m comment
--comment ' FARM_ZLB-ULG_1_ '
Thu Jun 5 10:29:26 2014 - * - 172.26.0.210 - admin - setting true to IP
forwarding
------------------------------------------------------------------------------
Learn Graph Databases - Download FREE O'Reilly Book
"Graph Databases" is the definitive new guide to graph databases and their
applications. Written by three acclaimed leaders in the field,
this first edition is now available. Download your free book today!
http://p.sf.net/sfu/NeoTech
_______________________________________________
Zenloadbalancer-support mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/zenloadbalancer-support
