That sounds as if your LVS server is dropping the connections, possibly because
they are idle for too long.
What I suggest is that you add something like
logger -t vmststat vmstat 60
to your machines so that they log at least one log per minute (to keep things
like firewalls and LVS from timing out the connection), and I think you'll find
it's useful data to have in the system
If you use clusterlabs.org for your failover instead of a pair of LVS servers
(which use clusterlabs.org for their failover) you will not have the timeout
problems that you are having with LVS.
I still think trying to bake cluster awareness into the senders is the wrong
thing to do, the senders cannot possibly coordinate to avoid overload on
active-active clusters, or to pick the 'same' system when you want
active-passive clusters.
There are a lot of things that only can be done in rsyslog, clustering is not
one of them. Rsyslog has better cluster support than nxlog and logstash (I don't
know about syslog-ng, haven't had to try with it yet :-) it just takes using the
right tool for the job rather than expecting the loging softwre to do it all.
remember the unix approach, do one thing and do it well, assemble systems from
tools that take this approach.
Clustering is rather complex topic. A great book on the topic is "In Search of
Clusters"
David Lang
On Mon, 12 Oct 2015, chenlin rao wrote:
Is this suggest any progress?
I find my rsyslog server behind LVS output many error logs like
"rsyslogd-2165: netstream session 0x7ff978739ce0 from 172.16.140.13 will be
closed due to error [v8.11.0 try http://www.rsyslog.com/e/2165 ]". If I
set the omfwd target of clients to only one server, the server wouldn't
output such message.
Don't sure where the problem is. But I remember this email, and think a
cluster support in client side maybe a good choice.
2015-06-05 1:53 GMT+08:00 David Lang <da...@lang.hm>:
On Thu, 4 Jun 2015, Brian Knox wrote:
I've been pondering other discovery support. So far, I've been wary of
introducing additional dependencies. There's so many discovery services
that are en vogue right now (zookeeper, etcd, consul...) and every
additional protocol supported is more overhead from a support and
maintenance standpoint.
This is exactly the problem.
If we start implementing specific discovery protocols, they will have to
be maintained for a long time, even when the fad fades and it's not a
popular thing any longer.
As far as I know, all of these support using name resolution for discovery
(either DNS or tweaking /etc/hosts) as well as their various proprietary
protocols, so as long as rsyslog does a name resolution check every time it
attempts to establish a new connection, these discovery tools could be
used. DNS propogation across the Internet is slow, but when used within
your own network you can disable all caching or set the cache to something
very short. Normal round-robin DNS (listing multiple IPs for one name)
should result in queries returning different answers even when caching is
involved.
So, Rsyslog will connect to whatever the service discovery system says is
correct, and it can be configured to reconnect every X messages (and it
will reconnect if it gets disconnected due to any error)
This should mean nothing is needed on the initiating (client) side.
On the receiving (server) side, there shouldn't be anything that needs to
happen either. The service discovery system needs to setup a health check
of some kind to decide if rsyslog is healthy or not, and if it is, it is
part of the cluster, if not, it's cut out. This is no different than using
any other load balancer (it can be "does it accept a TCP connection", or
"does it accept a message", or "does it recieve a message and output it to
a specific destination" depending on how paranoid you are)
unless rsyslog is caching name lookups for outbound connections, there
should be nothing that needs to be changed in rsyslog.
David Lang
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
