In message <[email protected]>, Clayton Dukes writes: >We would like to set up our servers in a high availability architecture >where 2 servers would send syslog messages to two receiving servers. >I have proposed that we utilize SEC on the two receiving servers as depicted >in the image below to monitor for any duplicate messages and discard the >duplicate. This would ensure delivery in the event that one of the servers >were to fail. Can SEC provide this?
You can see a slight discussion on redundancy (a few paragraphs) in http://www.cs.umb.edu/~rouilj/sec/sec_paper_full.pdf section 4.4. In that I recommend a heartbeat senario so only one SEC has its rules enabled. I did test that setup at one point and it worked pretty well (although it did take about 35 seconds worst case for the heartbeat (sent every 30 seconds) to fail, be detected and the second SEC instance to come on line. The ruleset: http://www.cs.umb.edu/~rouilj/sec/rulesets/01report_myself_only.sr may be of some use. It's meant to use my parallel rules framework, but the basic idea is: if the context filter_MYHOST.example.com exists, capture all the inbound events and signal to all the rest of the rules to ignore the event by creating the EVENT_PROCESSED context. (This could be done using the jump commands in newer SEC's much more easily.) in the redundant SEC, create the context filter_MYHOST.example.com when the heartbeat comes in with a lifetime of heartbeat interval + 1 second. the heartbeat event could be sent over a shared file (nfs...), netcat etc. On the primary/master set up a command that fires every heartbeat interval and writes the heartbeat event to the transfer mechanism. (IF the hearbeat should be less than a minute, you can use a calendar rule to create contexts with appropriate lifetimes say: 15, 30, 45 seconds that generate the heartbeat message when the expire). I could see this being adapted so that rather then heartbeat's being sent you send the actual events and turn them into short lived contexts. You would want to use an input channel that has a context associated with it. E.G. on the SEC command line -input=/tmp/heartbeats=heartbeats and place this rule first: type = single ptype = regexp pattern = (.*) context = heartbeats action = create $1 2 to create a context that can be matched against an event for 2 seconds. This only looks at events arriving over the heartbeat channel, and it consumes them so they are not passed on. Then the following rule should be applied to every event comming from syslog-ng: type = single ptype = regexp pattern = (.*) context = $1 action = none would capture and suppress the event from propigating through the redundant sec. However this assumes a couple of things: the master gets the event to the redundat system before the message arrives at the redundant system. If not the message gets emitted twice, once from each sec. that the delay between the two systems seeing the message is < 2. if the redundant system gets the message 3 seconds later it will process it. However one thing that I never got working right was duplicating the state of the two SECS so that they would have the same internal state and if one failed in the middle of a threshold count the redundant node would pick up counting where the first one died. Also in your senario it looks like you want multiple masters not just a master/redundant backup, which is very tricky and I can sort of see how I would do it, but there are a lot of timing and other details to deal with, and I am not sure I would try doing that. If you can afford to lose a few messages that are in transit it would be easier. Once other thing that could be done is to have syslog-ng feed a (custom written) queue that preserves inter-event timing. Then have the queue delay feeding the event by heartbeat seconds to the redundant sec. Think of the 4 second delay in broadcasting. If the heartbeat fails during the queue delay time, all events over the heartbeat delay time are stored in the queue and ready for replay. You still have the lack of internal SEC state to deal with, but this should preserve all the events. -- -- rouilj John Rouillard =========================================================================== My employers don't acknowledge my existence much less my opinions. ------------------------------------------------------------------------------ This SF.net email is sponsored by Make an app they can't live without Enter the BlackBerry Developer Challenge http://p.sf.net/sfu/RIM-dev2dev _______________________________________________ Simple-evcorr-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
