Thanks for your reply ! On 12/20/2011 09:16 PM, [email protected] wrote: > On Tue, 20 Dec 2011, David Donchez wrote: > >> Hi All, >> >> I'm experiencing Rsyslog 5.8.1 with RELP Module on a Debian Squeeze x64 >> system. >> >> My architecture est running fine but i have some question about the RELP >> features. >> >> I will explains my setup : >> >> ServerA -> RELP -> Ingestor Server -> RELP -> 3 Remote Server. >> >> My issue is : >> >> If one of the 3 last remote server is crashed (e.g. full disk), All the >> chain is slow and the server begin to queue all syslog messages. Is >> there a way to continue to push to others servers? >> I have seen with strace that the ingestor seems to be locked on a select >> (Service unavailable from the remote server). It will be fine if rsyslog >> can push to all others server even if one is crashed > > the simple answer is that you can do this, but you need to configure > rsyslog use separate action queues for each server so that only the > one queue gets stalled (and then you have to worry about what happens > when that queue fills up) Hmm i think i already use 3 queues for the 3 destinations. I will investigate on this way, but i can put my config :
## ## Global conf ## $WorkDirectory /var/spool/rsyslog/archive $ActionQueueType LinkedList $ActionQueueFileName rsyslog_archive $ActionQueueMaxDiskSpace 10g $ActionResumeRetryCount -1 $ActionQueueSaveOnShutdown on if $fromhost-ip startswith '10.' and $programname startswith 'TEST.' then :omrelp:10.119.0.3:514;RSYSLOG_ForwardFormat & ~ $WorkDirectory /var/spool/rsyslog/dispatcher $ActionQueueType LinkedList $ActionQueueFileName rsyslog_dispatcher $ActionQueueMaxDiskSpace 10g $ActionResumeRetryCount -1 $ActionQueueSaveOnShutdown on #if $fromhost-ip startswith '10.' and $programname startswith 'TEST.' then :omrelp:10.119.0.10:20514 #& ~ $WorkDirectory /var/spool/rsyslog/system $ActionQueueType LinkedList $ActionQueueFileName rsyslog_system $ActionQueueMaxDiskSpace 10g $ActionResumeRetryCount -1 $ActionQueueSaveOnShutdown on if ( $fromhost-ip startswith '10.' and not ( $programname startswith 'TEST.' )) then :omrelp:10.119.0.4:514;RSYSLOG_ForwardFormat & ~ Please let me know if there is a mistake in this configuration. > > What is the purpose of sending to the multiple servers? is there a way > of dealing with the problem on the server side? In fact, i send the logs to 3 servers : - One which is the head of a parsing software to generate statistics - One which archives theses logs in a gzip format - One which only store system logs > >> Also i have another question about file archiving. >> >> (One of the 3 Remove server above is a archive server). >> >> I archive all logs in files. >> >> $omfileziplevel 9 >> $template >> Service,"/mnt/space/%$YEAR%-%$MONTH%-%$DAY%/%programname%/%HOSTNAME%/TEST.log.gz" >> >> >> if $programname startswith 'TEST.' then ?Service >> & ~ >> >> >> This setup work fine. But if a server is queuing for 3 days, then push >> his queue, all logs will go in the "CURRENT DAY" directory on this >> archive server ( Due to the template %$YEAR%-%$MONTH%-%$DAY%) >> Is there a way to archive the log in the directory corresponding to the >> generated date file ? >> >> I have look to the "timegenerated" variables but it seems to be a little >> bit complicated. > > $YEAR $MINTH $DAY are the current time on the server as the log is > being processed. you will need to work with the $timegenerated > variables, and yes they are ugly, they are what is provided by the > machine sending the log (or if that machine doesn't put a timestamp in > it, by the first machine to receive and process the log) If you are > lucky enough for all your machines to use the same format for their > timestamps, then you should be able to use regex and field processing > to extract the values you want from that field. Yes, finally, i have use this ugly method like : %timereported:1:4:date-mysql%-%timereported:5:6:date-mysql%-%timereported:7:8:date-mysql% to have a generated date like YYYY-MM-DD > > David Lang > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/
