I went looking for the documentation on this and couldn't find it.
the setting is $InputFileReadMode
0 = defult, line based
1 = indented
2 = paragraph
please let me know how it works (or fails) for you.
David Lang
On Fri, 2 Sep 2011, [email protected] wrote:
Date: Fri, 2 Sep 2011 09:06:16 -0700 (PDT)
From: [email protected]
Reply-To: rsyslog-users <[email protected]>
To: rsyslog-users <[email protected]>
Subject: Re: [rsyslog] rsyslog (v 5.8.3) multi-line message support
Yes, I submitted a patch to imfile that allows you to specify what the log
separator is.
default, each line is a new message
indented, like your tomcat messages, a new log entry starts at the beginning
of a line, if a line starts with a space it's part of the log message before
it
paragraph, there is a blank line between log messages.
I don't knwo if this patch was applied to the 5.x series or only the 6.1
series.
when this combines the mulitline messages, it will replace the newline with a
#xxx number, but everything should come out as one line.
If this does not work, please let me know so that we can fix it (I think you
are the first person to as for this since I submitted it)
David Lang
On Fri, 2 Sep 2011, Felts, Tracy wrote:
Date: Fri, 2 Sep 2011 15:28:05 +0000
From: "Felts, Tracy" <[email protected]>
Reply-To: rsyslog-users <[email protected]>
To: "[email protected]" <[email protected]>
Subject: [rsyslog] rsyslog (v 5.8.3) multi-line message support
Does rsyslog support multi-line log messaging such as Tomcat stack traces
or kernel messages? During my testing I'm seeing a single multi-line
message broken up in to single line messages on the rsyslog
receiver/collector (see samples below). I've read through the various
posts in the forum and mailing lists on this topic but haven't come away
with a clear answer. I understand multi-line message support is not a
trivial solution, broken framing, trying to determine what constitutes the
beginning and end of a multi-line message.
I've tried several things read from various posts such as octet-frame
counting (http://kb.monitorware.com/multiline-messages-t10184.html),
disabling control character escaping ($EscapeControlCharactersOnReceive)
with little success. I've read David Lang has been working on multi-line
logging but I'm not sure the status today.
I've included the configurations for two servers I'm using for rsyslog
testing. You can find a multi-line log message sample below the
configurations.
Receiver/Collector/Server rsyslog.conf:
-------------------------------------------------
$EscapeControlCharactersOnReceive off
# Load desired modules.
# for TCP use:
$modload imtcp
# for UDP use:
#$modload imudp
# Provides kernel logging support (previously done by rklogd)
$ModLoad imklog
# Provides support for local system logging (e.g. via logger command)
$ModLoad imuxsock
# Reliable Event Logging Protocol module
$ModLoad imrelp
# Use traditional timestamp format
$ActionFileDefaultTemplate RSYSLOG_SyslogProtocol23Format
$template rawfmt,"%rawmsg%\n"
#$InputTCPServerBindRuleset remote5140
$InputTCPServerRun 5140
$InputRELPServerRun 10514
#$UDPServerRun 514
# Maximum rsyslog message size
$MaxMessageSize 32k
$template
AccessLog,"/var/log/rsyslog/%HOSTNAME%/%$YEAR%-%$MONTH%-%$DAY%/access.log"
$template
SecureLog,"/var/log/rsyslog/%HOSTNAME%/%$YEAR%-%$MONTH%-%$DAY%/secure.log"
$template
MessagesLog,"/var/log/rsyslog/%HOSTNAME%/%$YEAR%-%$MONTH%-%$DAY%/messages.log"
$template
CatalinaLog,"/var/log/rsyslog/%HOSTNAME%/%$YEAR%-%$MONTH%-%$DAY%/catalina.log"
# Log all kernel messages to the console.
# Logging much else clutters up the screen.
#kern.* /dev/console
# Log anything (except mail) of level info or higher.
# Don't log private authentication messages!
*.info;kern.notice;mail.none;authpriv.none;cron.none -/var/log/messages
# The authpriv file has restricted access.
#authpriv.* /var/log/secure
authpriv.*;auth.info /var/log/secure
# Log all the mail messages in one place.
mail.* -/var/log/maillog
# Log cron stuff
cron.* /var/log/cron
# Everybody gets emergency messages
*.emerg *
# Save news errors of level crit and higher in a special file.
uucp,news.crit /var/log/spooler
# Save boot messages also to boot.log
local7.* /var/log/boot.log
local6.*
-/var/opt/silvertail/log/silvertail.log
### RSYSLOG Rules
if $syslogfacility-text == 'local5' and $programname == 'apache' then
-?AccessLog
& ~
if $syslogfacility-text == 'local0' then -?CatalinaLog;rawfmt
& ~
#if $syslogfacility-text == 'local5' and $programname == 'logview' then
-?MessagesLog
#& ~
if $syslogfacility-text == 'authpriv' then -?SecureLog
Client rsyslog.conf:
---------------------------
$EscapeControlCharactersOnReceive off
# Provides kernel logging support (previously done by rklogd)
$ModLoad imklog
# Provides MARK support.
#$ModLoad immark
# Provides support for local system logging (e.g. via logger command)
$ModLoad imuxsock
# File Monitor configs
$ModLoad imfile
# Reliable Event Logging Protocol module
$ModLoad omrelp
# Reliable Event Logging Protocol module
#$ModLoad omhdfs
$RepeatedMsgReduction on
$MaxMessageSize 32k
# Use traditional timestamp format
$ActionFileDefaultTemplate RSYSLOG_SyslogProtocol23Format
$WorkDirectory /var/log/rsyslog/work # default location for work (spool)
files
$ActionQueueType LinkedList # use asynchronous processing
$ActionQueueFileName spool_data # set file name, also enables disk mode
$ActionResumeRetryCount -1 # infinite retries on insert failure
$ActionQueueSaveOnShutdown on # save in-memory data if rsyslog shuts down
$ActionSendResendLastMsgOnReconnect on
# File Monitor configs
#$InputFileName /var/log/messages
$InputFileName /adm/web/logview/logs/catalina.log
$InputFileTag logview:
$InputFileStateFile /var/log/rsyslog-messages.stat
$InputFileSeverity info
$InputFileFacility local0
$InputRunFileMonitor
# Log all kernel messages to the console.
# Logging much else clutters up the screen.
#kern.* /dev/console
# Log anything (except mail) of level info or higher.
# Don't log private authentication messages!
*.info;mail.none;authpriv.none;cron.none /var/log/messages
local5.* :omrelp:10.152.106.24:10514;RSYSLOG_ForwardFormat
&~
local0.* @@(o)10.152.106.24:5140
&~
authpriv.* :omrelp:10.152.106.24:10514;RSYSLOG_ForwardFormat
# Log all the mail messages in one place.
mail.* -/var/log/maillog
# Log cron stuff
cron.* /var/log/cron
# Everybody gets emergency messages
*.emerg *
# Save news errors of level crit and higher in a special file.
uucp,news.crit /var/log/spooler
# Save boot messages also to boot.log
local7.* /var/log/boot.log
Tomcat Message Sample (Original):
--------------------------------------------
2011-08-15 17:24:38,888 [:TP-Processor3] ERROR
org.apache.jk.common.ChannelSocket - Error, processing connection
java.lang.IndexOutOfBoundsException
at java.io.BufferedInputStream.read(BufferedInputStream.java:310)
at org.apache.jk.common.ChannelSocket.read(ChannelSocket.java:620)
at
org.apache.jk.common.ChannelSocket.receive(ChannelSocket.java:577)
at
org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:685)
at
org.apache.jk.common.ChannelSocket$SocketConnection.runIt(ChannelSocket.java:889)
at
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:690)
at java.lang.Thread.run(Thread.java:619)
Tomcat Message Sample (from rsyslog server/collector):
----------------------------------------------------------------------
<134>Sep 1 17:31:57 appserver1 logview: 2011-08-15
17:24:38,888[:TP-Processor3] ERROR org.apache.jk.common.ChannelSocket -
Error, processing connection
<134>Sep 1 17:31:57 appserver1 logview:
java.lang.IndexOutOfBoundsException
<134>Sep 1 17:31:57 appserver1 logview: at
java.io.BufferedInputStream.read(BufferedInputStream.java:310)
<134>Sep 1 17:31:57 appserver1 logview: at
org.apache.jk.common.ChannelSocket.read(ChannelSocket.java:620)
<134>Sep 1 17:31:57 appserver1 logview: at
org.apache.jk.common.ChannelSocket.receive(ChannelSocket.java:577)
<134>Sep 1 17:31:57 appserver1 logview: at
org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:685)
<134>Sep 1 17:31:57 appserver1 logview: at
org.apache.jk.common.ChannelSocket$SocketConnection.runIt(ChannelSocket.java:889)
<134>Sep 1 17:31:57 appserver1 logview: at
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:690)
<134>Sep 1 17:31:57 appserver1 logview: at
java.lang.Thread.run(Thread.java:619)
Please let me know if you need additional information or details from me.
Thanks in advance for your assistance.
Regards,
Tracy
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com
