It is listening and I am told waiting for json formatted messages:
Running tcpdump -s 0 -A -vvv port 10514 produced lots of data. e.g
10:44:25.666605 IP (tos 0x0, ttl 64, id 21488, offset 0, flags [DF], proto TCP
(6), length 1204, bad cksum 0 (->1dea)!)
syslog1.61484 > be-s2508-msl.local.local.10514: Flags [P.], cksum 0x0000
(incorrect -> 0xed99), seq 310693:311845, ack 1, win 64436, options [nop,nop,TS
val 440198685 ecr 2670255102], length 1152
OBFUSCATED {"message":"OBFUSCATED[6422]: [ID 748625 local0.info] LENGTH : '429'
ACTION :[192] 'select
dbname,instname,SUM(READ_TIME),SUM(WRITE_TIME),SUM(READS),SUM(WRITES),SUM(READ_ERRS),SUM(WRITE_ERRS),SUM(BYTES_READ),SUM(BYTES_WRITTEN)
from gv$OBFUSCATED group by dbname,instname","fromhost":" OBFUSCATED
","facility":"local0","priority":"info","timereported":"2018-04-21T08:29:03+02:00","timegenerated":"2018-04-24T10:44:24.125850+02:00"}413
{"message":"OBFUSCATED [6422]: [ID 748625 local0.info] LENGTH : '411' ACTION
:[174] 'select
dbname,SUM(READ_TIME),SUM(WRITE_TIME),SUM(READS),SUM(WRITES),SUM(READ_ERRS),SUM(WRITE_ERRS),SUM(BYTES_READ),SUM(BYTES_WRITTEN)
from gv$OBFUSCATED group by
dbname","fromhost":"OBFUSCATED","facility":"local0","priority":"info","timereported":"2018-04-21T08:29:08+02:00","timegenerated":"2018-04-24T10:44:24.125850+02:00"}431
{"message":" OBFUSCATED [6422]: [ID 748625 local0.info] LENGTH : '429' ACTION
:[192] 'select dbname,instname,SUM(READ_TIME),SUM(WRITE_TIME
),SUM(READS),SUM(WRITES),SUM(READ_ERRS),SUM(WRITE_ERRS),SUM(BYTES_READ),SUM(BYTES_WRITTEN)
from gv$OBFUSCATED group by dbname,instname","fromhost":"OBFUSCATED
10:44:25.670529 IP (tos 0x0, ttl 64, id 18164, offset 0, flags [DF], proto TCP
(6), length 52)
be-s2508-msl.local.local.10514 > syslog1.61484: Flags [.], cksum 0xe363
(correct), seq 1, ack 311845, win 0, options [nop,nop,TS val 2670255351 ecr
440198685], length 0
Best wishes,
Sophie
From: Flo Rance [mailto:[email protected]]
Sent: Tuesday, April 24, 2018 11:23 AM
To: rsyslog-users
Cc: LOEWENTHAL Sophie
Subject: Re: [rsyslog] server received messages but rsyslog did not write them
into a file
Hi,
Are you sure that the service at be-s2507-msl:10514 is expecting TCP connection
and syslog format ?
Flo
On Tue, Apr 24, 2018 at 10:43 AM, sophie.loewenthal--- via rsyslog
<[email protected]<mailto:[email protected]>> wrote:
Hi,
I stripped the config down and it worked after commenting out the json-template
directives in the Remote ruleset shown below:
How can I have this server also forward it's log via the json-templete to
another server?
$ModLoad imsolaris # for Solaris kernel logging
$ModLoad imtcp
$ModLoad imudp
#$MainMsgQueueSize 1000
#$InputTCPMaxSessions 2000
#$InputTCPMaxListeners 20
$ActionFileDefaultTemplate RSYSLOG_SyslogProtocol23Format
$template DynaFile,"/var/spool/rsyslog/%HOSTNAME%.log"
$template json-template2,
"{\"message\":\"%msg:::json%\",\"fromhost\":\"%HOSTNAME:::json%\",\"facility\":\"%syslogfacility-text%\",\"priority\":\"%syslogpriority-text%\",\"timereported\":\"%timereported:::date-rfc3339%\",\"timegenerated\":\"%timegenerated:::date-rfc3339%\"}"
$FileGroup uxadmin
$FileCreateMode 0640
$DirCreateMode 0750
$Umask 0022
$RuleSet Remote
*.debug ?DynaFile
#*.info @@(o)be-s2507-msl:10514;json-template2
#*.info @@(o)be-s2508-msl:10514;json-template2
& stop
$DefaultRuleset Remote
$InputTCPServerBindRuleset Remote
$InputTCPServerRun 514
$InputUDPServerBindRuleset Remote
$UDPServerRun 514
Best wishes,
Sophie
> -----Original Message-----
> From: rsyslog
> [mailto:[email protected]<mailto:[email protected]>]
> On Behalf Of
> sophie.loewenthal--- via rsyslog
> Sent: Tuesday, April 24, 2018 9:53 AM
> To: rsyslog-users
> Cc: LOEWENTHAL Sophie
> Subject: Re: [rsyslog] server received messages but rsyslog did not write them
> into a file
>
> Hi David,
>
> I disabled log rotation during my investigation, and this was not the cause.
>
> A cause won't be SELinux because we use Solaris 11.
> Impstats module is not available out of the box for Solaris. We don't have
> this or
> the ability to compile in production.
>
> Here is the configuration, in case this helps. The json-template0 template is
> defined but not used. json-template2 is used later on.
> Also the Local template is not used and the rsyslog server logs its own
> messages
> via DynaFile just like the clients.
> The legacy syntax is used because I cannot the new rsyslog syntax.
>
>
> $ModLoad imsolaris # for Solaris kernel logging
> $ModLoad imtcp
> $ModLoad imudp
> $MainMsgQueueSize 1000
> $InputTCPMaxSessions 2000
> $InputTCPMaxListeners 20
> $WorkDirectory /var/spool/rsyslog/work
> $ActionFileDefaultTemplate RSYSLOG_SyslogProtocol23Format
> $template DynaFile,"/var/spool/rsyslog/%HOSTNAME%.log"
> template(name="json-template0"
> type="list") {
> constant(value="{")
> constant(value="\"@timestamp\":\"") property(name="timereported"
> dateFormat="rfc3339")
> constant(value="\",\"@version\":\"1")
> constant(value="\",\"message\":\"") property(name="msg"
> format="json")
> constant(value="\",\"sysloghost\":\"") property(name="hostname")
> constant(value="\",\"severity\":\"")
> property(name="syslogseverity-text")
> constant(value="\",\"facility\":\"")
> property(name="syslogfacility-text")
> constant(value="\",\"programname\":\"") property(name="programname")
> constant(value="\",\"procid\":\"") property(name="procid")
> constant(value="\"}\n")
> }
> $template json-template2,
> "{\"message\":\"%msg:::json%\",\"fromhost\":\"%HOSTNAME:::json%\",\"facili
> ty\":\"%syslogfacility-text%\",\"priority\":\"%syslogpriority-
> text%\",\"timereported\":\"%timereported:::date-
> rfc3339%\",\"timegenerated\":\"%timegenerated:::date-rfc3339%\"}"
> $FileOwner root
> $FileGroup uxadmin
> $FileCreateMode 0640
> $DirCreateMode 0750
> $Umask 0022
> $RuleSet Local
> *.err;kern.debug;daemon.notice;mail.crit /var/adm/messages
> authpriv.* /var/adm/authpriv.log
> *.emerg :omusrmsg:*
> *.alert :omusrmsg:root
> & stop
> $RuleSet Remote
> *.info ?DynaFile
> *.info @@(o)b111l:10514;json-template2
> *.info @@(o)b112l:10514;json-template2
> & stop
> $DefaultRuleset Remote
> $InputTCPServerBindRuleset Remote
> $InputTCPServerRun 514
> $InputUDPServerBindRuleset Remote
> $UDPServerRun 514
>
>
> Best wishes,
> Sophie
>
> Team mailbox : [email protected]<mailto:[email protected]>
> or direct
> [email protected]<mailto:[email protected]>
>
>
>
>
> > -----Original Message-----
> > From: David Lang [mailto:[email protected]<mailto:[email protected]>]
> > Sent: Monday, April 23, 2018 10:27 PM
> > To: sophie.loewenthal--- via rsyslog
> > Cc: LOEWENTHAL Sophie
> > Subject: Re: [rsyslog] server received messages but rsyslog did not write
> > them
> > into a file
> >
> > On Mon, 23 Apr 2018, sophie.loewenthal--- via rsyslog wrote:
> >
> > > After 31st March our rsyslog v8.4.2 Solaris 11 servers stopped processing
> > > lots
> > of messages. I think we had a network change but do not yet know what.
> > >
> > > Tcpdump showed the test message arrived,
> > >
> > > # tcpdump -s 0 -A -vvv port 514 |grep sdfasdfsa
> > > dropped privs to nobody
> > > tcpdump: listening on ipmp0, link-type IPNET (Solaris ipnet), capture size
> > 262144 bytes
> > > .KB.....78 <13>Apr 23 15:01:54 be-s0784-z1a
> > > emerg.info<http://emerg.info>: [ID 702911
> > user.notice] sdfasdfsa
> > > 46 packets captured
> > >
> > > But the log file for this server remains empty. The rsyslog client sent
> > > the
> > message over TCP with,
> > >
> > > *.info @@(o)sysl1:514
> > > *.info @@(o)sysl2:514
> > >
> > > Could this be a routing issue? Or something else.
> >
> > It could be a lot of things, the most common is that the output logfile was
> > moved (or deleted), but rsyslog wasn't sent the HUP signal and so is still
> > writing to the old file.
> >
> > you can use lsof to look for all files that rsyslog has filehandles for and
> > see
> > what it says.
> >
> > The next most common cause is that some other output is blocked, and so
> > messages
> > are just queuing up instead of being written, diagnosing this requires
> > enabling
> > impstats in rsyslog, and since that requires a restart, the restart will
> > probably 'solve' the problem in the short term. If we could see the entire
> > config file (and anything included into it), we could make an educated
> > guess as
> > to if that is the problem or not.
> >
> > It could also be SELinux problems, but that doesn't usually start happening
> > after the system has been running sucessfully
> >
> > and there is a chance that it's something at the IP layer (routing or
> > iptables),
> > but again, those don't usually show up after a system is running
> > successfully.
> This message and any attachments (the "message") is
> intended solely for the intended addressees and is confidential.
> If you receive this message in error,or are not the intended recipient(s),
> please delete it and any copies from your systems and immediately notify
> the sender. Any unauthorized view, use that does not comply with its purpose,
> dissemination or disclosure, either whole or partial, is prohibited. Since the
> internet
> cannot guarantee the integrity of this message which may not be reliable, BNP
> PARIBAS
> (and its subsidiaries) shall not be liable for the message if modified,
> changed or
> falsified.
> Do not print this message unless it is necessary, consider the environment.
>
> --------------------------------------------------------------------------------------------------
> --------------------------------
>
> Ce message et toutes les pieces jointes (ci-apres le "message")
> sont etablis a l'intention exclusive de ses destinataires et sont
> confidentiels.
> Si vous recevez ce message par erreur ou s'il ne vous est pas destine,
> merci de le detruire ainsi que toute copie de votre systeme et d'en avertir
> immediatement l'expediteur. Toute lecture non autorisee, toute utilisation de
> ce message qui n'est pas conforme a sa destination, toute diffusion ou toute
> publication, totale ou partielle, est interdite. L'Internet ne permettant pas
> d'assurer
> l'integrite de ce message electronique susceptible d'alteration, BNP Paribas
> (et ses filiales) decline(nt) toute responsabilite au titre de ce message dans
> l'hypothese
> ou il aurait ete modifie, deforme ou falsifie.
> N'imprimez ce message que si necessaire, pensez a l'environnement.
>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T
> LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
