Re: Unable to use Syslog Parser
Farrukh, This error means that the syslog line you are passing in is not proper per the spec. Can you create a jira, with this info, and attach or otherwise include a SANITIZED (change IP, machine names, business stuff etc since this will be on the internet ) version of the failing line? I’ll be able to tell you what the issue is and what the options are once I can test it. Not everything sends properly formatted ( to the spec ) syslog. While simple-syslog ( the library I wrote that backs this parser ) makes allowances ( for missing priority, different date formats ) it cannot handle everything that is possible obviously. As a not, this same library is used in nifi for the 5424 processor/ record reader as well. On February 13, 2019 at 05:54:42, Farrukh Naveed Anjum ( anjum.farr...@gmail.com) wrote: Hi, I am trying to utilize for Syslog5424 I am recieving data from Nifi into the Kakfa. I am getting the Parser Exception any help will be appreciated. Following is the error. nerated.Rfc5424Parser.header(Rfc5424Parser.java:412) ~[stormjar.jar:?] at com.github.palindromicity.syslog.dsl.generated.Rfc5424Parser.syslog_msg(Rfc5424Parser.java:273) ~[stormjar.jar:?] at com.github.palindromicity.syslog.Rfc5424SyslogParser.parseLine(Rfc5424SyslogParser.java:93) ~[stormjar.jar:?] at com.github.palindromicity.syslog.Rfc5424SyslogParser.lambda$parseLines$0(Rfc5424SyslogParser.java:130) ~[stormjar.jar:?] at java.util.ArrayList.forEach(ArrayList.java:1249) [?:1.8.0_112] at com.github.palindromicity.syslog.Rfc5424SyslogParser.parseLines(Rfc5424SyslogParser.java:128) ~[stormjar.jar:?] at org.apache.metron.parsers.syslog.Syslog5424Parser.parseOptionalResult(Syslog5424Parser.java:103) ~[stormjar.jar:?] at org.apache.metron.parsers.ParserRunnerImpl.execute(ParserRunnerImpl.java:146) ~[stormjar.jar:?] at org.apache.metron.parsers.bolt.ParserBolt.execute(ParserBolt.java:253) [stormjar.jar:?] at org.apache.storm.daemon.executor$fn__10195$tuple_action_fn__10197.invoke(executor.clj:735) [storm-core-1.1.0.2.6.5.1050-37.jar:1.1.0.2.6.5.1050-37] at org.apache.storm.daemon.executor$mk_task_receiver$fn__10114.invoke(executor.clj:466) [storm-core-1.1.0.2.6.5.1050-37.jar:1.1.0.2.6.5.1050-37] at org.apache.storm.disruptor$clojure_handler$reify__4137.onEvent(disruptor.clj:40) [storm-core-1.1.0.2.6.5.1050-37.jar:1.1.0.2.6.5.1050-37] at org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:472) [storm-core-1.1.0.2.6.5.1050-37.jar:1.1.0.2.6.5.1050-37] at org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:451) [storm-core-1.1.0.2.6.5.1050-37.jar:1.1.0.2.6.5.1050-37] at org.apache.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:73) [storm-core-1.1.0.2.6.5.1050-37.jar:1.1.0.2.6.5.1050-37] at org.apache.storm.daemon.executor$fn__10195$fn__10208$fn__10263.invoke(executor.clj:855) [storm-core-1.1.0.2.6.5.1050-37.jar:1.1.0.2.6.5.1050-37] at org.apache.storm.util$async_loop$fn__1221.invoke(util.clj:484) [storm-core-1.1.0.2.6.5.1050-37.jar:1.1.0.2.6.5.1050-37] at clojure.lang.AFn.run(AFn.java:22) [clojure-1.7.0.jar:?] at java.lang.Thread.run(Thread.java:745) [?:1.8.0_112] Caused by: org.antlr.v4.runtime.NoViableAltException at org.antlr.v4.runtime.atn.ParserATNSimulator.noViableAlt(ParserATNSimulator.java:1894) ~[stormjar.jar:?] at org.antlr.v4.runtime.atn.ParserATNSimulator.execATN(ParserATNSimulator.java:498) ~[stormjar.jar:?] at org.antlr.v4.runtime.atn.ParserATNSimulator.adaptivePredict(ParserATNSimulator.java:424) ~[stormjar.jar:?] at com.github.palindromicity.syslog.dsl.generated.Rfc5424Parser.header(Rfc5424Parser.java:373) ~[stormjar.jar:?] ... 18 more 2019-02-13 15:52:03.138 o.a.s.d.executor Thread-12-parserBolt-executor[5 5] [ERROR] com.github.palindromicity.syslog.dsl.ParseException: Syntax error @ 1:5 no viable alternative at input 'F' at com.github.palindromicity.syslog.dsl.DefaultErrorListener.syntaxError(DefaultErrorListener.java:17) ~[stormjar.jar:?] at org.antlr.v4.runtime.ProxyErrorListener.syntaxError(ProxyErrorListener.java:65) ~[stormjar.jar:?] at org.antlr.v4.runtime.Parser.notifyErrorListeners(Parser.java:558) ~[stormjar.jar:?] at org.antlr.v4.runtime.DefaultErrorStrategy.reportNoViableAlternative(DefaultErrorStrategy.java:310) ~[stormjar.jar:?] at org.antlr.v4.runtime.DefaultErrorStrategy.reportError(DefaultErrorStrategy.java:147) ~[stormjar.jar:?] at com.github.palindromicity.syslog.dsl.generated.Rfc5424Parser.header(Rfc5424Parser.java:412) ~[stormjar.jar:?] at com.github.palindromicity.syslog.dsl.generated.Rfc5424Parser.syslog_msg(Rfc5424Parser.java:273) ~[stormjar.jar:?] at com.github.palindromicity.syslog.Rfc5424SyslogParser.parseLine(Rfc5424SyslogParser.java:93) ~[stormjar
Re: Unable to use Syslog Parser
Also include the configuration of the parser please. On February 13, 2019 at 09:00:08, Otto Fowler (ottobackwa...@gmail.com) wrote: Farrukh, This error means that the syslog line you are passing in is not proper per the spec. Can you create a jira, with this info, and attach or otherwise include a SANITIZED (change IP, machine names, business stuff etc since this will be on the internet ) version of the failing line? I’ll be able to tell you what the issue is and what the options are once I can test it. Not everything sends properly formatted ( to the spec ) syslog. While simple-syslog ( the library I wrote that backs this parser ) makes allowances ( for missing priority, different date formats ) it cannot handle everything that is possible obviously. As a not, this same library is used in nifi for the 5424 processor/ record reader as well. On February 13, 2019 at 05:54:42, Farrukh Naveed Anjum ( anjum.farr...@gmail.com) wrote: Hi, I am trying to utilize for Syslog5424 I am recieving data from Nifi into the Kakfa. I am getting the Parser Exception any help will be appreciated. Following is the error. nerated.Rfc5424Parser.header(Rfc5424Parser.java:412) ~[stormjar.jar:?] at com.github.palindromicity.syslog.dsl.generated.Rfc5424Parser.syslog_msg(Rfc5424Parser.java:273) ~[stormjar.jar:?] at com.github.palindromicity.syslog.Rfc5424SyslogParser.parseLine(Rfc5424SyslogParser.java:93) ~[stormjar.jar:?] at com.github.palindromicity.syslog.Rfc5424SyslogParser.lambda$parseLines$0(Rfc5424SyslogParser.java:130) ~[stormjar.jar:?] at java.util.ArrayList.forEach(ArrayList.java:1249) [?:1.8.0_112] at com.github.palindromicity.syslog.Rfc5424SyslogParser.parseLines(Rfc5424SyslogParser.java:128) ~[stormjar.jar:?] at org.apache.metron.parsers.syslog.Syslog5424Parser.parseOptionalResult(Syslog5424Parser.java:103) ~[stormjar.jar:?] at org.apache.metron.parsers.ParserRunnerImpl.execute(ParserRunnerImpl.java:146) ~[stormjar.jar:?] at org.apache.metron.parsers.bolt.ParserBolt.execute(ParserBolt.java:253) [stormjar.jar:?] at org.apache.storm.daemon.executor$fn__10195$tuple_action_fn__10197.invoke(executor.clj:735) [storm-core-1.1.0.2.6.5.1050-37.jar:1.1.0.2.6.5.1050-37] at org.apache.storm.daemon.executor$mk_task_receiver$fn__10114.invoke(executor.clj:466) [storm-core-1.1.0.2.6.5.1050-37.jar:1.1.0.2.6.5.1050-37] at org.apache.storm.disruptor$clojure_handler$reify__4137.onEvent(disruptor.clj:40) [storm-core-1.1.0.2.6.5.1050-37.jar:1.1.0.2.6.5.1050-37] at org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:472) [storm-core-1.1.0.2.6.5.1050-37.jar:1.1.0.2.6.5.1050-37] at org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:451) [storm-core-1.1.0.2.6.5.1050-37.jar:1.1.0.2.6.5.1050-37] at org.apache.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:73) [storm-core-1.1.0.2.6.5.1050-37.jar:1.1.0.2.6.5.1050-37] at org.apache.storm.daemon.executor$fn__10195$fn__10208$fn__10263.invoke(executor.clj:855) [storm-core-1.1.0.2.6.5.1050-37.jar:1.1.0.2.6.5.1050-37] at org.apache.storm.util$async_loop$fn__1221.invoke(util.clj:484) [storm-core-1.1.0.2.6.5.1050-37.jar:1.1.0.2.6.5.1050-37] at clojure.lang.AFn.run(AFn.java:22) [clojure-1.7.0.jar:?] at java.lang.Thread.run(Thread.java:745) [?:1.8.0_112] Caused by: org.antlr.v4.runtime.NoViableAltException at org.antlr.v4.runtime.atn.ParserATNSimulator.noViableAlt(ParserATNSimulator.java:1894) ~[stormjar.jar:?] at org.antlr.v4.runtime.atn.ParserATNSimulator.execATN(ParserATNSimulator.java:498) ~[stormjar.jar:?] at org.antlr.v4.runtime.atn.ParserATNSimulator.adaptivePredict(ParserATNSimulator.java:424) ~[stormjar.jar:?] at com.github.palindromicity.syslog.dsl.generated.Rfc5424Parser.header(Rfc5424Parser.java:373) ~[stormjar.jar:?] ... 18 more 2019-02-13 15:52:03.138 o.a.s.d.executor Thread-12-parserBolt-executor[5 5] [ERROR] com.github.palindromicity.syslog.dsl.ParseException: Syntax error @ 1:5 no viable alternative at input 'F' at com.github.palindromicity.syslog.dsl.DefaultErrorListener.syntaxError(DefaultErrorListener.java:17) ~[stormjar.jar:?] at org.antlr.v4.runtime.ProxyErrorListener.syntaxError(ProxyErrorListener.java:65) ~[stormjar.jar:?] at org.antlr.v4.runtime.Parser.notifyErrorListeners(Parser.java:558) ~[stormjar.jar:?] at org.antlr.v4.runtime.DefaultErrorStrategy.reportNoViableAlternative(DefaultErrorStrategy.java:310) ~[stormjar.jar:?] at org.antlr.v4.runtime.DefaultErrorStrategy.reportError(DefaultErrorStrategy.java:147) ~[stormjar.jar:?] at com.github.palindromicity.syslog.dsl.generated.Rfc5424Parser.header(Rfc5424Parser.java:412) ~[stormjar.jar:?] at com.github.palindromicity.syslog.dsl.generated.Rfc5424Parser.syslog_msg(Rfc5424Parser.java:2
Re: Unable to use Syslog Parser
Hi,
Thanks for reply, I did not made any configuration changes, But I can send
you sample Events
For example
SYSLOG | severity:ERR uid:CvS7064cni4HcD7FU6 id.orig_p:514 id.resp_p:514
proto:udp id.orig_h:10.2.2.1 message:Feb 14 13:16:52 suricata[88128]:
[1:2007994:20] ET MALWARE Suspicious User-Agent (1 space) [Classification: A
Network Trojan was Detected] [Priority: 1] {TCP} 10.2.2.229:37423 ->
168.235.205.6:80 facility:LOCAL5 ts:1550132212.404591 id.resp_h:172.16.4.18
Default Bro Syslog parser does not crunch it and just paste it as this
message
Feb 14 13:16:52 suricata[88128]: [1:2007994:20] ET MALWARE Suspicious
User-Agent (1 space) [Classification: A Network Trojan was Detected]
[Priority: 1] {TCP} 10.2.2.229:37423 -> 168.235.205.6:80 Now the problem is
IP_SRC and IP_DST are being populated as the local IP instead of these ips.
Similar classifications is not set. Please suggest also about windows
events logs for detecting Failed Logins
Feb 14 14:32:18 DC12.tap.local MSWinEventLog 5 Security 182049 Thu Feb 14
14:32:10 2019 4634 Microsoft-Windows-Security-Auditing N/A Audit Success
DC12.tap.local 12545 An account was logged off. Subject: Security ID:
S-1-5-21-761976910-1883327070-1659661340-1104 Account Name: EXG$ Account
Domain: TAP Logon ID: 0x3E3F0A7 Logon Type: 3 This event is generated when
a logon session is destroyed. It may be positively correlated with a logon
event using the Logon ID value. Logon IDs are only unique between reboots
on the same computer.
On Wed, Feb 13, 2019 at 7:01 PM Otto Fowler wrote:
> Also include the configuration of the parser please.
>
>
>
> On February 13, 2019 at 09:00:08, Otto Fowler (ottobackwa...@gmail.com)
> wrote:
>
> Farrukh,
>
> This error means that the syslog line you are passing in is not proper per
> the spec.
> Can you create a jira, with this info, and attach or otherwise include a
> SANITIZED (change IP, machine names, business stuff etc since this will be
> on the internet ) version of
> the failing line?
> I’ll be able to tell you what the issue is and what the options are once I
> can test it.
>
> Not everything sends properly formatted ( to the spec ) syslog. While
> simple-syslog ( the library I wrote that backs this parser ) makes
> allowances ( for missing priority, different date formats ) it
> cannot handle everything that is possible obviously.
>
> As a not, this same library is used in nifi for the 5424 processor/ record
> reader as well.
>
>
>
>
> On February 13, 2019 at 05:54:42, Farrukh Naveed Anjum (
> anjum.farr...@gmail.com) wrote:
>
> Hi,
> I am trying to utilize for Syslog5424 I am recieving data from Nifi into
> the Kakfa.
>
> I am getting the Parser Exception any help will be appreciated. Following
> is the error.
>
> nerated.Rfc5424Parser.header(Rfc5424Parser.java:412) ~[stormjar.jar:?]
> at
> com.github.palindromicity.syslog.dsl.generated.Rfc5424Parser.syslog_msg(Rfc5424Parser.java:273)
> ~[stormjar.jar:?]
> at
> com.github.palindromicity.syslog.Rfc5424SyslogParser.parseLine(Rfc5424SyslogParser.java:93)
> ~[stormjar.jar:?]
> at
> com.github.palindromicity.syslog.Rfc5424SyslogParser.lambda$parseLines$0(Rfc5424SyslogParser.java:130)
> ~[stormjar.jar:?]
> at java.util.ArrayList.forEach(ArrayList.java:1249) [?:1.8.0_112]
> at
> com.github.palindromicity.syslog.Rfc5424SyslogParser.parseLines(Rfc5424SyslogParser.java:128)
> ~[stormjar.jar:?]
> at
> org.apache.metron.parsers.syslog.Syslog5424Parser.parseOptionalResult(Syslog5424Parser.java:103)
> ~[stormjar.jar:?]
> at
> org.apache.metron.parsers.ParserRunnerImpl.execute(ParserRunnerImpl.java:146)
> ~[stormjar.jar:?]
> at
> org.apache.metron.parsers.bolt.ParserBolt.execute(ParserBolt.java:253)
> [stormjar.jar:?]
> at
> org.apache.storm.daemon.executor$fn__10195$tuple_action_fn__10197.invoke(executor.clj:735)
> [storm-core-1.1.0.2.6.5.1050-37.jar:1.1.0.2.6.5.1050-37]
> at
> org.apache.storm.daemon.executor$mk_task_receiver$fn__10114.invoke(executor.clj:466)
> [storm-core-1.1.0.2.6.5.1050-37.jar:1.1.0.2.6.5.1050-37]
> at
> org.apache.storm.disruptor$clojure_handler$reify__4137.onEvent(disruptor.clj:40)
> [storm-core-1.1.0.2.6.5.1050-37.jar:1.1.0.2.6.5.1050-37]
> at
> org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:472)
> [storm-core-1.1.0.2.6.5.1050-37.jar:1.1.0.2.6.5.1050-37]
> at
> org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:451)
> [storm-core-1.1.0.2.6.5.1050-37.jar:1.1.0.2.6.5.1050-37]
> at
> org.apache.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:73)
> [storm-core-1.1.0.2.6.5.1050-37.jar:1.1.0.2.6.5.1050-37]
> at
> org.apache.storm.daemon.executor$fn__10195$fn__10208$fn__10263.invoke(executor.clj:855)
> [storm-core-1.1.0.2.6.5.1050-37.jar:1.1.0.2.6.5.1050-37]
> at org.apache.storm.util$async_loop$fn__1221.invoke(util.clj:484)
> [sto
Re: Unable to use Syslog Parser
Please create a jira and attach the a file with the logs that fail.
On February 14, 2019 at 04:30:52, Farrukh Naveed Anjum (
anjum.farr...@gmail.com) wrote:
Hi,
Thanks for reply, I did not made any configuration changes, But I can send
you sample Events
For example
SYSLOG | severity:ERR uid:CvS7064cni4HcD7FU6 id.orig_p:514 id.resp_p:514
proto:udp id.orig_h:10.2.2.1 message:Feb 14 13:16:52 suricata[88128]:
[1:2007994:20] ET MALWARE Suspicious User-Agent (1 space) [Classification:
A Network Trojan was Detected] [Priority: 1] {TCP} 10.2.2.229:37423 ->
168.235.205.6:80 facility:LOCAL5 ts:1550132212.404591 id.resp_h:172.16.4.18
Default Bro Syslog parser does not crunch it and just paste it as this
message
Feb 14 13:16:52 suricata[88128]: [1:2007994:20] ET MALWARE Suspicious
User-Agent (1 space) [Classification: A Network Trojan was Detected]
[Priority: 1] {TCP} 10.2.2.229:37423 -> 168.235.205.6:80 Now the problem is
IP_SRC and IP_DST are being populated as the local IP instead of these ips.
Similar classifications is not set. Please suggest also about windows
events logs for detecting Failed Logins
Feb 14 14:32:18 DC12.tap.local MSWinEventLog 5 Security 182049 Thu Feb 14
14:32:10 2019 4634 Microsoft-Windows-Security-Auditing N/A Audit Success
DC12.tap.local 12545 An account was logged off. Subject: Security ID:
S-1-5-21-761976910-1883327070-1659661340-1104 Account Name: EXG$ Account
Domain: TAP Logon ID: 0x3E3F0A7 Logon Type: 3 This event is generated when
a logon session is destroyed. It may be positively correlated with a logon
event using the Logon ID value. Logon IDs are only unique between reboots
on the same computer.
On Wed, Feb 13, 2019 at 7:01 PM Otto Fowler wrote:
> Also include the configuration of the parser please.
>
>
>
> On February 13, 2019 at 09:00:08, Otto Fowler (ottobackwa...@gmail.com)
> wrote:
>
> Farrukh,
>
> This error means that the syslog line you are passing in is not proper per
> the spec.
> Can you create a jira, with this info, and attach or otherwise include a
> SANITIZED (change IP, machine names, business stuff etc since this will be
> on the internet ) version of
> the failing line?
> I’ll be able to tell you what the issue is and what the options are once I
> can test it.
>
> Not everything sends properly formatted ( to the spec ) syslog. While
> simple-syslog ( the library I wrote that backs this parser ) makes
> allowances ( for missing priority, different date formats ) it
> cannot handle everything that is possible obviously.
>
> As a not, this same library is used in nifi for the 5424 processor/ record
> reader as well.
>
>
>
>
> On February 13, 2019 at 05:54:42, Farrukh Naveed Anjum (
> anjum.farr...@gmail.com) wrote:
>
> Hi,
> I am trying to utilize for Syslog5424 I am recieving data from Nifi into
> the Kakfa.
>
> I am getting the Parser Exception any help will be appreciated. Following
> is the error.
>
> nerated.Rfc5424Parser.header(Rfc5424Parser.java:412) ~[stormjar.jar:?]
> at
> com.github.palindromicity.syslog.dsl.generated.Rfc5424Parser.syslog_msg(Rfc5424Parser.java:273)
> ~[stormjar.jar:?]
> at
> com.github.palindromicity.syslog.Rfc5424SyslogParser.parseLine(Rfc5424SyslogParser.java:93)
> ~[stormjar.jar:?]
> at
> com.github.palindromicity.syslog.Rfc5424SyslogParser.lambda$parseLines$0(Rfc5424SyslogParser.java:130)
> ~[stormjar.jar:?]
> at java.util.ArrayList.forEach(ArrayList.java:1249) [?:1.8.0_112]
> at
> com.github.palindromicity.syslog.Rfc5424SyslogParser.parseLines(Rfc5424SyslogParser.java:128)
> ~[stormjar.jar:?]
> at
> org.apache.metron.parsers.syslog.Syslog5424Parser.parseOptionalResult(Syslog5424Parser.java:103)
> ~[stormjar.jar:?]
> at
> org.apache.metron.parsers.ParserRunnerImpl.execute(ParserRunnerImpl.java:146)
> ~[stormjar.jar:?]
> at
> org.apache.metron.parsers.bolt.ParserBolt.execute(ParserBolt.java:253)
> [stormjar.jar:?]
> at
> org.apache.storm.daemon.executor$fn__10195$tuple_action_fn__10197.invoke(executor.clj:735)
> [storm-core-1.1.0.2.6.5.1050-37.jar:1.1.0.2.6.5.1050-37]
> at
> org.apache.storm.daemon.executor$mk_task_receiver$fn__10114.invoke(executor.clj:466)
> [storm-core-1.1.0.2.6.5.1050-37.jar:1.1.0.2.6.5.1050-37]
> at
> org.apache.storm.disruptor$clojure_handler$reify__4137.onEvent(disruptor.clj:40)
> [storm-core-1.1.0.2.6.5.1050-37.jar:1.1.0.2.6.5.1050-37]
> at
> org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:472)
> [storm-core-1.1.0.2.6.5.1050-37.jar:1.1.0.2.6.5.1050-37]
> at
> org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:451)
> [storm-core-1.1.0.2.6.5.1050-37.jar:1.1.0.2.6.5.1050-37]
> at
> org.apache.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:73)
> [storm-core-1.1.0.2.6.5.1050-37.jar:1.1.0.2.6.5.1050-37]
> at
> org.apache.storm.daemon.executor$fn__10195$fn__10208$fn__10263.invoke(
Re: Unable to use Syslog Parser
I don’t understand what “Default Bro Syslog parser does not crunch it……”
means.
Can you explain your data flow?
On February 14, 2019 at 04:30:52, Farrukh Naveed Anjum (
anjum.farr...@gmail.com) wrote:
Hi,
Thanks for reply, I did not made any configuration changes, But I can send
you sample Events
For example
SYSLOG | severity:ERR uid:CvS7064cni4HcD7FU6 id.orig_p:514 id.resp_p:514
proto:udp id.orig_h:10.2.2.1 message:Feb 14 13:16:52 suricata[88128]:
[1:2007994:20] ET MALWARE Suspicious User-Agent (1 space) [Classification:
A Network Trojan was Detected] [Priority: 1] {TCP} 10.2.2.229:37423 ->
168.235.205.6:80 facility:LOCAL5 ts:1550132212.404591 id.resp_h:172.16.4.18
Default Bro Syslog parser does not crunch it and just paste it as this
message
Feb 14 13:16:52 suricata[88128]: [1:2007994:20] ET MALWARE Suspicious
User-Agent (1 space) [Classification: A Network Trojan was Detected]
[Priority: 1] {TCP} 10.2.2.229:37423 -> 168.235.205.6:80 Now the problem is
IP_SRC and IP_DST are being populated as the local IP instead of these ips.
Similar classifications is not set. Please suggest also about windows
events logs for detecting Failed Logins
Feb 14 14:32:18 DC12.tap.local MSWinEventLog 5 Security 182049 Thu Feb 14
14:32:10 2019 4634 Microsoft-Windows-Security-Auditing N/A Audit Success
DC12.tap.local 12545 An account was logged off. Subject: Security ID:
S-1-5-21-761976910-1883327070-1659661340-1104 Account Name: EXG$ Account
Domain: TAP Logon ID: 0x3E3F0A7 Logon Type: 3 This event is generated when
a logon session is destroyed. It may be positively correlated with a logon
event using the Logon ID value. Logon IDs are only unique between reboots
on the same computer.
On Wed, Feb 13, 2019 at 7:01 PM Otto Fowler wrote:
> Also include the configuration of the parser please.
>
>
>
> On February 13, 2019 at 09:00:08, Otto Fowler (ottobackwa...@gmail.com)
> wrote:
>
> Farrukh,
>
> This error means that the syslog line you are passing in is not proper per
> the spec.
> Can you create a jira, with this info, and attach or otherwise include a
> SANITIZED (change IP, machine names, business stuff etc since this will be
> on the internet ) version of
> the failing line?
> I’ll be able to tell you what the issue is and what the options are once I
> can test it.
>
> Not everything sends properly formatted ( to the spec ) syslog. While
> simple-syslog ( the library I wrote that backs this parser ) makes
> allowances ( for missing priority, different date formats ) it
> cannot handle everything that is possible obviously.
>
> As a not, this same library is used in nifi for the 5424 processor/ record
> reader as well.
>
>
>
>
> On February 13, 2019 at 05:54:42, Farrukh Naveed Anjum (
> anjum.farr...@gmail.com) wrote:
>
> Hi,
> I am trying to utilize for Syslog5424 I am recieving data from Nifi into
> the Kakfa.
>
> I am getting the Parser Exception any help will be appreciated. Following
> is the error.
>
> nerated.Rfc5424Parser.header(Rfc5424Parser.java:412) ~[stormjar.jar:?]
> at
> com.github.palindromicity.syslog.dsl.generated.Rfc5424Parser.syslog_msg(Rfc5424Parser.java:273)
> ~[stormjar.jar:?]
> at
> com.github.palindromicity.syslog.Rfc5424SyslogParser.parseLine(Rfc5424SyslogParser.java:93)
> ~[stormjar.jar:?]
> at
> com.github.palindromicity.syslog.Rfc5424SyslogParser.lambda$parseLines$0(Rfc5424SyslogParser.java:130)
> ~[stormjar.jar:?]
> at java.util.ArrayList.forEach(ArrayList.java:1249) [?:1.8.0_112]
> at
> com.github.palindromicity.syslog.Rfc5424SyslogParser.parseLines(Rfc5424SyslogParser.java:128)
> ~[stormjar.jar:?]
> at
> org.apache.metron.parsers.syslog.Syslog5424Parser.parseOptionalResult(Syslog5424Parser.java:103)
> ~[stormjar.jar:?]
> at
> org.apache.metron.parsers.ParserRunnerImpl.execute(ParserRunnerImpl.java:146)
> ~[stormjar.jar:?]
> at
> org.apache.metron.parsers.bolt.ParserBolt.execute(ParserBolt.java:253)
> [stormjar.jar:?]
> at
> org.apache.storm.daemon.executor$fn__10195$tuple_action_fn__10197.invoke(executor.clj:735)
> [storm-core-1.1.0.2.6.5.1050-37.jar:1.1.0.2.6.5.1050-37]
> at
> org.apache.storm.daemon.executor$mk_task_receiver$fn__10114.invoke(executor.clj:466)
> [storm-core-1.1.0.2.6.5.1050-37.jar:1.1.0.2.6.5.1050-37]
> at
> org.apache.storm.disruptor$clojure_handler$reify__4137.onEvent(disruptor.clj:40)
> [storm-core-1.1.0.2.6.5.1050-37.jar:1.1.0.2.6.5.1050-37]
> at
> org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:472)
> [storm-core-1.1.0.2.6.5.1050-37.jar:1.1.0.2.6.5.1050-37]
> at
> org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:451)
> [storm-core-1.1.0.2.6.5.1050-37.jar:1.1.0.2.6.5.1050-37]
> at
> org.apache.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:73)
> [storm-core-1.1.0.2.6.5.1050-37.jar:1.1.0.2.6.5.1050-37]
> at
> org.apache.storm.daemon.e
Re: Unable to use Syslog Parser
Yes, I can explain, All, I am looking for is to parse the Message (Tokenize
it), that i am recieveing from the Syslog (Windows Event Logger).
Please have a look at following two ElasticSearch Objects. They don't get
stored in a meaningful way. Is there a way I can extract out Logged Out,
Failed Passwords from it ? Which parser will be best suited for it ?
{
"_index": "bro_index_2019.02.15.10",
"_type": "bro_doc",
"_id": "f411cc08-bbdf-4875-ac46-fcc69f3deace",
"_version": 1,
"_score": null,
"_source": {
"bro_timestamp": "1550208625.997473",
"ip_dst_port": 514,
"adapter:geoadapter:begin:ts": "1550208626893",
"parallelenricher:enrich:end:ts": "1550208626896",
"uid": "Cw7P6g38y3tWWpC9R4",
"protocol": "syslog",
"source:type": "bro",
"adapter:threatinteladapter:end:ts": "1550208626896",
"original_string": "SYSLOG | severity:INFO uid:Cw7P6g38y3tWWpC9R4
id.orig_p:60607 id.resp_p:514 proto:udp id.orig_h:10.60.60.81 message:Feb
15 10:33:38 DC12.tap.local MSWinEventLog\t6\tApplication\t238922\tFri Feb
15 10:33:31
2019\t902\tMicrosoft-Windows-Security-SPP\t\tN/A\tInformation\tDC12.tap.local\t0\tThe
Software Protection service has started.\r\n6.3.9600.19101\n facility:KERN
ts:1550208625.997473 id.resp_h:172.16.4.18",
"ip_dst_addr": "172.16.4.18",
"adapter:hostfromjsonlistadapter:end:ts": "1550208626893",
"adapter:geoadapter:end:ts": "1550208626893",
"ip_src_addr": "10.60.60.81",
"timestamp": 1550208625997,
"severity": "INFO",
"parallelenricher:enrich:begin:ts": "1550208626895",
"adapter:hostfromjsonlistadapter:begin:ts": "1550208626893",
"message": "Feb 15 10:33:38 DC12.tap.local
MSWinEventLog\t6\tApplication\t238922\tFri Feb 15 10:33:31
2019\t902\tMicrosoft-Windows-Security-SPP\t\tN/A\tInformation\tDC12.tap.local\t0\tThe
Software Protection service has started.\r\n6.3.9600.19101\n",
"parallelenricher:splitter:begin:ts": "1550208626895",
"ip_src_port": 60607,
"proto": "udp",
"parallelenricher:splitter:end:ts": "1550208626895",
"adapter:threatinteladapter:begin:ts": "1550208626895",
"guid": "f411cc08-bbdf-4875-ac46-fcc69f3deace",
"facility": "KERN"
},
"fields": {
"parallelenricher:enrich:begin:ts": [
1550208626895
],
"adapter:geoadapter:begin:ts": [
1550208626893
],
"adapter:hostfromjsonlistadapter:begin:ts": [
1550208626893
],
"parallelenricher:enrich:end:ts": [
1550208626896
],
"parallelenricher:splitter:begin:ts": [
1550208626895
],
"adapter:threatinteladapter:end:ts": [
1550208626896
],
"adapter:hostfromjsonlistadapter:end:ts": [
1550208626893
],
"parallelenricher:splitter:end:ts": [
1550208626895
],
"adapter:threatinteladapter:begin:ts": [
1550208626895
],
"adapter:geoadapter:end:ts": [
1550208626893
],
"timestamp": [
1550208625997
]
},
"highlight": {
"original_string": [
"SYSLOG | severity:INFO uid:Cw7P6g38y3tWWpC9R4 id.orig_p:60607
id.resp_p:514 proto:udp id.orig_h:@kibana-highlighted-field@10.60.60.81@
/kibana-highlighted-field@ message:Feb 15 10:33:38 DC12.tap.local
MSWinEventLog\t6\tApplication\t238922\tFri Feb 15 10:33:31
2019\t902\tMicrosoft-Windows-Security-SPP\t\tN/A\tInformation\tDC12.tap.local\t0\tThe
Software Protection service has started.\r\n6.3.9600.19101\n facility:KERN
ts:1550208625.997473 id.resp_h:172.16.4.18"
]
},
"sort": [
1550208625997
]
}
Another Sample Object
{
"_index": "bro_index_2019.02.15.10",
"_type": "bro_doc",
"_id": "7107a0b8-4999-4956-b20f-40fd666bed46",
"_version": 1,
"_score": null,
"_source": {
"bro_timestamp": "1550209568.304029",
"ip_dst_port": 514,
"adapter:geoadapter:begin:ts": "1550209569921",
"parallelenricher:enrich:end:ts": "1550209569923",
"uid": "Cw7P6g38y3tWWpC9R4",
"protocol": "syslog",
"source:type": "bro",
"adapter:threatinteladapter:end:ts": "1550209569923",
"original_string": "SYSLOG | severity:NOTICE uid:Cw7P6g38y3tWWpC9R4
id.orig_p:60607 id.resp_p:514 proto:udp id.orig_h:10.60.60.81 message:Feb
15 10:49:20 DC12.tap.local MSWinEventLog\t5\tSecurity\t239656\tFri Feb 15
10:49:11 2019\t4634\tMicrosoft-Windows-Security-Auditing\t\tN/A\tAudit
Success\tDC12.tap.local\t12545\tAn account was logged
off.\r\n\r\nSubject:\r\n\tSecurity
ID:\t\tS-1-5-21-761976910-1883327070-1659661340-1104\r\n\tAccount
Name:\t\tEXG$\r\n\tAccount Domain:\t\tTAP\r\n\tLogon
ID:\t\t0x505F5B4\r\n\r\nLogon Type:\t\t\t3\r\n\r\nThis event is generated
when a logon session is destroyed. It may be positively correlated with a
logon event using the Logon ID value. Logon IDs are only unique between
reboots on the same computer.\n facility:KERN ts:1550209568.304029
id.resp_h:172.16.4.18",
"ip_dst_addr": "172.16.4.18",
"adapter:hostfromjsonlistadapter:end:ts": "1550209569921",
"adapter:geoadapter:end:ts": "1550209569921",
"ip_src_addr": "10.60.60.8
Re: Unable to use Syslog Parser
How are your messages getting into metron/kafka?
windows-syslog -> bro -> bro-kafka -> kafka -> bro parser ?
On February 15, 2019 at 00:49:24, Farrukh Naveed Anjum (
anjum.farr...@gmail.com) wrote:
Yes, I can explain, All, I am looking for is to parse the Message (Tokenize
it), that i am recieveing from the Syslog (Windows Event Logger).
Please have a look at following two ElasticSearch Objects. They don't get
stored in a meaningful way. Is there a way I can extract out Logged Out,
Failed Passwords from it ? Which parser will be best suited for it ?
{
"_index": "bro_index_2019.02.15.10",
"_type": "bro_doc",
"_id": "f411cc08-bbdf-4875-ac46-fcc69f3deace",
"_version": 1,
"_score": null,
"_source": {
"bro_timestamp": "1550208625.997473",
"ip_dst_port": 514,
"adapter:geoadapter:begin:ts": "1550208626893",
"parallelenricher:enrich:end:ts": "1550208626896",
"uid": "Cw7P6g38y3tWWpC9R4",
"protocol": "syslog",
"source:type": "bro",
"adapter:threatinteladapter:end:ts": "1550208626896",
"original_string": "SYSLOG | severity:INFO uid:Cw7P6g38y3tWWpC9R4
id.orig_p:60607 id.resp_p:514 proto:udp id.orig_h:10.60.60.81 message:Feb
15 10:33:38 DC12.tap.local MSWinEventLog\t6\tApplication\t238922\tFri Feb
15 10:33:31
2019\t902\tMicrosoft-Windows-Security-SPP\t\tN/A\tInformation\tDC12.tap.local\t0\tThe
Software Protection service has started.\r\n6.3.9600.19101\n facility:KERN
ts:1550208625.997473 id.resp_h:172.16.4.18",
"ip_dst_addr": "172.16.4.18",
"adapter:hostfromjsonlistadapter:end:ts": "1550208626893",
"adapter:geoadapter:end:ts": "1550208626893",
"ip_src_addr": "10.60.60.81",
"timestamp": 1550208625997,
"severity": "INFO",
"parallelenricher:enrich:begin:ts": "1550208626895",
"adapter:hostfromjsonlistadapter:begin:ts": "1550208626893",
"message": "Feb 15 10:33:38 DC12.tap.local
MSWinEventLog\t6\tApplication\t238922\tFri Feb 15 10:33:31
2019\t902\tMicrosoft-Windows-Security-SPP\t\tN/A\tInformation\tDC12.tap.local\t0\tThe
Software Protection service has started.\r\n6.3.9600.19101\n",
"parallelenricher:splitter:begin:ts": "1550208626895",
"ip_src_port": 60607,
"proto": "udp",
"parallelenricher:splitter:end:ts": "1550208626895",
"adapter:threatinteladapter:begin:ts": "1550208626895",
"guid": "f411cc08-bbdf-4875-ac46-fcc69f3deace",
"facility": "KERN"
},
"fields": {
"parallelenricher:enrich:begin:ts": [
1550208626895
],
"adapter:geoadapter:begin:ts": [
1550208626893
],
"adapter:hostfromjsonlistadapter:begin:ts": [
1550208626893
],
"parallelenricher:enrich:end:ts": [
1550208626896
],
"parallelenricher:splitter:begin:ts": [
1550208626895
],
"adapter:threatinteladapter:end:ts": [
1550208626896
],
"adapter:hostfromjsonlistadapter:end:ts": [
1550208626893
],
"parallelenricher:splitter:end:ts": [
1550208626895
],
"adapter:threatinteladapter:begin:ts": [
1550208626895
],
"adapter:geoadapter:end:ts": [
1550208626893
],
"timestamp": [
1550208625997
]
},
"highlight": {
"original_string": [
"SYSLOG | severity:INFO uid:Cw7P6g38y3tWWpC9R4 id.orig_p:60607
id.resp_p:514 proto:udp id.orig_h:@kibana-highlighted-field@10.60.60.81@
/kibana-highlighted-field@ message:Feb 15 10:33:38 DC12.tap.local
MSWinEventLog\t6\tApplication\t238922\tFri Feb 15 10:33:31
2019\t902\tMicrosoft-Windows-Security-SPP\t\tN/A\tInformation\tDC12.tap.local\t0\tThe
Software Protection service has started.\r\n6.3.9600.19101\n facility:KERN
ts:1550208625.997473 id.resp_h:172.16.4.18"
]
},
"sort": [
1550208625997
]
}
Another Sample Object
{
"_index": "bro_index_2019.02.15.10",
"_type": "bro_doc",
"_id": "7107a0b8-4999-4956-b20f-40fd666bed46",
"_version": 1,
"_score": null,
"_source": {
"bro_timestamp": "1550209568.304029",
"ip_dst_port": 514,
"adapter:geoadapter:begin:ts": "1550209569921",
"parallelenricher:enrich:end:ts": "1550209569923",
"uid": "Cw7P6g38y3tWWpC9R4",
"protocol": "syslog",
"source:type": "bro",
"adapter:threatinteladapter:end:ts": "1550209569923",
"original_string": "SYSLOG | severity:NOTICE uid:Cw7P6g38y3tWWpC9R4
id.orig_p:60607 id.resp_p:514 proto:udp id.orig_h:10.60.60.81 message:Feb
15 10:49:20 DC12.tap.local MSWinEventLog\t5\tSecurity\t239656\tFri Feb 15
10:49:11 2019\t4634\tMicrosoft-Windows-Security-Auditing\t\tN/A\tAudit
Success\tDC12.tap.local\t12545\tAn account was logged
off.\r\n\r\nSubject:\r\n\tSecurity
ID:\t\tS-1-5-21-761976910-1883327070-1659661340-1104\r\n\tAccount
Name:\t\tEXG$\r\n\tAccount Domain:\t\tTAP\r\n\tLogon
ID:\t\t0x505F5B4\r\n\r\nLogon Type:\t\t\t3\r\n\r\nThis event is generated
when a logon session is destroyed. It may be positively correlated with a
logon event using the Logon ID value. Logon IDs are only unique between
reboots on the same computer.\n facility:KERN ts:1550209568.30402
