[graylog2] Re: how to resolve issue with indexer
Thank you! Resolved with renaming level to log_level in logstash
configuration
rename => [ "level" , "log_level" ]
Regards,
Jiri
On Thursday, February 16, 2017 at 10:02:37 AM UTC+1, Jochen Schalanda wrote:
>
> Hi Jiří,
>
> the "level" message field has to be a numeric value, i. e. the numeric
> severity level of syslog messages:
> https://en.wikipedia.org/wiki/Syslog#Severity_level
>
> You can use message processor pipeline rules to change that in Graylog:
> http://docs.graylog.org/en/2.2/pages/pipelines.html
>
> Although in your specific case, I'd recommend converting the "level" field
> in Logstash.
>
> Cheers,
> Jochen
>
>
> On Thursday, 16 February 2017 09:12:00 UTC+1, Jiří Kolb wrote:
>>
>> Hello,
>> I have following architecture Logstash -> RabbitMQ ->Graylog
>>
>> I have issue in indexer with parsing:
>> 17 minutes ago graylog_0 d58fe350-f41b-11e6-8407-000c29438b97
>> MapperParsingException[failed to parse [level]]; nested:
>> NumberFormatException[For input string: "information"]
>>
>> In logstash debug I can see message like this:
>>
>> {
>> "devid" => "FGVMEV00",
>>"raw_message" => "date=2017-02-16 time=00:55:46
>> devname=FortiGate-VM64 devid=FGVMEV00 logid=0100040704 type=event
>> subtype=system level=notice vd=root logdesc=\"System performance
>> statistics\" action=\"perf-stats\" cpu=0 mem=47 totalsession=3 disk=0
>> bandwidth=0/0 setuprate=0 disklograte=0 fazlograte=0 msg=\"Performance
>> statistics: average CPU: 0, memory: 47, concurrent sessions: 3,
>> setup-rate: 0\"",
>> "fazlograte" => "0",
>> "type" => "fortigate",
>>"mem" => "47",
>> "syslog5424_pri" => "189",
>> "@version" => "1",
>> "host" => "10.0.0.90",
>> "action" => "perf-stats",
>>"disklograte" => "0",
>> "device_id" => "FortiGate-VM64",
>> "level" => "notice",
>> "bandwidth" => "0/0",
>>"cpu" => "0",
>> "<189>date" => "2017-02-16",
>>"message" => "<189>date=2017-02-16 time=00:55:46
>> devname=FortiGate-VM64 devid=FGVMEV00 logid=0100040704 type=event
>> subtype=system level=notice vd=root logdesc=\"System performance
>> statistics\" action=\"perf-stats\" cpu=0 mem=47 totalsession=3 disk=0
>> bandwidth=0/0 setuprate=0 disklograte=0 fazlograte=0 msg=\"Performance
>> statistics: average CPU: 0, memory: 47, concurrent sessions: 3,
>> setup-rate: 0\"",
>> "vd" => "root",
>>"logdesc" => "System performance statistics",
>> "disk" => "0",
>> "@timestamp" => 2017-02-16T07:55:39.383Z,
>> "setuprate" => "0",
>> "logid" => "0100040704",
>> "time" => "00:55:46",
>> "totalsession" => "3"
>> }
>>
>>
>>
>> Can you please help? I do not understand why level is not parsed it is
>> string like others that are parsed correctly.
>>
>> Thank you!
>>
>> Jiri
>>
>
--
You received this message because you are subscribed to the Google Groups
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/graylog2/6a3ea6d8-e485-453b-aa55-c5e1bc188112%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
[graylog2] how to resolve issue with indexer
Hello,
I have following architecture Logstash -> RabbitMQ ->Graylog
I have issue in indexer with parsing:
17 minutes ago graylog_0 d58fe350-f41b-11e6-8407-000c29438b97
MapperParsingException[failed to parse [level]]; nested:
NumberFormatException[For input string: "information"]
In logstash debug I can see message like this:
{
"devid" => "FGVMEV00",
"raw_message" => "date=2017-02-16 time=00:55:46
devname=FortiGate-VM64 devid=FGVMEV00 logid=0100040704 type=event
subtype=system level=notice vd=root logdesc=\"System performance
statistics\" action=\"perf-stats\" cpu=0 mem=47 totalsession=3 disk=0
bandwidth=0/0 setuprate=0 disklograte=0 fazlograte=0 msg=\"Performance
statistics: average CPU: 0, memory: 47, concurrent sessions: 3,
setup-rate: 0\"",
"fazlograte" => "0",
"type" => "fortigate",
"mem" => "47",
"syslog5424_pri" => "189",
"@version" => "1",
"host" => "10.0.0.90",
"action" => "perf-stats",
"disklograte" => "0",
"device_id" => "FortiGate-VM64",
"level" => "notice",
"bandwidth" => "0/0",
"cpu" => "0",
"<189>date" => "2017-02-16",
"message" => "<189>date=2017-02-16 time=00:55:46
devname=FortiGate-VM64 devid=FGVMEV00 logid=0100040704 type=event
subtype=system level=notice vd=root logdesc=\"System performance
statistics\" action=\"perf-stats\" cpu=0 mem=47 totalsession=3 disk=0
bandwidth=0/0 setuprate=0 disklograte=0 fazlograte=0 msg=\"Performance
statistics: average CPU: 0, memory: 47, concurrent sessions: 3,
setup-rate: 0\"",
"vd" => "root",
"logdesc" => "System performance statistics",
"disk" => "0",
"@timestamp" => 2017-02-16T07:55:39.383Z,
"setuprate" => "0",
"logid" => "0100040704",
"time" => "00:55:46",
"totalsession" => "3"
}
Can you please help? I do not understand why level is not parsed it is
string like others that are parsed correctly.
Thank you!
Jiri
--
You received this message because you are subscribed to the Google Groups
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/graylog2/65dfdded-e8d0-46f9-a6d3-a89f35aec58c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
[graylog2] cannot start AMQP input
Hi,
Trying to add AMQP input to connect with RabbitMQ, but input does not
start. Can you please help? Following is graylog server log:
2017-02-13_12:36:08.35670 2017-02-13 13:36:08,342 ERROR:
org.graylog2.shared.inputs.InputLauncher - The
[org.graylog2.inputs.gelf.amqp.GELFAMQPInput] input with ID
<58a1a833ea84240352ab0c9e> misfired. Reason: Connection refused.
2017-02-13_12:36:08.35733 org.graylog2.plugin.inputs.MisfireException:
org.graylog2.plugin.inputs.MisfireException: Could not launch AMQP consumer.
2017-02-13_12:36:08.36173 at
org.graylog2.plugin.inputs.MessageInput.launch(MessageInput.java:156)
~[graylog.jar:?]
2017-02-13_12:36:08.36503 at
org.graylog2.shared.inputs.InputLauncher$1.run(InputLauncher.java:84)
[graylog.jar:?]
2017-02-13_12:36:08.36512 at
com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176)
[graylog.jar:?]
2017-02-13_12:36:08.36614 at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[?:1.8.0_101]
2017-02-13_12:36:08.39847 at
java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_101]
2017-02-13_12:36:08.40479 at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[?:1.8.0_101]
2017-02-13_12:36:08.40688 at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[?:1.8.0_101]
2017-02-13_12:36:08.40894 at java.lang.Thread.run(Thread.java:745)
[?:1.8.0_101]
2017-02-13_12:36:08.41575 Caused by:
org.graylog2.plugin.inputs.MisfireException: Could not launch AMQP consumer.
2017-02-13_12:36:08.43687 at
org.graylog2.inputs.transports.AmqpTransport.doLaunch(AmqpTransport.java:179)
~[graylog.jar:?]
2017-02-13_12:36:08.43858 at
org.graylog2.plugin.inputs.transports.ThrottleableTransport.launch(ThrottleableTransport.java:75)
~[graylog.jar:?]
2017-02-13_12:36:08.44099 at
org.graylog2.plugin.inputs.MessageInput.launch(MessageInput.java:153)
~[graylog.jar:?]
2017-02-13_12:36:08.45928 ... 7 more
2017-02-13_12:36:08.46113 Caused by: java.net.ConnectException: Connection
refused
2017-02-13_12:36:08.46239 at java.net.PlainSocketImpl.socketConnect(Native
Method) ~[?:1.8.0_101]
2017-02-13_12:36:08.46372 at
java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
~[?:1.8.0_101]
2017-02-13_12:36:08.46735 at
java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
~[?:1.8.0_101]
2017-02-13_12:36:08.47077 at
java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
~[?:1.8.0_101]
2017-02-13_12:36:08.47511 at
java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) ~[?:1.8.0_101]
2017-02-13_12:36:08.47630 at java.net.Socket.connect(Socket.java:589)
~[?:1.8.0_101]
2017-02-13_12:36:08.48921 at
com.rabbitmq.client.impl.FrameHandlerFactory.create(FrameHandlerFactory.java:47)
~[graylog.jar:?]
2017-02-13_12:36:08.52276 at
com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:822)
~[graylog.jar:?]
2017-02-13_12:36:08.52512 at
com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:778)
~[graylog.jar:?]
2017-02-13_12:36:08.53003 at
com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:868)
~[graylog.jar:?]
2017-02-13_12:36:08.53488 at
org.graylog2.inputs.transports.AmqpConsumer.connect(AmqpConsumer.java:176)
~[graylog.jar:?]
2017-02-13_12:36:08.54119 at
org.graylog2.inputs.transports.AmqpConsumer.run(AmqpConsumer.java:108)
~[graylog.jar:?]
2017-02-13_12:36:08.55159 at
org.graylog2.inputs.transports.AmqpTransport.doLaunch(AmqpTransport.java:176)
~[graylog.jar:?]
2017-02-13_12:36:08.57116 at
org.graylog2.plugin.inputs.transports.ThrottleableTransport.launch(ThrottleableTransport.java:75)
~[graylog.jar:?]
2017-02-13_12:36:08.57121 at
org.graylog2.plugin.inputs.MessageInput.launch(MessageInput.java:153)
~[graylog.jar:?]
2017-02-13_12:36:08.57762 ... 7 more
2017-02-13_12:36:08.58163 2017-02-13 13:36:08,361 INFO :
org.graylog2.inputs.InputStateListener - Input [GELF
AMQP/58a1a833ea84240352ab0c9e] is now TERMINATED
2017-02-13_12:36:08.58165 2017-02-13 13:36:08,345 ERROR:
com.google.common.eventbus.EventBus.graylog-eventbus - Exception thrown by
subscriber method
inputStateChanged(org.graylog2.plugin.events.inputs.IOStateChangedEvent) on
subscriber org.graylog2.inputs.InputStateListener@47629063 when dispatching
event: IOStateChangedEvent{oldState=STARTING, newState=FAILED,
changedState=InputState{stoppable=GELFAMQPInput{title=rabbitmq,
type=org.graylog2.inputs.gelf.amqp.GELFAMQPInput, nodeId=null},
state=FAILED, startedAt=2017-02-13T12:36:08.321Z, detailedMessage='null'}}
2017-02-13_12:36:08.59262 java.lang.NullPointerException
2017-02-13_12:36:08.59516 at
java.util.Objects.requireNonNull(Objects.java:203) ~[?:1.8.0_101]
2017-02-13_12:36:08.59523 at
org.graylog2.shared.system.activities.Activity.(Activity.java:34)
~[graylog.jar:?]
2017-02-13_12:36:08.62209 at
[graylog2] winlogbeats and graylog
Hello, First I would like to thanks this community for prompt answers. I would like to capture logs from windows. I installed winlogbeats and configured beats input on graylog. I can see that connection is estabilished, but I receive no answer. I used wireshark and curl to debug this, but for HTTP GET I receive no response. How to debug beats input on graylog? Thank you! Jiri -- You received this message because you are subscribed to the Google Groups "Graylog Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/graylog2/361d1def-efc5-427f-84b8-2cc2c629e746%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[graylog2] Graylog log sources
Hello, How to captures logs that are stored on database systems? Is there any collector for it? Thank you! -- You received this message because you are subscribed to the Google Groups "Graylog Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/graylog2/c05f8144-5266-4724-88b7-2c53073c1412%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[graylog2] Multi tenancy
Hi, just choosing open source solution that is capable of multi tenancy. Does anybody using graylog in MSSP environment? Have not found information regarding multi tenancy. Thank you for your answer! Best Regards, Jiri Kolb -- You received this message because you are subscribed to the Google Groups "Graylog Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/graylog2/8abf72ba-b807-49ba-825e-8b10df95f473%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
