[graylog2] 2 JSON Extractor questions

2016-06-21 Thread John Schneider
Hi, I would like to rewrite a message on an input before it hits the extractors. Where would I look to do this? I have JSON messages coming in from the fluent GELF plugin which adds it's own (numeric) level attribute which collides with the level we already have, I need to nuke one of them.

[graylog2] Shipping/Extracting Oracle logs

2016-06-21 Thread Michael Martin
* I posted this over on /r/graylog but I hope to get more visibility here as this group seems to be more active. Can anyone point me to information/documentation on how to get Oracle logs (alert,tnslistener,etc) shipped to and extracted into Graylog ? I have done some searching, but so far

[graylog2] Graylog Cluster pulling the same node information

2016-06-21 Thread William Thompson
Problem description The nodes listed in graylog.example.org/system/nodes is populating the same information. Steps to reproduce the problem Basic setup and configuration following doc.graylog.org using apache httpd. */etc/graylog/server/server.conf* # REST interface rest_listen_uri =

[graylog2] Re: graylog server warning every 5-30 minutes

2016-06-21 Thread Frederic Desjarlais
Would it make sense to increase the 'stale_master_timeout' setting to something like 5 minutes? What would be the issues to consider with a large cluster (say 32 Graylog Server nodes) having this set at 5 minutes (instead of 2000ms)? My understanding is that the master is only needed to run

[graylog2] Re: graylog server warning every 5-30 minutes

2016-06-21 Thread Ariel Godinez
Hello Jochen, Thanks for the response and paraphrase explanation, that helped me make more sense of what was going on. I took another look at my NTP configuration and as it turns out, the system clock wasn't syncing as it should have been. I fixed that, and the warnings from graylog stopped.

Re: [graylog2] server not running even though graylog-ctl says it is

2016-06-21 Thread 123Dev
Thanks Jan, To answer your questions. > did you check your /etc/hosts for this IP? > Not sure what you want me to check? confirm that it has a hostname defined? Server 1 ubuntu@graylog-server1:~/bak$ cat /etc/hosts 127.0.0.1 localhost # The following lines are desirable for IPv6 capable

[graylog2] Re: Graylog 2.0.2 fails to connect to Elasticsearch 2.3.3

2016-06-21 Thread ironmanmk42
Perfect. That worked. Graylog-server connected now properly to the elasticsearch. For the record: I set the following in my graylog-server/server.conf elasticsearch_network_host = Thanks much, -- You received this message because you are subscribed to the Google Groups "Graylog

[graylog2] Re: Graylog 2.0.2 fails to connect to Elasticsearch 2.3.3

2016-06-21 Thread Jochen Schalanda
Hi, try setting elasticsearch_network_host to an IP address (or host name) that is reachable for the other Elasticsearch nodes in the ES cluster ( https://github.com/Graylog2/graylog2-server/blob/2.0.3/misc/graylog.conf#L245-L250 ). Cheers, Jochen On Tuesday, 21 June 2016 16:02:37 UTC+2,

[graylog2] Graylog 2.0.2 fails to connect to Elasticsearch 2.3.3

2016-06-21 Thread ironmanmk42
I am testing out graylog 2.0.2 with elasticsearch 2.3.3 (with 2 nodes for now for elasticsearch, will become 3 to avoid split-brain ) and mongodb 2.4.14 (in 3-way replica set running on 2 graylog nodes and 1 mongo arbiter on the loadbalancer in front of graylog ). However, I'm constantly

[graylog2] Re: Alerts not getting triggered Graylog v2.0.1

2016-06-21 Thread Justin Hildreth
I am seeing the same behavior. Has anything further been determined about this issue? I have pulled up messages in search that I believe should have generated an alert, and then tested them against the appropriate stream. They do show as matching (green) for all conditions of the stream.

Re: [graylog2] Re: Stream Stopped - Audit Stream Activity

2016-06-21 Thread Michael Brosnan
*Thanks* Jochen, > I guess as the alert has 4 x rules in it, it somehow it is taking too long. For mitigation purposes, I will create a stream an alert for StreamFaultManager so at least I will know if a stream goes down and can divide up the rules if it continues. Is there anywhere to get a

[graylog2] Re: Elasticsearch cluster unhealthy (RED)

2016-06-21 Thread Jochen Schalanda
Hi Arief, a cluster health status of YELLOW is "good enough", but not ideal. Make sure that there are no unassigned shards or replicas in your Elasticsearch cluster. Cheers, Jochen On Tuesday, 21 June 2016 06:39:18 UTC+2, Arief Hydayat wrote: > > Hi Jochen, > > Thanks for your reply. I did

Re: [graylog2] Re: Stream Stopped - Audit Stream Activity

2016-06-21 Thread Jan Doberstein
Hi Michael, On 21. Juni 2016 at 13:34:22, Michael Brosnan (brosnan.mich...@gmail.com) wrote: > I know have found the error. Any idea what might have caused this? > > "WARN [StreamFaultManager] Processing of stream failed > to return within 2000ms" i guess that your elasticsearch is under load

Re: [graylog2] Re: Stream Stopped - Audit Stream Activity

2016-06-21 Thread Jochen Schalanda
Hi Michael, streams that are taking too long to compute automatically paused by Graylog, see https://github.com/Graylog2/graylog2-server/blob/2.0.3/misc/graylog.conf#L350-L358 for relevant configuration settings. Cheers, Jochen On Tuesday, 21 June 2016 13:34:22 UTC+2, Michael Brosnan

[graylog2] Re: parsing syslog messages on many services

2016-06-21 Thread Jochen Schalanda
Hi, On Tuesday, 21 June 2016 12:58:13 UTC+2, Андрей Грошев wrote: > > In the case of "pipelines" each string will processed two times, > This may have an effect under heavy loads. > Right? > Yes, correct. Cheers, Jochen -- You received this message because you are subscribed to the Google

Re: [graylog2] server not running even though graylog-ctl says it is

2016-06-21 Thread Jan Doberstein
Hej, On 20. Juni 2016 at 16:37:48, 123Dev (hr...@123loadboard.com) wrote: > > What happens if you kill the curl and try to restart graylag-server? > > > > Aha, Thanks for pointing that out > Graylog server starts > The entire steps below. > > It looks like Graylog-Server is trying to a local

Re: [graylog2] Re: Stream Stopped - Audit Stream Activity

2016-06-21 Thread Michael Brosnan
Hi Jan, thanks for responding. Actually I have found the Stream ID in the alert emails that this stream generates. I know have found the error. Any idea what might have caused this? "WARN [StreamFaultManager] Processing of stream failed to return within 2000ms" On Tue, Jun 21, 2016 at 12:13

Re: [graylog2] Re: Stream Stopped - Audit Stream Activity

2016-06-21 Thread Jan Doberstein
Dear Michael, what is your last action you had done? What can you find in your graylog server log file? regards Jan On 21. Juni 2016 at 13:02:36, Michael Brosnan (brosnan.mich...@gmail.com) wrote: > > When I click "Start Stream", I get "Resuming Stream failed with status: > Unauthorized" > >

[graylog2] Re: Stream Stopped - Audit Stream Activity

2016-06-21 Thread Michael Brosnan
When I click "Start Stream", I get "Resuming Stream failed with status: Unauthorized" On Tuesday, June 21, 2016 at 11:55:36 AM UTC+1, Michael Brosnan wrote: > > Hi all, > > I have a stream that (seemingly) paused by itself. Any reason why this > might happen on graylog 1.3? > > Also, is there

[graylog2] Re: Stream Stopped - Audit Stream Activity

2016-06-21 Thread Michael Brosnan
When I click Start Stream I get "Resuming Stream failed with status: Unauthorized" Hi all, > > I have a stream that (seemingly) paused by itself. Any reason why this > might happen on graylog 1.3? > > Also, is there a way to search the activity history of streams ... e.g > identify if a stream

[graylog2] Re: parsing syslog messages on many services

2016-06-21 Thread Андрей Грошев
> > > Using the new processing pipelines in Graylog 2.x (see > http://docs.graylog.org/en/2.0/pages/pipelines.html for details), you > could also use 1 input and run different rules for each source > device/service. > In the case of "pipelines" each string will processed two times, This may

[graylog2] Stream Stopped - Audit Stream Activity

2016-06-21 Thread Michael Brosnan
Hi all, I have a stream that (seemingly) paused by itself. Any reason why this might happen on graylog 1.3? Also, is there a way to search the activity history of streams ... e.g identify if a stream had been stopped by a user - has a stream some identifier? Thank you. -- You received

[graylog2] Re: graylog server warning every 5-30 minutes

2016-06-21 Thread Jochen Schalanda
Hi Ariel, just for reference, I'll paraphrase the explanation from IRC: Each Graylog node "registers" itself (node id, URI to the Graylog REST API, > timestamp of the last heartbeat) in MongoDB (see the nodes collection). > The timeout/cleanup interval is quite aggressive (2s, see >

Re: [graylog2] Could not create extractor (404)

2016-06-21 Thread Dietmar Schurr
Hello Edmundo, now I configured NTP to work with our local time server and the cluster works better (I can see the inputs on the both cluster nodes again). May be you can add a hint in the documentation to (re)configure NTP if your graylog servers don't have access to public time servers.