[graylog2] missing alerts menu

[Wallace Turner]

my (latest) graylog installation is missing the 'Alerts' menu item




I'm trying to add/view the alerts. the docs at this page indicate it should 
 be present

http://docs.graylog.org/en/2.2/pages/getting_started/stream_alerts.html

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/9af23d16-ce47-49a3-8e13-a90cf8fdd164%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: graylog api not working for gelf - 404

[Wallace Turner]

all good i figured it out after posting. i had to add an input (System -> 
Input -> Launch new Input) - which listened on port 12201
working now, RTFM

On Friday, 10 February 2017 07:57:30 UTC+8, Wallace Turner wrote:
>
> I am sending a log message to the api 
>
> POST http://192.168.0.9:9000/gelf HTTP/1.1
> Authorization: Basic YWRtaW46YWRtaW4=
> Content-Type: application/json; charset=utf-8
> Host: 192.168.0.9:9000
> Content-Length: 396
> Connection: Keep-Alive
>
> {
>   "facility": "RandomPhrases",
>   "file": "?",
>   "host": "DESKTOP-U6AFHCT",
>   "level": 6,
>   "line": "?",
>   "timestamp": "1486684392.85276",
>   "version": "1.0",
>   "LoggerName": "ConsoleApplication2.Program",
>   "full_message": "[7] Program - foo",
>   "short_message": "[7] Program - foo",
>   "_app": "RandomSentence",
>   "_version": "1.0",
>   "_Environment": "Dev",
>   "_Level": "INFO"
> }
>
> and the response back from the server
>
> HTTP/1.1 404 Not Found
> X-Graylog-Node-ID: 204cca32-be19-4980-b35a-bac6b102780f
> Content-Type: application/json
> Date: Thu, 09 Feb 2017 23:53:11 GMT
> Content-Length: 50
>
> {"type":"ApiError","message":"HTTP 404 Not Found"}
>
> Please tell me what i am doing wrong? according to the docs the path is 
> correct (/gelf) and the server is listening as its otherwise responding..
>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/2e9e9390-d3bc-444b-81fa-7dbb008838cb%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] graylog api not working for gelf - 404

[Wallace Turner]

I am sending a log message to the api 

POST http://192.168.0.9:9000/gelf HTTP/1.1
Authorization: Basic YWRtaW46YWRtaW4=
Content-Type: application/json; charset=utf-8
Host: 192.168.0.9:9000
Content-Length: 396
Connection: Keep-Alive

{
  "facility": "RandomPhrases",
  "file": "?",
  "host": "DESKTOP-U6AFHCT",
  "level": 6,
  "line": "?",
  "timestamp": "1486684392.85276",
  "version": "1.0",
  "LoggerName": "ConsoleApplication2.Program",
  "full_message": "[7] Program - foo",
  "short_message": "[7] Program - foo",
  "_app": "RandomSentence",
  "_version": "1.0",
  "_Environment": "Dev",
  "_Level": "INFO"
}

and the response back from the server

HTTP/1.1 404 Not Found
X-Graylog-Node-ID: 204cca32-be19-4980-b35a-bac6b102780f
Content-Type: application/json
Date: Thu, 09 Feb 2017 23:53:11 GMT
Content-Length: 50

{"type":"ApiError","message":"HTTP 404 Not Found"}

Please tell me what i am doing wrong? according to the docs the path is 
correct (/gelf) and the server is listening as its otherwise responding..

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/1c5e7b74-df77-4ea2-839f-77083c98e82e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Does graylog automatically detect duplicate messages on ingest?

[Matthew Shapiro]

I am evaluating Graylog in order for us to manage both log analysis and 
alerts for our applications.  Right now our Azure web applications are 
writing structured logs (in JSON) into file storage and I am trying to get 
those logs into Graylog.

I created an input, ingested some logs, then created some extractors to get 
information from those logs.  However, since those logs were already 
ingested prior to the extractors I need to re-ingest them again so that 
they get run through the extractors properly this time.

Due to the setup that we have, it's technically possible I will need to 
re-add my logs into Graylog and would like to remove the possibility of 
duplicates from my statistics.  While experimenting with the ELK stack I 
did this by telling logstash to set the document_id property in the output 
to the ID I specified (in this case every event I generate has a GUID field 
that can be used to uniquely identify each message).

However, when I tried to create an extractor that took my Id field and cut 
it to _id I got a 400 error, so it seems this is explicitly disallowed by 
Graylog.

Does Graylog have any detection of duplicate messages to overwrite, and if 
not is there any way to force an id on a message via an extractor?

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/a67e3b8e-6259-43bf-bb5c-8ea371048e22%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Query to get report for number of logon attempts??

[mark . a . lindquist]

I am NEW to Debian and barely got through the GrayLog 2 installation 
(alive).  It is up and running and working.  I need to get good data from 
this but I seem to be missing HOW.  I have read the graylog documentation 
and have used it to create some generic "reports" and added to the 
"Dashboard" but I need better data.  I went through every single entry and 
added anything that might be usable to the "dashboard".  

How do I get a report/query that will show the number of logon attempts 
(per minute, hour, day, etc.).  What I have tried does not work or produces 
nothing.

I need to get this added to view the data, also need data (such as this) 
for government reporting purposes.

Can anyone assist?

Mark

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/f6b39f6e-e46a-41f7-b15d-d3c3341bcf84%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: Monitoring Windows DHCP Server Activity

[Rob Repp]

The files are definitely updating. One interesting thing, I tried do 
establish this by just tailing the file with both Notepad++ and with a 
freeware "tail" utility for Windows and it never updated. I had to manually 
reload the file to see any changes. Further, I never saw any update in the 
file Date Modified. Is there some way to force collector sidecar to poll 
the files even if they don't show any obvious activity?

On Tuesday, February 7, 2017 at 1:55:07 AM UTC-6, Jochen Schalanda wrote:
>
> Hi Rob,
>
> this sounds like either there is simply no new content in the files you've 
> configured nxlog to watch, or that the file pattern is wrong. Try using 
> another File pattern in the nxlog im_file input or switch to Filebeat.
>
> Cheers,
> Jochen
>
> On Monday, 6 February 2017 23:22:59 UTC+1, Rob Repp wrote:
>>
>> Okay, I did a packet capture that's showing traffic between the two 
>> boxes. There seems to be the Graylog host sending a json of the nxlog.conf 
>> config data to the DHCP server once every four seconds or so, and the DHCP 
>> server sending back HTTP requests on port 9000. None of the exchanges look 
>> like they contain data from the DHCP logs.
>>
>> On Monday, February 6, 2017 at 10:37:44 AM UTC-6, Jochen Schalanda wrote:
>>>
>>> Hi Rob,
>>>
>>> since the configuration doesn't show any obvious errors, please use 
>>> Wireshark or a similar tool like tcpdump to check if the log messages from 
>>> nxlog are sent to the correct host and if the UDP packets actually arrive 
>>> at the Graylog GELF UDP input.
>>>
>>> Cheers,
>>> Jochen
>>>
>>> On Monday, 6 February 2017 17:08:21 UTC+1, Rob Repp wrote:

 The traffic is not being blocked. There's no firewall on either 
 machine, and the network path is unobstructed. Further, the Collector 
 status for that Collector is showing green, with Backend "Nxlog: running." 
 It looks like it's connected and responsive. It's just that there never 
 seem to be any messages on the associated Input.
 Tks,
 R.

 On Saturday, February 4, 2017 at 3:30:18 AM UTC-6, Jochen Schalanda 
 wrote:
>
> Hi Rob,
>
> the configuration looks good so far. Make sure that the host 
> "re.da.ct.ed" can be accessed by your Windows machine and that port 
> 5441/udp is open and not blocked by a firewall.
>
> Cheers,
> Jochen
>
> On Friday, 3 February 2017 23:10:50 UTC+1, Rob Repp wrote:
>>
>> Okay, in order:
>>
>> 1. I'm using the OVA VM image from Graylog, so most of the 
>> configuration is already done. All I did was add a Connector with one 
>> nxlog 
>> input and one nxlog output, and then the GELF UDP input that the WinDHCP 
>> json created.
>>
>> The WinDHCP input is configured like this:
>>
>> WinDHCPLogs-gelf GELF UDP RUNNING
>> On node 771f3128 / graylog 
>> 
>>
>>- bind_address:
>>0.0.0.0
>>- decompress_size_limit:
>>8388608
>>- override_source:
>>**
>>- port:
>>5441
>>- recv_buffer_size:
>>1048576
>>
>>
>> 2. The nxlog.conf file is:
>>
>> define ROOT C:\Program Files (x86)\nxlog
>>
>> 
>>   Module xm_gelf
>> 
>>
>> Moduledir %ROOT%\modules
>> CacheDir %ROOT%\data
>> Pidfile %ROOT%\data\nxlog.pid
>> SpoolDir %ROOT%\data
>> LogFile %ROOT%\data\nxlog.log
>> LogLevel INFO
>>
>> 
>> Module  xm_fileop
>> 
>> When@daily
>> Execfile_cycle('%ROOT%\data\nxlog.log', 7);
>>  
>> 
>>
>> 
>> Module im_file
>> File 'C:\Windows\System32\dhcp\DhcpSrvLog-*.log'
>> PollInterval 1
>> SavePos True
>> ReadFromLast True
>> Recursive False
>> RenameCheck True
>> Exec $FileName = file_name(); # Send file name with each message
>> 
>>
>> 
>> Module om_udp
>> Host re.da.ct.ed
>> Port 5441
>> OutputType  GELF
>> Exec $short_message = $raw_event; # Avoids truncation of the 
>> short_message field.
>> Exec $gl2_source_collector = '9960a8cd-7abe-4021-939f-89b22909aa32';
>> Exec $Hostname = hostname_fqdn();
>> 
>>
>> 
>>   Path 588bc33f682c990374bab049 => 588bc2db682c990374baafe0
>> 
>>
>> 3. collector_sidecar.yml is this:
>>
>> server_url: http://re.da.ct.ed:9000/api 
>> update_interval: 10
>> tls_skip_verify: false
>> send_status: true
>> list_log_files:
>> node_id: NS1
>> collector_id: file:C:\Program 
>> Files\graylog\collector-sidecar\collector-id
>> cache_path: C:\Program Files\graylog\collector-sidecar\cache
>> log_path: C:\Program Files\graylog\collector-sidecar\logs
>> log_rotation_time: 86400
>> log_max_age: 604800
>> tags: dhcp
>> backends:
>> - name: nxlog
>>   enabled: true

[graylog2] Re: GrayLog 2.1.3 - Error injecting constructor, java.lang.NullPointerException

[David Casey]

Just upgraded to 2.2.0 and it's working.  Disregard!

On Thursday, February 9, 2017 at 11:02:21 AM UTC-7, David Casey wrote:
>
> I'm having a heck of a time getting GrayLog to start up the web interface 
> part.  The service starts up fine but the web interface never shows up. 
>  I'm seeing the following error in the server.log file:
>
> 2017-02-09T09:57:57.568-07:00 INFO  [CmdLineTool] Loaded plugin: Elastic 
> Beats Input 1.1.5 [org.graylog.plugins.beats.BeatsInputPlugin]
> 2017-02-09T09:57:57.570-07:00 INFO  [CmdLineTool] Loaded plugin: Collector 
> 1.1.3 [org.graylog.plugins.collector.CollectorPlugin]
> 2017-02-09T09:57:57.570-07:00 INFO  [CmdLineTool] Loaded plugin: 
> Enterprise Integration Plugin 1.1.3 
> [org.graylog.plugins.enterprise_integration.EnterpriseIntegrationPlugin]
> 2017-02-09T09:57:57.571-07:00 INFO  [CmdLineTool] Loaded plugin: 
> MapWidgetPlugin 1.1.3 [org.graylog.plugins.map.MapWidgetPlugin]
> 2017-02-09T09:57:57.571-07:00 INFO  [CmdLineTool] Loaded plugin: Pipeline 
> Processor Plugin 1.1.3 
> [org.graylog.plugins.pipelineprocessor.ProcessorPlugin]
> 2017-02-09T09:57:57.571-07:00 INFO  [CmdLineTool] Loaded plugin: Anonymous 
> Usage Statistics 2.1.3 
> [org.graylog.plugins.usagestatistics.UsageStatsPlugin]
> 2017-02-09T09:57:57.671-07:00 INFO  [CmdLineTool] Running with JVM 
> arguments: -Xms1g -Xmx1g -XX:NewRatio=1 -XX:+ResizeTLAB 
> -XX:+UseConcMarkSweepGC -XX:+CMSConcurrentMTEnabled 
> -XX:+CMSClassUnloadingEnabled -XX:+UseParNewGC 
> -XX:-OmitStackTraceInFastThrow 
> -Dlog4j.configurationFile=file:///etc/graylog/server/log4j2.xml 
> -Djava.library.path=/usr/share/graylog-server/lib/sigar 
> -Dgraylog2.installation_source=deb
> 2017-02-09T09:57:57.876-07:00 INFO  [Version] HV01: Hibernate 
> Validator 5.2.4.Final
> 2017-02-09T09:58:00.067-07:00 INFO  [InputBufferImpl] Message journal is 
> enabled.
> 2017-02-09T09:58:00.089-07:00 INFO  [NodeId] Node ID: 
> 7b717cb0-eac2-4097-9f92-4a5ac9b81ef3
> 2017-02-09T09:58:00.293-07:00 INFO  [LogManager] Loading logs.
> 2017-02-09T09:58:00.341-07:00 INFO  [LogManager] Logs loading complete.
> 2017-02-09T09:58:00.342-07:00 INFO  [KafkaJournal] Initialized Kafka based 
> journal at /var/lib/graylog-server/journal
> 2017-02-09T09:58:00.358-07:00 INFO  [InputBufferImpl] Initialized 
> InputBufferImpl with ring size <65536> and wait strategy 
> , running 2 parallel message handlers.
> 2017-02-09T09:58:00.376-07:00 INFO  [cluster] Cluster created with 
> settings {hosts=[localhost:27017], mode=SINGLE, 
> requiredClusterType=UNKNOWN, serverSelectionTimeout='3 ms', 
> maxWaitQueueSize=5000}
> 2017-02-09T09:58:00.426-07:00 INFO  [cluster] No server chosen by 
> ReadPreferenceServerSelector{readPreference=primary} from cluster 
> description ClusterDescription{type=UNKNOWN, connectionMode=SINGLE, 
> serverDescriptions=[ServerDescription{address=localhost:27017, 
> type=UNKNOWN, state=CONNECTING}]}. Waiting for 3 ms before timing out
> 2017-02-09T09:58:00.438-07:00 INFO  [connection] Opened connection 
> [connectionId{localValue:1, serverValue:9}] to localhost:27017
> 2017-02-09T09:58:00.439-07:00 INFO  [cluster] Monitor thread successfully 
> connected to server with description 
> ServerDescription{address=localhost:27017, type=STANDALONE, 
> state=CONNECTED, ok=true, version=ServerVersion{versionList=[2, 6, 10]}, 
> minWireVersion=0, maxWireVersion=2, maxDocumentSize=16777216, 
> roundTripTimeNanos=512192}
> 2017-02-09T09:58:00.448-07:00 INFO  [connection] Opened connection 
> [connectionId{localValue:2, serverValue:10}] to localhost:27017
> 2017-02-09T09:58:00.770-07:00 INFO  [node] 
> [graylog-7b717cb0-eac2-4097-9f92-4a5ac9b81ef3] version[2.3.5], pid[28236], 
> build[90f439f/2016-07-27T10:36:52Z]
> 2017-02-09T09:58:00.770-07:00 INFO  [node] 
> [graylog-7b717cb0-eac2-4097-9f92-4a5ac9b81ef3] initializing ...
> 2017-02-09T09:58:00.776-07:00 INFO  [plugins] 
> [graylog-7b717cb0-eac2-4097-9f92-4a5ac9b81ef3] modules [], plugins 
> [graylog-monitor], sites []
> 2017-02-09T09:58:02.438-07:00 INFO  [node] 
> [graylog-7b717cb0-eac2-4097-9f92-4a5ac9b81ef3] initialized
> 2017-02-09T09:58:02.576-07:00 INFO  [ProcessBuffer] Initialized 
> ProcessBuffer with ring size <65536> and wait strategy 
> .
> 2017-02-09T09:58:04.383-07:00 INFO  [RulesEngineProvider] No static rules 
> file loaded.
> 2017-02-09T09:58:04.431-07:00 INFO  [OutputBuffer] Initialized 
> OutputBuffer with ring size <65536> and wait strategy 
> .
> 2017-02-09T09:58:04.640-07:00 INFO  [ProcessBuffer] Initialized 
> ProcessBuffer with ring size <65536> and wait strategy 
> .
> 2017-02-09T09:58:04.762-07:00 INFO  [ProcessBuffer] Initialized 
> ProcessBuffer with ring size <65536> and wait strategy 
> .
> 2017-02-09T09:58:05.052-07:00 INFO  [ProcessBuffer] Initialized 
> ProcessBuffer with ring size <65536> and wait strategy 
> .
> 2017-02-09T09:58:05.157-07:00 INFO  [ProcessBuffer] Initialized 
> ProcessBuffer with ring size <65536> and wait strategy 
> .
> 2017-02-09T09:58:05.240-07:

[graylog2] GrayLog 2.1.3 - Error injecting constructor, java.lang.NullPointerException

[David Casey]

I'm having a heck of a time getting GrayLog to start up the web interface 
part.  The service starts up fine but the web interface never shows up. 
 I'm seeing the following error in the server.log file:

2017-02-09T09:57:57.568-07:00 INFO  [CmdLineTool] Loaded plugin: Elastic 
Beats Input 1.1.5 [org.graylog.plugins.beats.BeatsInputPlugin]
2017-02-09T09:57:57.570-07:00 INFO  [CmdLineTool] Loaded plugin: Collector 
1.1.3 [org.graylog.plugins.collector.CollectorPlugin]
2017-02-09T09:57:57.570-07:00 INFO  [CmdLineTool] Loaded plugin: Enterprise 
Integration Plugin 1.1.3 
[org.graylog.plugins.enterprise_integration.EnterpriseIntegrationPlugin]
2017-02-09T09:57:57.571-07:00 INFO  [CmdLineTool] Loaded plugin: 
MapWidgetPlugin 1.1.3 [org.graylog.plugins.map.MapWidgetPlugin]
2017-02-09T09:57:57.571-07:00 INFO  [CmdLineTool] Loaded plugin: Pipeline 
Processor Plugin 1.1.3 
[org.graylog.plugins.pipelineprocessor.ProcessorPlugin]
2017-02-09T09:57:57.571-07:00 INFO  [CmdLineTool] Loaded plugin: Anonymous 
Usage Statistics 2.1.3 
[org.graylog.plugins.usagestatistics.UsageStatsPlugin]
2017-02-09T09:57:57.671-07:00 INFO  [CmdLineTool] Running with JVM 
arguments: -Xms1g -Xmx1g -XX:NewRatio=1 -XX:+ResizeTLAB 
-XX:+UseConcMarkSweepGC -XX:+CMSConcurrentMTEnabled 
-XX:+CMSClassUnloadingEnabled -XX:+UseParNewGC 
-XX:-OmitStackTraceInFastThrow 
-Dlog4j.configurationFile=file:///etc/graylog/server/log4j2.xml 
-Djava.library.path=/usr/share/graylog-server/lib/sigar 
-Dgraylog2.installation_source=deb
2017-02-09T09:57:57.876-07:00 INFO  [Version] HV01: Hibernate Validator 
5.2.4.Final
2017-02-09T09:58:00.067-07:00 INFO  [InputBufferImpl] Message journal is 
enabled.
2017-02-09T09:58:00.089-07:00 INFO  [NodeId] Node ID: 
7b717cb0-eac2-4097-9f92-4a5ac9b81ef3
2017-02-09T09:58:00.293-07:00 INFO  [LogManager] Loading logs.
2017-02-09T09:58:00.341-07:00 INFO  [LogManager] Logs loading complete.
2017-02-09T09:58:00.342-07:00 INFO  [KafkaJournal] Initialized Kafka based 
journal at /var/lib/graylog-server/journal
2017-02-09T09:58:00.358-07:00 INFO  [InputBufferImpl] Initialized 
InputBufferImpl with ring size <65536> and wait strategy 
, running 2 parallel message handlers.
2017-02-09T09:58:00.376-07:00 INFO  [cluster] Cluster created with settings 
{hosts=[localhost:27017], mode=SINGLE, requiredClusterType=UNKNOWN, 
serverSelectionTimeout='3 ms', maxWaitQueueSize=5000}
2017-02-09T09:58:00.426-07:00 INFO  [cluster] No server chosen by 
ReadPreferenceServerSelector{readPreference=primary} from cluster 
description ClusterDescription{type=UNKNOWN, connectionMode=SINGLE, 
serverDescriptions=[ServerDescription{address=localhost:27017, 
type=UNKNOWN, state=CONNECTING}]}. Waiting for 3 ms before timing out
2017-02-09T09:58:00.438-07:00 INFO  [connection] Opened connection 
[connectionId{localValue:1, serverValue:9}] to localhost:27017
2017-02-09T09:58:00.439-07:00 INFO  [cluster] Monitor thread successfully 
connected to server with description 
ServerDescription{address=localhost:27017, type=STANDALONE, 
state=CONNECTED, ok=true, version=ServerVersion{versionList=[2, 6, 10]}, 
minWireVersion=0, maxWireVersion=2, maxDocumentSize=16777216, 
roundTripTimeNanos=512192}
2017-02-09T09:58:00.448-07:00 INFO  [connection] Opened connection 
[connectionId{localValue:2, serverValue:10}] to localhost:27017
2017-02-09T09:58:00.770-07:00 INFO  [node] 
[graylog-7b717cb0-eac2-4097-9f92-4a5ac9b81ef3] version[2.3.5], pid[28236], 
build[90f439f/2016-07-27T10:36:52Z]
2017-02-09T09:58:00.770-07:00 INFO  [node] 
[graylog-7b717cb0-eac2-4097-9f92-4a5ac9b81ef3] initializing ...
2017-02-09T09:58:00.776-07:00 INFO  [plugins] 
[graylog-7b717cb0-eac2-4097-9f92-4a5ac9b81ef3] modules [], plugins 
[graylog-monitor], sites []
2017-02-09T09:58:02.438-07:00 INFO  [node] 
[graylog-7b717cb0-eac2-4097-9f92-4a5ac9b81ef3] initialized
2017-02-09T09:58:02.576-07:00 INFO  [ProcessBuffer] Initialized 
ProcessBuffer with ring size <65536> and wait strategy 
.
2017-02-09T09:58:04.383-07:00 INFO  [RulesEngineProvider] No static rules 
file loaded.
2017-02-09T09:58:04.431-07:00 INFO  [OutputBuffer] Initialized OutputBuffer 
with ring size <65536> and wait strategy .
2017-02-09T09:58:04.640-07:00 INFO  [ProcessBuffer] Initialized 
ProcessBuffer with ring size <65536> and wait strategy 
.
2017-02-09T09:58:04.762-07:00 INFO  [ProcessBuffer] Initialized 
ProcessBuffer with ring size <65536> and wait strategy 
.
2017-02-09T09:58:05.052-07:00 INFO  [ProcessBuffer] Initialized 
ProcessBuffer with ring size <65536> and wait strategy 
.
2017-02-09T09:58:05.157-07:00 INFO  [ProcessBuffer] Initialized 
ProcessBuffer with ring size <65536> and wait strategy 
.
2017-02-09T09:58:05.240-07:00 INFO  [ProcessBuffer] Initialized 
ProcessBuffer with ring size <65536> and wait strategy 
.
2017-02-09T09:58:05.408-07:00 INFO  [ProcessBuffer] Initialized 
ProcessBuffer with ring size <65536> and wait strategy 
.
2017-02-09T09:58:05.501-07:00 INFO  [ProcessBuffer] Initialized 
ProcessBuffer with ring 

[graylog2] Re: Graylog is ignoring some UDP packets sent by a particular host

[IJFK]

Maybe you ran into the same issue I had, the time difference. Is the time 
set correctly on the router and does it match the Graylog server?

If it works when you are relaying through another device, then that device 
may update the timestamp and make it work.

I'm obviously a greenhorn but it may be worth a look.


On Thursday, February 9, 2017 at 5:50:58 AM UTC-6, tomaszik...@gmail.com 
wrote:
>
>
>
> On Thursday, February 9, 2017 at 10:00:49 AM UTC+1, Jochen Schalanda wrote:
>>
>> Hi,
>>
>> Graylog itself doesn't care where the packets are coming from.
>>
>> Is the routing to Graylog working for the "ignored" host?
>> Is the networking set up correctly on all hosts?
>> Are there any firewall rules in place?
>> How did you configure the Syslog UDP and the Raw/Plaintext UDP inputs?
>>
>> Cheers,
>> Jochen
>>
>> On Wednesday, 8 February 2017 19:43:38 UTC+1, tomaszik...@gmail.com 
>> wrote:
>>>
>>> Hello,
>>>
>>> I've recently set up a working Graylog server. It's collecting logs from 
>>> many network switches and routers. One particular router (ironically, the 
>>> most important one) doesn't appear in the Sources list though. Graylog 
>>> keeps ignoring all packets coming from that host. Here's an example of a 
>>> packet which is *not* ignored by Graylog:
>>>
>>> 19:12:15.705167 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto 
>>> UDP (17), length 115)
>>> 10.50.255.44.40810 > Silenoz.syslog: [udp sum ok] [|syslog]
>>>  0x:  4500 0073  4000 4011 27e3 0a32 ff2c  E..s..@.@.'..2.,
>>>  0x0010:  0a32 ff06 9f6a 0202 005f 01d1 6468 6370  .2...j..._..dhcp
>>>  0x0020:  2c77 6172 6e69 6e67 2067 706f 6e2d 6d6e  ,warning.gpon-mn
>>>  0x0030:  6720 6f66 6665 7269 6e67 206c 6561 7365  g.offering.lease
>>>  0x0040:  2031 302e 3530 2e32 3338 2e33 3520 666f  .10.50.238.35.fo
>>>  0x0050:  7220 3030 3a30 323a 3731 3a35 413a 3036  r.00:02:71:5A:06
>>>  0x0060:  3a42 3820 7769 7468 6f75 7420 7375 6363  :B8.without.succ
>>>  0x0070:  6573 73 
>>>
>>> And below you can see a packet which *is* ignored by Graylog:
>>>
>>> 10.50.255.111.56993 > Silenoz.syslog: [udp sum ok] SYSLOG, length: 
>>> 154
>>>  Facility local7 (23), Severity notice (5)
>>>  Msg: Feb 8 19:12:17: %SYSLOG-5-NOTICE: aaad: SubSessionAUTHFAIL user: 
>>> pppoe16344@mn (24) Authentication failure [Circuit handle: 1/4:511:63:31
>>> /6/2/47661]\0x0a
>>>  0x:  3c31 3839 3e46 6562 2038 2031 393a 3132
>>>  0x0010:  3a31 373a 2025 5359 534c 4f47 2d35 2d4e
>>>  0x0020:  4f54 4943 453a 2061 6161 643a 2053 7562
>>>  0x0030:  5365 7373 696f 6e41 5554 4846 4149 4c20
>>>  0x0040:  7573 6572 3a20 7070 706f 6531 3633 3434
>>>  0x0050:  406d 6e20 2832 3429 2041 7574 6865 6e74
>>>  0x0060:  6963 6174 696f 6e20 6661 696c 7572 6520
>>>  0x0070:  5b43 6972 6375 6974 2068 616e 646c 653a
>>>  0x0080:  2031 2f34 3a35 3131 3a36 333a 3331 2f36
>>>  0x0090:  2f32 2f34 3736 3631 5d0a
>>>  0x:  4500 00b6 77da  4011 ef82 0a32 ff6f  E...w...@2.o
>>>  0x0010:  0a32 ff06 dea1 0202 00a2 28d8 3c31 3839  .2(.<189
>>>  0x0020:  3e46 6562 2038 2031 393a 3132 3a31 373a  >Feb.8.19:12:17:
>>>  0x0030:  2025 5359 534c 4f47 2d35 2d4e 4f54 4943  .%SYSLOG-5-NOTIC
>>>  0x0040:  453a 2061 6161 643a 2053 7562 5365 7373  E:.aaad:.SubSess
>>>  0x0050:  696f 6e41 5554 4846 4149 4c20 7573 6572  ionAUTHFAIL.user
>>>  0x0060:  3a20 7070 706f 6531 3633 3434 406d 6e20  :.pppoe16344@mn.
>>>  0x0070:  2832 3429 2041 7574 6865 6e74 6963 6174  (24).Authenticat
>>>  0x0080:  696f 6e20 6661 696c 7572 6520 5b43 6972  ion.failure.[Cir
>>>  0x0090:  6375 6974 2068 616e 646c 653a 2031 2f34  cuit.handle:.1/4
>>>  0x00a0:  3a35 3131 3a36 333a 3331 2f36 2f32 2f34  :511:63:31/6/2/4
>>>  0x00b0:  3736 3631 5d0a   7661].
>>>
>>> As you can see, the packet is much longer, but it doesn't exceed the 
>>> maximum UDP packet size that can be processed by Graylog (8192). My guess 
>>> is that logs coming from 10.50.255.111 are not RFC compatible and thus 
>>> they're discarded by Graylog. How can I debug it / fix it? I didn't find 
>>> any related messages in the Elasticsearch log (there were no errors related 
>>> to parsing a message).
>>> I deleted the default Input object and added a new RAW UDP Input object. 
>>> It didn't fix the issue - logs from 10.50.255.111 are still not parsed.
>>>
>>
> Thanks for your answer Jochen. It's not a problem with routing nor with 
> firewall. I can capture these packets with TCPDump (as shown in my post) on 
> the very same machine, on which the Graylog server is running. I don't have 
> a Syslog UDP input, I deleted it and created a Raw UDP Input instead. 
> Here's the configuration of this input:
> - Type: Raw/Plaintext UDP
> - Title: Main
> - Global: unchecked
> - Node: 01fd7feb/Silenoz (that's the only node I have)
> - Bind address: 10.50.255.6
> - Port: 10514
> - Buffer size: 262144
>
> As you can see I'm using the 10514 port, not 514. I can't use 514, because 
> the Graylog server would have to run with root privileges.

[graylog2] Re: Incoming Gelf UDP messages not showing up

[IJFK]

I think I have an idea as to what is going on, it looks to be a time zone / 
time issue. The packets were formatted correctly, hence no parsing errors. 
I was able to verify that by sending malformed packets and observing 
parsing errors.

However the packets sent by the test app seem to have been sent with a 
timestamp of local time, whereas the VM appliance running Graylog is in 
UTC. Subsequently the messages don't show up because they technically 
happened in the past.

On Thursday, February 9, 2017 at 2:50:46 AM UTC-6, Jochen Schalanda wrote:
>
> Hi,
>
> On Thursday, 9 February 2017 06:54:30 UTC+1, IJFK wrote:
>>
>> I'm sending Syslog packets in Gelf format (I successfully validated the 
>> Json), and no matter what I do, the packets don't show up. There is no 
>> parsing error or anything, the data just doesn't show up.
>>
>
> How exactly are you sending messages? How did you configure the clients? 
> How did you configure the inputs (and which types) in Graylog? 
>
>
> I already created a Raw/UDP input & stream, which does show the messages 
>> coming in, I also verified with tcpdump that they are actually making it to 
>> the server.
>>
>
> This sounds like they are simply not valid GELF messages.
>
> Cheers,
> Jochen
>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/0417175d-532f-4992-a9a1-49b78ef7eebd%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: collect logs from remote machine

[Jochen Schalanda]

Hi Wallace,

On Thursday, 9 February 2017 12:20:26 UTC+1, Wallace Turner wrote:
>
> Ho Jochen so you need to install Sidecar (and then possibly nxlog) on each 
> machine you want to watch a logfile?
>

You can use any other means of shipping the logs from your systems to 
Graylog that you like. You also don't have to use the Graylog Collector 
Sidecar, but it usually makes things easier.

Cheers,
Jochen

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/aa16fb75-0f86-4617-87e5-d68c03a5594c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: How to parse OpenVPN logs in Graylog?

[Jochen Schalanda]

Hi César,

first you have to ship the logs to your Graylog server, either by 
forwarding the messages via your syslog daemon on that system or by reading 
from a log file on that system.

See http://docs.graylog.org/en/2.2/pages/sending_data.html 
and http://docs.graylog.org/en/2.2/pages/collector_sidecar.html for details.

Then you need to extract the interesting information from the plaintext 
logs using extractors or message pipeline rules.

See http://docs.graylog.org/en/2.2/pages/extractors.html 
and http://docs.graylog.org/en/2.2/pages/pipelines.html for details.

Cheers,
Jochen

On Thursday, 9 February 2017 12:18:32 UTC+1, CESAR Fabre wrote:
>
> Hi guys,
>
> I have the PfSense 2.3.2 with OpenVPN enabled. I want to parse OpenVPN 
> logs in Graylog with Dashboards.
>
> I have no idea. Can you help me?
>
>
> Thanks a lot!
>
>
> César
>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/7e4152a7-2286-4798-b7fc-da3ac92851fb%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: Graylog is ignoring some UDP packets sent by a particular host

[tomaszikasperczyk]

On Thursday, February 9, 2017 at 10:00:49 AM UTC+1, Jochen Schalanda wrote:
>
> Hi,
>
> Graylog itself doesn't care where the packets are coming from.
>
> Is the routing to Graylog working for the "ignored" host?
> Is the networking set up correctly on all hosts?
> Are there any firewall rules in place?
> How did you configure the Syslog UDP and the Raw/Plaintext UDP inputs?
>
> Cheers,
> Jochen
>
> On Wednesday, 8 February 2017 19:43:38 UTC+1, tomaszik...@gmail.com wrote:
>>
>> Hello,
>>
>> I've recently set up a working Graylog server. It's collecting logs from 
>> many network switches and routers. One particular router (ironically, the 
>> most important one) doesn't appear in the Sources list though. Graylog 
>> keeps ignoring all packets coming from that host. Here's an example of a 
>> packet which is *not* ignored by Graylog:
>>
>> 19:12:15.705167 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto 
>> UDP (17), length 115)
>> 10.50.255.44.40810 > Silenoz.syslog: [udp sum ok] [|syslog]
>>  0x:  4500 0073  4000 4011 27e3 0a32 ff2c  E..s..@.@.'..2.,
>>  0x0010:  0a32 ff06 9f6a 0202 005f 01d1 6468 6370  .2...j..._..dhcp
>>  0x0020:  2c77 6172 6e69 6e67 2067 706f 6e2d 6d6e  ,warning.gpon-mn
>>  0x0030:  6720 6f66 6665 7269 6e67 206c 6561 7365  g.offering.lease
>>  0x0040:  2031 302e 3530 2e32 3338 2e33 3520 666f  .10.50.238.35.fo
>>  0x0050:  7220 3030 3a30 323a 3731 3a35 413a 3036  r.00:02:71:5A:06
>>  0x0060:  3a42 3820 7769 7468 6f75 7420 7375 6363  :B8.without.succ
>>  0x0070:  6573 73 
>>
>> And below you can see a packet which *is* ignored by Graylog:
>>
>> 10.50.255.111.56993 > Silenoz.syslog: [udp sum ok] SYSLOG, length: 
>> 154
>>  Facility local7 (23), Severity notice (5)
>>  Msg: Feb 8 19:12:17: %SYSLOG-5-NOTICE: aaad: SubSessionAUTHFAIL user: 
>> pppoe16344@mn (24) Authentication failure [Circuit handle: 1/4:511:63:31/
>> 6/2/47661]\0x0a
>>  0x:  3c31 3839 3e46 6562 2038 2031 393a 3132
>>  0x0010:  3a31 373a 2025 5359 534c 4f47 2d35 2d4e
>>  0x0020:  4f54 4943 453a 2061 6161 643a 2053 7562
>>  0x0030:  5365 7373 696f 6e41 5554 4846 4149 4c20
>>  0x0040:  7573 6572 3a20 7070 706f 6531 3633 3434
>>  0x0050:  406d 6e20 2832 3429 2041 7574 6865 6e74
>>  0x0060:  6963 6174 696f 6e20 6661 696c 7572 6520
>>  0x0070:  5b43 6972 6375 6974 2068 616e 646c 653a
>>  0x0080:  2031 2f34 3a35 3131 3a36 333a 3331 2f36
>>  0x0090:  2f32 2f34 3736 3631 5d0a
>>  0x:  4500 00b6 77da  4011 ef82 0a32 ff6f  E...w...@2.o
>>  0x0010:  0a32 ff06 dea1 0202 00a2 28d8 3c31 3839  .2(.<189
>>  0x0020:  3e46 6562 2038 2031 393a 3132 3a31 373a  >Feb.8.19:12:17:
>>  0x0030:  2025 5359 534c 4f47 2d35 2d4e 4f54 4943  .%SYSLOG-5-NOTIC
>>  0x0040:  453a 2061 6161 643a 2053 7562 5365 7373  E:.aaad:.SubSess
>>  0x0050:  696f 6e41 5554 4846 4149 4c20 7573 6572  ionAUTHFAIL.user
>>  0x0060:  3a20 7070 706f 6531 3633 3434 406d 6e20  :.pppoe16344@mn.
>>  0x0070:  2832 3429 2041 7574 6865 6e74 6963 6174  (24).Authenticat
>>  0x0080:  696f 6e20 6661 696c 7572 6520 5b43 6972  ion.failure.[Cir
>>  0x0090:  6375 6974 2068 616e 646c 653a 2031 2f34  cuit.handle:.1/4
>>  0x00a0:  3a35 3131 3a36 333a 3331 2f36 2f32 2f34  :511:63:31/6/2/4
>>  0x00b0:  3736 3631 5d0a   7661].
>>
>> As you can see, the packet is much longer, but it doesn't exceed the 
>> maximum UDP packet size that can be processed by Graylog (8192). My guess 
>> is that logs coming from 10.50.255.111 are not RFC compatible and thus 
>> they're discarded by Graylog. How can I debug it / fix it? I didn't find 
>> any related messages in the Elasticsearch log (there were no errors related 
>> to parsing a message).
>> I deleted the default Input object and added a new RAW UDP Input object. 
>> It didn't fix the issue - logs from 10.50.255.111 are still not parsed.
>>
>
Thanks for your answer Jochen. It's not a problem with routing nor with 
firewall. I can capture these packets with TCPDump (as shown in my post) on 
the very same machine, on which the Graylog server is running. I don't have 
a Syslog UDP input, I deleted it and created a Raw UDP Input instead. 
Here's the configuration of this input:
- Type: Raw/Plaintext UDP
- Title: Main
- Global: unchecked
- Node: 01fd7feb/Silenoz (that's the only node I have)
- Bind address: 10.50.255.6
- Port: 10514
- Buffer size: 262144

As you can see I'm using the 10514 port, not 514. I can't use 514, because 
the Graylog server would have to run with root privileges. As a workaround 
I created an iptables rule which routes all packets from port 514 to 10514:
-A PREROUTING -i eth0 -p udp -m udp --dport 514 -j REDIRECT --to-ports 10514
I'm 100% sure that it works, because all other hosts *from the same subnet* 
(10.50.255.0/24) are sending their logs to 10.50.255.6:10514 and they are 
properly parsed by the Graylog server.
Here's a funny thing though: the following frame got parsed by Graylog (it 
came from the problematic router):

uluru.miconet.pl.58254 > Silenoz.sys

[graylog2] Re: collect logs from remote machine

[Wallace Turner]

Ho Jochen so you need to install Sidecar (and then possibly nxlog) on each 
machine you want to watch a logfile?

On Thursday, 9 February 2017 16:43:28 UTC+8, Jochen Schalanda wrote:
>
> Hi Wallace,
>
> On Thursday, 9 February 2017 06:03:07 UTC+1, Wallace Turner wrote:
>>
>> What i am trying to do is for graylog to retrieve (or monitor) a log file 
>> at a network location (windows servers) and bring the contents of the plain 
>> text log file to graylog.
>>
>> Is this possible (on windows)?
>>
>
> Yes, that's possible. You could simply use the Graylog Collector Sidecar 
> for this: http://docs.graylog.org/en/2.1/pages/collector_sidecar.html
>
> Cheers,
> Jochen
>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/26c2f3f6-ceb9-494f-b8a0-8eec712ed6a3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] How to parse OpenVPN logs in Graylog?

[CESAR Fabre]

Hi guys,

I have the PfSense 2.3.2 with OpenVPN enabled. I want to parse OpenVPN logs 
in Graylog with Dashboards.

I have no idea. Can you help me?


Thanks a lot!


César

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/b23b0bf5-559f-40ca-9e6c-62148fece604%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: Notification condition [NO_MASTER] has been fixed.

[Jochen Schalanda]

Hi Peter,

please upgrade to the latest stable version of Graylog (which is Graylog 
2.1.3 at the time of writing, and soon Graylog 2.2.0) and check if it 
solves your problems.

Cheers,
Jochen

On Wednesday, 8 February 2017 13:48:53 UTC+1, Peter Dudas wrote:
>
> Hi Jochen!
>
> Did you found any relevant ifo in the attached logs?
>
> Thank you!
>
>
> On Wednesday, 1 February 2017 16:23:23 UTC+1, Peter Dudas wrote:
>>
>> Hi,
>>
>> what do you mean on logs? The whole /var/log/graylog folder?
>>
>> These are the /var/log/graylog/server/current file - that's it, nothing 
>> else in these files.
>> Please find the whole /var/log/graylog folders attached.
>>
>> Clustered nodes were  imported from OVA files, running on VMWARE ESX 
>> 6.0U2 in a Vsphere cluster from locally attached datastores.
>> 4 VCPU, 16GB ram each. (no CPU shares - running on 3 different hosts in a 
>> HA cluster).
>>
>> Graylog (standalone server) runs on a production cluster - which is not 
>> the above mentioned Vsphere cluster.
>> This one running 8 months ago - no lost messages from clients, just these 
>> annoying messages in the System/Overview.
>> It is not a 2.1.3 issue - I've seen them before.
>>
>> All of the servers has synced NTP, also open-vm-tools installed.
>>
>> Peter Dudas
>>
>> On Wednesday, 1 February 2017 16:05:11 UTC+1, Jochen Schalanda wrote:
>>>
>>> Hi Peter,
>>>
>>> are these really the complete logs? I'm missing startup messages of 
>>> Graylog…
>>>
>>> This being said, the current_server.txt file shows some Java GC 
>>> messages with really long pauses. This causes the Graylog node to drop out 
>>> of the list of "active" nodes.
>>>
>>> What are the hardware specs of these machines running Graylog and how 
>>> did you configure the nodes (also JVM settings)?
>>>
>>> Cheers,
>>> Jochen
>>>
>>> On Wednesday, 1 February 2017 15:56:41 UTC+1, Peter Dudas wrote:

 Hi Jochen!

 please find server logs attached from all the 3 nodes 
 (Server/Node1/Node2).
 Also please find the log from the standalone server 
 (curent_graylog.txt).

 Thank you1

 Peter Dudas




-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/a33c8f0f-ba05-4a49-82ac-a039ba55f417%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: Overwriting Timestamp field using Pipeline rules

[Jochen Schalanda]

Hi Al,

you might want to try to use your pattern with lower case 'y' for the year 
component of the date pattern.

Cheers,
Jochen

On Wednesday, 8 February 2017 21:09:19 UTC+1, Al Reynolds wrote:
>
> I've noticed another error. The timestamp field is being replaced 
> correctly, but the "gl2_processing_error" field is showing the following 
> error (on all messages):
> For rule 'WO-CS-RAS': In call to function 'parse_date' at 8:15 an 
> exception was thrown: Invalid format: "2017-02-08 15:05:59,170" is 
> malformed at "17-02-08 15:05:59,170"
>
>
> It doesn't seem to have any adverse effects, but I'm curious as to what 
> might be causing it?
>
> On Wednesday, February 8, 2017 at 1:56:17 PM UTC-5, Al Reynolds wrote:
>>
>> Figured it out--parse_date needed the timestamp . New rule looks like 
>> this:
>> rule "WO-CS-RAS" 
>> when 
>> 
>> contains(to_string($message.file),"centralserver\\ras-server\\log\\ras_cs_")
>> then
>> set_field("WO_Log_Source","RAS-CS");
>> let matches = grok(pattern: "%{WO_CS_RAS_CS_MESSAGE}", value: 
>> to_string($message.message));
>> set_fields(matches);
>> let date = parse_date(to_string($message.WO_Timestamp), "-MM-dd 
>> HH:mm:ss,SSS", "EST");
>> set_field("timestamp", date);
>> route_to_stream("WideOrbit Logs");
>> end
>>
>> I was under the impression that the timezone was optional? 
>>
>> Thanks for all your help with this Jochen--it's greatly appreciated!
>>
>> Cheers,
>> Al
>>
>> On Wednesday, February 8, 2017 at 11:05:22 AM UTC-5, Al Reynolds wrote:
>>>
>>> That's what I get for typing it out...thank you for catching that! 
>>> Unfortunately, even after correcting for the incorrect milliseconds value, 
>>> it's still not replacing timestamp value. I sent the parsed date to a new 
>>> field (in this case, "log_timestamp") to verify that the output data was in 
>>> the correct format, which it is now, but it still won't replace the 
>>> timestamp field.
>>>
>>> Message sample with "log_timestamp" field:
>>> WO_CS_RAS_CS_MESSAGE
>>> 2017-02-08 11:00:34,980 WARN  [Task 'ATLANTA-FS' FS timer.1] 
>>> FriendshipTasksServiceImpl = Could not obtain task info for:  
>>> 2c95ac8e-57e3-91b2-0158-495b880b24e8REQUEST FAILED ==> STATUS CODE: 404, 
>>> RESPONSE BODY:
>>> WO_LogLevel
>>> WARN
>>> WO_Log_Source
>>> RAS-CS
>>> WO_Message
>>> Could not obtain task info for:  2c95ac8e-57e3-91b2-0158-
>>> 495b880b24e8REQUEST FAILED ==> STATUS CODE: 404, RESPONSE BODY:
>>> WO_Process
>>> Task 'ATLANTA-FS' FS timer.1
>>> WO_SubProcess
>>> FriendshipTasksServiceImpl
>>> WO_Timestamp
>>> 2017-02-08 11:00:34,980
>>> facility
>>> filebeat
>>> file
>>> d:\centralserver\ras-server\log\ras_cs_WO-ATL-CS.log
>>> input_type
>>> log
>>> log_timestamp
>>> 2017-02-08T11:00:34.980Z
>>> message
>>> 2017-02-08 11:00:34,980 WARN  [Task 'ATLANTA-FS' FS timer.1] 
>>> FriendshipTasksServiceImpl = Could not obtain task info for:  
>>> 2c95ac8e-57e3-91b2-0158-495b880b24e8REQUEST FAILED ==> STATUS CODE: 404, 
>>> RESPONSE BODY:
>>> name
>>> WO-ATL-CS
>>> offset
>>> 2372156
>>> source
>>> WO-ATL-CS
>>> timestamp
>>> 2017-02-08T16:00:35.864Z
>>> type
>>> log
>>>
>>> Corrected rule: 
>>> rule "WO-CS-RAS" 
>>> when 
>>> 
>>> contains(to_string($message.file),"centralserver\\ras-server\\log\\ras_cs_")
>>> then
>>> set_field("WO_Log_Source","RAS-CS");
>>> let matches = grok(pattern: "%{WO_CS_RAS_CS_MESSAGE}", value: 
>>> to_string($message.message));
>>> set_fields(matches);
>>> let date = parse_date(to_string($message.WO_Timestamp), "-MM-dd 
>>> HH:mm:ss,SSS");
>>> set_field("timestamp", date);
>>> route_to_stream("WideOrbit Logs");
>>> end
>>>
>>> Thanks!
>>>
>>> Cheers,
>>> Al
>>>
>>> On Wednesday, February 8, 2017 at 10:55:03 AM UTC-5, Jochen Schalanda 
>>> wrote:

 Hi Al,

 On Wednesday, 8 February 2017 15:46:07 UTC+1, Al Reynolds wrote:
>
> WO_Timestamp
> 2017-02-08 09:42:30,056
>
> Those messages are with the date parsing disabled. I'm attempting to 
> replace "timestamp" with the "WO_Timestamp" field. 
>

 The string in WO_Timestamp doesn't match the pattern "-MM-dd 
 HH:mm:ss,sss" used in parse_date(). See 
 http://www.joda.org/joda-time/apidocs/org/joda/time/format/DateTimeFormat.html
  
 for details.

 Hint: 's' and 'S' are not the same thing.
  

 Side note: The full_message field is empty on my filebeat inputs--is 
> that expected behavior? 
>

 Yes, that's expected.

 What would you expect to find in the (optional) full_message field?

 Cheers,
 Jochen

>>>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/561bf48b-bfdc-4ee6-b0a0-f7e706dc1e5c%40googlegroups.com.

[graylog2] Re: Graylog is ignoring some UDP packets sent by a particular host

[Jochen Schalanda]

Hi,

Graylog itself doesn't care where the packets are coming from.

Is the routing to Graylog working for the "ignored" host?
Is the networking set up correctly on all hosts?
Are there any firewall rules in place?
How did you configure the Syslog UDP and the Raw/Plaintext UDP inputs?

Cheers,
Jochen

On Wednesday, 8 February 2017 19:43:38 UTC+1, tomaszik...@gmail.com wrote:
>
> Hello,
>
> I've recently set up a working Graylog server. It's collecting logs from 
> many network switches and routers. One particular router (ironically, the 
> most important one) doesn't appear in the Sources list though. Graylog 
> keeps ignoring all packets coming from that host. Here's an example of a 
> packet which is *not* ignored by Graylog:
>
> 19:12:15.705167 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto 
> UDP (17), length 115)
> 10.50.255.44.40810 > Silenoz.syslog: [udp sum ok] [|syslog]
>  0x:  4500 0073  4000 4011 27e3 0a32 ff2c  E..s..@.@.'..2.,
>  0x0010:  0a32 ff06 9f6a 0202 005f 01d1 6468 6370  .2...j..._..dhcp
>  0x0020:  2c77 6172 6e69 6e67 2067 706f 6e2d 6d6e  ,warning.gpon-mn
>  0x0030:  6720 6f66 6665 7269 6e67 206c 6561 7365  g.offering.lease
>  0x0040:  2031 302e 3530 2e32 3338 2e33 3520 666f  .10.50.238.35.fo
>  0x0050:  7220 3030 3a30 323a 3731 3a35 413a 3036  r.00:02:71:5A:06
>  0x0060:  3a42 3820 7769 7468 6f75 7420 7375 6363  :B8.without.succ
>  0x0070:  6573 73 
>
> And below you can see a packet which *is* ignored by Graylog:
>
> 10.50.255.111.56993 > Silenoz.syslog: [udp sum ok] SYSLOG, length: 154
>  Facility local7 (23), Severity notice (5)
>  Msg: Feb 8 19:12:17: %SYSLOG-5-NOTICE: aaad: SubSessionAUTHFAIL user: 
> pppoe16344@mn (24) Authentication failure [Circuit handle: 1/4:511:63:31/6
> /2/47661]\0x0a
>  0x:  3c31 3839 3e46 6562 2038 2031 393a 3132
>  0x0010:  3a31 373a 2025 5359 534c 4f47 2d35 2d4e
>  0x0020:  4f54 4943 453a 2061 6161 643a 2053 7562
>  0x0030:  5365 7373 696f 6e41 5554 4846 4149 4c20
>  0x0040:  7573 6572 3a20 7070 706f 6531 3633 3434
>  0x0050:  406d 6e20 2832 3429 2041 7574 6865 6e74
>  0x0060:  6963 6174 696f 6e20 6661 696c 7572 6520
>  0x0070:  5b43 6972 6375 6974 2068 616e 646c 653a
>  0x0080:  2031 2f34 3a35 3131 3a36 333a 3331 2f36
>  0x0090:  2f32 2f34 3736 3631 5d0a
>  0x:  4500 00b6 77da  4011 ef82 0a32 ff6f  E...w...@2.o
>  0x0010:  0a32 ff06 dea1 0202 00a2 28d8 3c31 3839  .2(.<189
>  0x0020:  3e46 6562 2038 2031 393a 3132 3a31 373a  >Feb.8.19:12:17:
>  0x0030:  2025 5359 534c 4f47 2d35 2d4e 4f54 4943  .%SYSLOG-5-NOTIC
>  0x0040:  453a 2061 6161 643a 2053 7562 5365 7373  E:.aaad:.SubSess
>  0x0050:  696f 6e41 5554 4846 4149 4c20 7573 6572  ionAUTHFAIL.user
>  0x0060:  3a20 7070 706f 6531 3633 3434 406d 6e20  :.pppoe16344@mn.
>  0x0070:  2832 3429 2041 7574 6865 6e74 6963 6174  (24).Authenticat
>  0x0080:  696f 6e20 6661 696c 7572 6520 5b43 6972  ion.failure.[Cir
>  0x0090:  6375 6974 2068 616e 646c 653a 2031 2f34  cuit.handle:.1/4
>  0x00a0:  3a35 3131 3a36 333a 3331 2f36 2f32 2f34  :511:63:31/6/2/4
>  0x00b0:  3736 3631 5d0a   7661].
>
> As you can see, the packet is much longer, but it doesn't exceed the 
> maximum UDP packet size that can be processed by Graylog (8192). My guess 
> is that logs coming from 10.50.255.111 are not RFC compatible and thus 
> they're discarded by Graylog. How can I debug it / fix it? I didn't find 
> any related messages in the Elasticsearch log (there were no errors related 
> to parsing a message).
> I deleted the default Input object and added a new RAW UDP Input object. 
> It didn't fix the issue - logs from 10.50.255.111 are still not parsed.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/2329a7b6-d34d-4764-8204-147edcc86e5d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: Incoming Gelf UDP messages not showing up

[Jochen Schalanda]

Hi,

On Thursday, 9 February 2017 06:54:30 UTC+1, IJFK wrote:
>
> I'm sending Syslog packets in Gelf format (I successfully validated the 
> Json), and no matter what I do, the packets don't show up. There is no 
> parsing error or anything, the data just doesn't show up.
>

How exactly are you sending messages? How did you configure the clients? 
How did you configure the inputs (and which types) in Graylog? 


I already created a Raw/UDP input & stream, which does show the messages 
> coming in, I also verified with tcpdump that they are actually making it to 
> the server.
>

This sounds like they are simply not valid GELF messages.

Cheers,
Jochen

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/f765796b-d5b6-48ec-b1cc-b5af51b2249c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: Forward from One graylog to another

[Jochen Schalanda]

Hi Tom,

On Thursday, 9 February 2017 04:46:31 UTC+1, Tom Powers wrote:
>
> Is there any good doc on setting up the tls on the stream output and then 
> the receiving side at the new graylog instance?


Please refer to the documentation 
at http://docs.graylog.org/en/2.1/pages/streams.html#outputs

It's simply setting up the stream GELF output (Streams -> Manage Stream 
Output -> Launch new output) and a GELF input on the other instance of 
Graylog (System -> Inputs).

Cheers,
Jochen

>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/2ce7f6a5-a6b6-4bcc-a13b-bdcfa03f631a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: collect logs from remote machine

[Jochen Schalanda]

Hi Wallace,

On Thursday, 9 February 2017 06:03:07 UTC+1, Wallace Turner wrote:
>
> What i am trying to do is for graylog to retrieve (or monitor) a log file 
> at a network location (windows servers) and bring the contents of the plain 
> text log file to graylog.
>
> Is this possible (on windows)?
>

Yes, that's possible. You could simply use the Graylog Collector Sidecar 
for this: http://docs.graylog.org/en/2.1/pages/collector_sidecar.html

Cheers,
Jochen

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/f5575f15-fae6-494e-b992-a180cbe7001b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.