[graylog2] Re: Compress collected data or move to a new HDD?

[Phil Sumner]

Can't talk about compression, but moving data to a new disk is referenced 
in the manual:

http://docs.graylog.org/en/2.1/pages/configuration/graylog_ctl.html#extend-ova-disk



On Wednesday, 21 September 2016 03:43:24 UTC+1, 8bits...@gmail.com wrote:
>
> I have Elasticsearch data and it's logs written to a 2nd HDD than where 
> the OS is.  This HDD, 100GB, is constantly getting maxed out with ES's logs 
> which I manually delete, but I see the indices are slowly creeping up in 
> size too.  Is there a compression option that I am missing?  Or how would I 
> move data to a 3rd HDD, bigger in size of course, without losing anything 
> collected this far?  Would it be as simple as stopping Graylog, copying the 
> folders over, define new path, and restart Graylog?
>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/324c767d-2361-4434-ae8b-91409361efd8%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [graylog2] Web interface flashes logon page in every reload

[Phil Sumner]

Thank you!

On Wednesday, 7 September 2016 16:27:24 UTC+1, Edmundo Alvarez wrote:
>
> Hi, 
>
> This is a known presentation issue, please check this Github issue for 
> more information: https://github.com/Graylog2/graylog2-server/issues/2770 
>
> Regards, 
> Edmundo 
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/6079331e-8fca-4017-8e2e-b1f844902391%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Web interface flashes logon page in every reload

[Phil Sumner]

Since upgrading to 2.1.0 from 2.0.3, the web interface has started showing 
(briefly) the logon page whenever the reload action happens.

Not sure what information I can provide to be useful here.  Anyone got any 
idea how to stop it?

Thanks,
Phil

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/6461a08e-f1a8-45e1-84ba-db85c3d4c816%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: Parsing Linux audit log messages

[Phil Sumner]

You could use the key=value copy extractor 
(http://docs.graylog.org/en/2.0/pages/extractors.html#automatically-extract-all-key-value-pairs)

That gets all the data into fields.  Then after that, it depends what you 
want to achieve.

On Monday, 22 August 2016 13:10:42 UTC+1, Aleksey Chudov wrote:
>
> Hi,
>
> I've searched Google and Graylog Marketplace for a plugin to parse Linux 
> audit log messages with no success.
>
> Some details about audit logs
>
> https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Security_Guide/sec-Understanding_Audit_Log_Files.html
>
> Actually, audit event consists of three records, which share the same time 
> stamp and serial number. Each record consists of several name=value pairs 
> separated by a white space or a comma. 
>
> What is the best way to parse audit log messages? I'm thinking of writing 
> custom Graylog plugin.
>
> Regards,
> Aleksey
>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/73087615-744a-448c-9fe1-4e97c33e255d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: Extractor not running on inputs that should match

[Phil Sumner]

I ended up deleting the input and recreating it, importing the extractors, 
and everything works as expected on the re-created input.

No idea what was going on...

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/567c66f5-43a5-45cc-b6f0-a9d3fa977000%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [graylog2] Extractor not running on inputs that should match

[Phil Sumner]

I've changed the grok pattern to include the end of the message and it 
doesn't appear to have made any difference.
  %{HOSTNAME:source_unit} diskmonitor\:%{GREEDYDATA:UNWANTED}partition %{
WORD:partition} has only %{POSINT:percent_free}\% free

I've since discovered that there are other extractors on the same input 
which aren't extracting:

message: ip-10-244-56-13 tmm6[11383]: Rule /Common/iRules-WebServices-
Sandbox-Production-WhiteList : 166.84.7.123 is not 
permitted to WebServices Sandbox
grok: %{HOSTNAME:source_unit} tmm%{GREEDYDATA:UNWANTED}: Rule %{UNIXPATH:
irule} : %{IP:source_address} is not permitted to %{
GREEDYDATA:service}

Using the "Try" button on the extractor edit page, it all works as 
expected, but new incoming messages do not show any of the additional 
fields.

I've restarted the service using graylog-ctl, deleted the extractors and 
recreated them, but no change.  Any ideas what else could be going on?

Thanks,
Phil

On Wednesday, 3 August 2016 09:55:10 UTC+1, Jan Doberstein wrote:
>
> Hi Phil,
>
>
> the Grok pattern need to match the hole line and in your case it does not.
>
> An example Grok pattern:
> %{HOSTNAME:source_unit} diskmonitor\:%{GREEDYDATA:UNWANTED}partition %{
> WORD:partition} has only %{POSINT:percent_free}
>
> And an example input message:
> ip-10-244-63-14 diskmonitor: 011d0004:3: Disk partition var has only 12% 
> free
>
>
> regards
> Jan
>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/ba51d376-e0c4-40c6-aeb1-da1f480a44a3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Extractor not running on inputs that should match

[Phil Sumner]

I've set up some simple Grok extractors and tested that they match against 
a sample of input messages in the Graylog interface, but when further 
messages come in the extractors do not seem to "kick in", and the 
additional fields that I see on other inputs with similar extractors don't 
get added on.  This was working at some point, but I deleted and recreated 
the extractors for some reason I've now forgotten.

An example Grok pattern:
%{HOSTNAME:source_unit} diskmonitor\:%{GREEDYDATA:UNWANTED}partition %{WORD:
partition} has only %{POSINT:percent_free}

And an example input message:
ip-10-244-63-14 diskmonitor: 011d0004:3: Disk partition var has only 12% 
free

Below is an example of a message that came in after I updated the extractor:



I can't figure out what's going on here, am I missing something obvious?

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/ac14ada5-997c-4214-be14-c6dcc98996e4%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [graylog2] graylog-sidecar on Windows Server 2008R2

[Phil Sumner]

gah! So long I've been looking and I didn't spot that.

Thank you.



On Friday, 29 July 2016 13:29:50 UTC+1, Marius Sturm wrote:
>
> In your collector_sidecar.yml the nxlog backend is disabled by 'enabled: 
> false' guess this shout be flipped.
>
> On 29 July 2016 at 13:49, Phil Sumner <sumne...@gmail.com > 
> wrote:
>
>> C:\Windows\system32>"C:\Program Files 
>> (x86)\graylog\collector-sidecar\graylog-collector-sidecar.exe" -version
>> Graylog Collector Sidecar version 0.0.9 (amd64)
>>
>>
>> On Friday, 29 July 2016 12:47:16 UTC+1, Marius Sturm wrote:
>>>
>>> Which version of the Sidecar is this? There should be a little bit more 
>>> in the output actually...
>>>
>>> On 29 July 2016 at 13:37, Phil Sumner <sumne...@gmail.com> wrote:
>>>
>>>> Thanks, this is all I get... I left it like this earlier for about 30m 
>>>> and nothing changed.
>>>>
>>>> C:\Windows\system32>"C:\Program Files 
>>>> (x86)\graylog\collector-sidecar\graylog-collector-sidecar.exe"
>>>> time="2016-07-29T12:35:57+01:00" level=info msg="Using collector-id: 
>>>> c2b603af-ed6a-4246-873d-86b820194e78"
>>>> time="2016-07-29T12:35:57+01:00" level=info msg="Fetching 
>>>> configurations taggedby: [windows]"
>>>> time="2016-07-29T12:35:57+01:00" level=info msg="Starting collector 
>>>> supervisor"
>>>>
>>>>
>>>>
>>>> On Friday, 29 July 2016 12:29:06 UTC+1, Marius Sturm wrote:
>>>>>
>>>>> Hi Phil,
>>>>> could you try to start the Sidecar in foreground mode. Just open a 
>>>>> shell and go to the installation directory. Start the Sidecar with 
>>>>> graylog-collector-sidecar.
>>>>> Post the output you see here, maybe we can see some problems.
>>>>>
>>>>> Cheers,
>>>>> Marius
>>>>>
>>>>> -- 
>>>> You received this message because you are subscribed to the Google 
>>>> Groups "Graylog Users" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send 
>>>> an email to graylog2+u...@googlegroups.com.
>>>> To view this discussion on the web visit 
>>>> https://groups.google.com/d/msgid/graylog2/b06b80b9-eda5-4134-8ce7-f457018fefe9%40googlegroups.com
>>>>  
>>>> <https://groups.google.com/d/msgid/graylog2/b06b80b9-eda5-4134-8ce7-f457018fefe9%40googlegroups.com?utm_medium=email_source=footer>
>>>> .
>>>>
>>>> For more options, visit https://groups.google.com/d/optout.
>>>>
>>>
>>>
>>>
>>> -- 
>>> Developer
>>>
>>> Tel.: +49 (0)40 609 452 077
>>> Fax.: +49 (0)40 609 452 078
>>>
>>> TORCH GmbH - A Graylog Company
>>> Poolstraße 21
>>> 20335 Hamburg
>>> Germany
>>>
>>> https://www.graylog.com <https://www.torch.sh/>
>>>
>>> Commercial Reg. (Registergericht): Amtsgericht Hamburg, HRB 125175
>>> Geschäftsführer: Lennart Koopmann (CEO)
>>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Graylog Users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to graylog2+u...@googlegroups.com .
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/graylog2/1f74890f-7f4b-47ea-86d2-3182c61e779b%40googlegroups.com
>>  
>> <https://groups.google.com/d/msgid/graylog2/1f74890f-7f4b-47ea-86d2-3182c61e779b%40googlegroups.com?utm_medium=email_source=footer>
>> .
>>
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
>
> -- 
> Developer
>
> Tel.: +49 (0)40 609 452 077
> Fax.: +49 (0)40 609 452 078
>
> TORCH GmbH - A Graylog Company
> Poolstraße 21
> 20335 Hamburg
> Germany
>
> https://www.graylog.com <https://www.torch.sh/>
>
> Commercial Reg. (Registergericht): Amtsgericht Hamburg, HRB 125175
> Geschäftsführer: Lennart Koopmann (CEO)
>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/93cc2451-ddb8-4733-aedf-a846e3b56d3c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [graylog2] graylog-sidecar on Windows Server 2008R2

[Phil Sumner]

C:\Windows\system32>"C:\Program Files 
(x86)\graylog\collector-sidecar\graylog-collector-sidecar.exe" -version
Graylog Collector Sidecar version 0.0.9 (amd64)


On Friday, 29 July 2016 12:47:16 UTC+1, Marius Sturm wrote:
>
> Which version of the Sidecar is this? There should be a little bit more in 
> the output actually...
>
> On 29 July 2016 at 13:37, Phil Sumner <sumne...@gmail.com > 
> wrote:
>
>> Thanks, this is all I get... I left it like this earlier for about 30m 
>> and nothing changed.
>>
>> C:\Windows\system32>"C:\Program Files 
>> (x86)\graylog\collector-sidecar\graylog-collector-sidecar.exe"
>> time="2016-07-29T12:35:57+01:00" level=info msg="Using collector-id: 
>> c2b603af-ed6a-4246-873d-86b820194e78"
>> time="2016-07-29T12:35:57+01:00" level=info msg="Fetching configurations 
>> taggedby: [windows]"
>> time="2016-07-29T12:35:57+01:00" level=info msg="Starting collector 
>> supervisor"
>>
>>
>>
>> On Friday, 29 July 2016 12:29:06 UTC+1, Marius Sturm wrote:
>>>
>>> Hi Phil,
>>> could you try to start the Sidecar in foreground mode. Just open a shell 
>>> and go to the installation directory. Start the Sidecar with 
>>> graylog-collector-sidecar.
>>> Post the output you see here, maybe we can see some problems.
>>>
>>> Cheers,
>>> Marius
>>>
>>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Graylog Users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to graylog2+u...@googlegroups.com .
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/graylog2/b06b80b9-eda5-4134-8ce7-f457018fefe9%40googlegroups.com
>>  
>> <https://groups.google.com/d/msgid/graylog2/b06b80b9-eda5-4134-8ce7-f457018fefe9%40googlegroups.com?utm_medium=email_source=footer>
>> .
>>
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
>
> -- 
> Developer
>
> Tel.: +49 (0)40 609 452 077
> Fax.: +49 (0)40 609 452 078
>
> TORCH GmbH - A Graylog Company
> Poolstraße 21
> 20335 Hamburg
> Germany
>
> https://www.graylog.com <https://www.torch.sh/>
>
> Commercial Reg. (Registergericht): Amtsgericht Hamburg, HRB 125175
> Geschäftsführer: Lennart Koopmann (CEO)
>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/1f74890f-7f4b-47ea-86d2-3182c61e779b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [graylog2] graylog-sidecar on Windows Server 2008R2

[Phil Sumner]

Thanks, this is all I get... I left it like this earlier for about 30m and 
nothing changed.

C:\Windows\system32>"C:\Program Files 
(x86)\graylog\collector-sidecar\graylog-collector-sidecar.exe"
time="2016-07-29T12:35:57+01:00" level=info msg="Using collector-id: 
c2b603af-ed6a-4246-873d-86b820194e78"
time="2016-07-29T12:35:57+01:00" level=info msg="Fetching configurations 
taggedby: [windows]"
time="2016-07-29T12:35:57+01:00" level=info msg="Starting collector 
supervisor"



On Friday, 29 July 2016 12:29:06 UTC+1, Marius Sturm wrote:
>
> Hi Phil,
> could you try to start the Sidecar in foreground mode. Just open a shell 
> and go to the installation directory. Start the Sidecar with 
> graylog-collector-sidecar.
> Post the output you see here, maybe we can see some problems.
>
> Cheers,
> Marius
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/b06b80b9-eda5-4134-8ce7-f457018fefe9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] graylog-sidecar on Windows Server 2008R2

[Phil Sumner]

Hi,

I'm having trouble getting sidecar/NXLog working with Windows.

I've installed graylog-sidecar & nxlog on Windows Server 2008R2 and 
installed/started it as a service.

I've installed NXLog and removed the service.

My collector_sidecar.yml looks like this (paths to nxlog all checked to be 
correct):
server_url: http://10.x.x.x:12900
update_interval: 10
tls_skip_verify: false
node_id: shaadc02
collector_id: file:C:\Program Files (x86)\graylog\collector-sidecar\
collector-id
log_path: C:\Program Files (x86)\graylog\collector-sidecar\logs\
log_rotation_time: 86400
log_max_age: 604800
tags:
- windows
backends:
- name: nxlog
  enabled: false
  binary_path: C:\Program Files (x86)\nxlog\nxlog.exe
  configuration_path: C:\Program Files (x86)\graylog\collector-sidecar\
generated\nxlog.conf

I've set up a collector configuration:





I can see the new collector checking in:




The snippet in the tag config is the default:
{{if .Linux}}
User nxlog
Group nxlog
Moduledir /usr/lib/nxlog/modules
CacheDir /var/spool/collector-sidecar/nxlog
PidFile /var/run/graylog/collector-sidecar/nxlog.pid
define LOGFILE /var/log/graylog/collector-sidecar/nxlog.log
LogFile %LOGFILE%
LogLevel INFO


Module  xm_fileop

When@daily
Execfile_cycle('%LOGFILE%', 7);
 

{{end}}

{{if .Windows}}
Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log
LogLevel INFO


Module  xm_fileop

When@daily
Execfile_cycle('%ROOT%\data\nxlog.log', 7);
 

{{end}}


nxlog.conf is this:
define ROOT C:\Program Files (x86)\nxlog

Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log


Module xm_gelf



But I never see anything in the ...\generated\ folder, and nxlog never 
starts.


What am I missing?

Thanks,
Phil

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/2f94f695-e838-4e61-b4fa-8ea038df85f4%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.