[graylog2] exporting data from searches not working properly

[graylogtesting]

Hello 

I'm using the "production" OVA (not the beta) of Graylog

I noticed that when I try to export the results of a search, the message 
field is trunked, see example below:

The full message is full_message





*{"1331892651000, 4776, "Success", "Security", 
"Microsoft-Windows-Security-Auditing", "The computer attempted to validate 
the credentials for an account.Authentication Package: 
MICROSOFT_AUTHENTICATION_PACKAGE_V1_0Logon Account: mr636cSource 
Workstation: INHYIMR636CError Code: 0x0" "}* 

In the exported CSV log I have only this:

*{"1331892651000, 4634, "Success", "Security", "Microsoft-Windows*

Is there anyway to fix this?

Thanks a lot
Mark

-- 
You received this message because you are subscribed to the Google Groups 
"graylog2" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [graylog2] Re: Multiple source IP addresses to one Stream group - HOW? POSSIBLE? A BETTER WAY?

[Brandon]

As far as I know the source is not mandatory. You can create a proper 
regex to pull in messages meeting the criteria from one of many sources. 
Maybe setting up extractors and then using the exists clause from a 
stream would give you want you want.
Using an extractor you can set a specific field as "true" or whatever 
you want then use the stream to pull in logs having only that field set.


On 05/29/2015 04:06 PM, Henrik Johansen wrote:

Hi Aidan,

I am curious - why do you need a stream per source / keyword combination?

Could you outline what you want to achieve with that solution - 
perhaps you're just approaching the problem the wrong way?


The only reason I can think of for doing what you have outlined is 
permissions (ie strict delegation of access based on source / keyword 
combinations) ... ?




---
HenrikJ

On 29. maj 2015 kl. 21.55.11 CEST, Aidan Venn  wrote:

Hi Jochemb,

They could be a thousand sources but I only want to Create and EDIT 
one set of related streams that are applied to the sources when 
edited. A one to many approach. ONE set of streams MANY source ip 
addresses.


Stream set:

stream 1-keyword:disconnect
steram 2-keyword:loss
stream 3-keyword:fail
stream 4-keyword:error
steram 5-keyword:connect
stream 6-keyword:deauthenticate
stream 7-keyword:reconnect
steram 8-keyword:failure
stream 9-keyword:crash

These would then be applied to 1000+ sources. If I then need to make 
a change I only have to do it once.


Thanks for taking an interest.

Kind Regards

Aidan Venn

On Friday, May 29, 2015 at 1:27:01 PM UTC+1, Jochemb wrote:

Make three streams:

stream 1-keyword:disconnect
steram 2-keyword:loss
stream 3-keyword:fail

Without a source?

Op donderdag 28 mei 2015 10:40:20 UTC+2 schreef Aidan Venn:




Hi,

Garylog Newbie

Please see picture attached.

I have three streams matching a single source IP and warning
keywords from logs:

source IP: 192.168.0.1

stream 1-keyword:disconnect
steram 2-keyword:loss
stream 3-keyword:fail

I want to "group" these streams and apply to multiple (1000
+) source IP addresses to benefit future scalability and
large scale administration. Basically for each source IP they
will be three or more streams but I only have to
configure/edit the group once.

I don`t want to have 1000 devices then have to copy each
stream and then change the source IP address match. 10
keyword stream x 1000 devices would then equal 1 streams
in total to configure and edit. This would be very time
consuming. Especially if I had to make a change.

One change to the group would apply to all. A one to many
relationship. How can I do this?

Perhaps my approach/idaea is incorrect so any recommendations
would be great.

Kind Regards

Aidan Venn

--
You received this message because you are subscribed to the Google 
Groups "graylog2" group.
To unsubscribe from this group and stop receiving emails from it, 
send an email to graylog2+unsubscr...@googlegroups.com 
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google 
Groups "graylog2" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to graylog2+unsubscr...@googlegroups.com 
.

For more options, visit https://groups.google.com/d/optout.


--
You received this message because you are subscribed to the Google Groups 
"graylog2" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] how to keep the log message in one field?

[graylogtesting]

Hello

I'm having a problem with graylog and nxlog feed 

I have a huge archive of windows event logs, I have been trying to import 
these logs into graylog using nxlog and gelf

It all works well, nxlog pickup the logs and imports them but the messages 
are being split in several records rather tha a single one, 


Example if the event log contains the follow


*{"1331892664000, 4624, "Success", "Security", 
"Microsoft-Windows-Security-Auditing", "An account was successfully logged 
on.*

*Subject:*
* Security ID: S-1-0-0*
* Account Name: -*
* Account Domain: -*
* Logon ID: 0x0*

*Logon Type: 3*


*This event is generated when a logon session is created. It is generated 
on the computer that was accessed.*

*Key length indicates the length of the generated session key. This will be 
0 if no session key was requested." "}  *


It gets loaded into graylog as:

Record 1: *{"1331892664000, 4624, "Success", "Security", 
"Microsoft-Windows-Security-Auditing", "An account was successfully logged 
on.*
Record 2: *Subject*
Record 3*: **Security ID: S-1-0-0*

etc.
etc


I just would like to have all the message stored in one record

Do you have any idea how this could be achieved?

Thanks!
Mark





-- 
You received this message because you are subscribed to the Google Groups 
"graylog2" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [graylog2] Re: grok extractors not working

[Arie]

Hi,

Are you using the latest version of NXLog? There was a problem in an older 
version
concerning Graylog/GELF.

Arie.

Op vrijdag 29 mei 2015 20:41:52 UTC+2 schreef Jesse Skrivseth:
>
> I'm not sure why, but suddenly the extractors are working today without 
> any further action on my part. There seems to be a very long delay between 
> when an extractor is configured and when it is in effect, at least in this 
> environment. 
>
> Another thing to note is that the data on this input is TLS encrypted GELF 
> via TCP, and the data is coming in from NXLog using GELF_TCP.
>
> On Thursday, May 28, 2015 at 3:25:05 PM UTC-6, Kay Röpke wrote:
>>
>> I'm not an expert on the OVAs so I would recommend simply setting up a 
>> test instance to check this. Or you can wait until I get to it in the (my) 
>> morning ;)
>>
>>>
>>> 

-- 
You received this message because you are subscribed to the Google Groups 
"graylog2" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [graylog2] Re: grok extractors not working

[Bernd Ahlers]

Jesse,

thank you for the update. I created an issue in GitHub for this with a
link to this mailing list thread.

https://github.com/Graylog2/graylog2-server/issues/1192

I also started to test with the detailed data you submitted but did not
see any problems. I was testing on 1.1.0-rc.1 though.

Next step is to test all of this with 1.0.2 (which you are running).

I will let you know once I have any updates.

Thank you!

Bernd

Jesse Skrivseth [Fri, May 29, 2015 at 11:41:52AM -0700] wrote:
>I'm not sure why, but suddenly the extractors are working today without any 
>further action on my part. There seems to be a very long delay between when 
>an extractor is configured and when it is in effect, at least in this 
>environment. 
>
>Another thing to note is that the data on this input is TLS encrypted GELF 
>via TCP, and the data is coming in from NXLog using GELF_TCP.
>
>On Thursday, May 28, 2015 at 3:25:05 PM UTC-6, Kay Röpke wrote:
>>
>> I'm not an expert on the OVAs so I would recommend simply setting up a 
>> test instance to check this. Or you can wait until I get to it in the (my) 
>> morning ;)
>>
>>>
>>> 
>
>-- 
>You received this message because you are subscribed to the Google Groups 
>"graylog2" group.
>To unsubscribe from this group and stop receiving emails from it, send an 
>email to graylog2+unsubscr...@googlegroups.com.
>For more options, visit https://groups.google.com/d/optout.


-- 
Developer

Tel.: +49 (0)40 609 452 077
Fax.: +49 (0)40 609 452 078

TORCH GmbH - A Graylog company
Steckelhörn 11
20457 Hamburg
Germany

Commercial Reg. (Registergericht): Amtsgericht Hamburg, HRB 125175
Geschäftsführer: Lennart Koopmann (CEO)

-- 
You received this message because you are subscribed to the Google Groups 
"graylog2" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [graylog2] Re: collector questions

[Bernd Ahlers]

Arie,

great to hear! Thank you for your patience, testing and the details
reports! :)

Regards,
Bernd

Arie [Fri, May 29, 2015 at 01:25:52PM -0700] wrote:
>Bernd,
>
>Tested on installing, removing and managing the service from the script and 
>console,
>everything without problems on server 2003. We are running/testing it on 3 
>servers
>succesfully collecting Application and system logs form windows.
>
>Thank you. Good weekend to you all.
>
>Arie.
>
>Op vrijdag 29 mei 2015 20:00:06 UTC+2 schreef Bernd Ahlers:
>>
>> Arie, 
>>
>> wow, thank you. Can you verify that this one works now? 
>>
>>
>> https://gist.githubusercontent.com/bernd/57fa40c557ffbb303801/raw/01ec841120ef202a6eab159f741755f06c93fa34/graylog-collector-service.bat
>>  
>>
>> Thank you very much! 
>>
>> Bernd 
>>
>> Arie [Fri, May 29, 2015 at 05:25:49AM -0700] wrote: 
>> >Bernd,, 
>> > 
>> >Other than the path, I have exactly the same output. But I have nailed 
>> the 
>> >basterd :-) 
>> > 
>> >It is in this line: 
>> > 
>> >SET 
>> >PROCRUN="%COLLECTOR_BIN_DIR%windows\graylog-collector-service-%ARCH%.exe" 
>> > 
>> >should be 
>> > 
>> >SET 
>> >"PROCRUN=%COLLECTOR_BIN_DIR%windows\graylog-collector-service-%ARCH%.exe" 
>> > 
>> >Up and running here 
>> > 
>> > 
>> >Arie 
>> > 
>> >=== 
>> >C:\collector\bin>my-collector-service.bat install GC 
>> >Installing service for Graylog Collector 
>> > 
>> >Service name: "GC" 
>> >JAVA_HOME:"C:\Program Files\Java\jre7\" 
>> >ARCH: "x86" 
>> > 
>> >WARNING: JAVA_HOME points to a JRE and not JDK installation; a client 
>> (not 
>> >a server) JVM will be used... 
>> > 
>> >""C:\collector\bin\\windows\graylog-collector-service-x86.exe"" //IS//GC 
>> >--Classpath ""C:\collector\graylog-collector.jar"" --Jvm 
>> >"C:\Program Files\Java\jre7\\bin\client\jvm.dll" --JvmMs 12m --JvmMx 64m 
>> >--JvmOptions -Djava.library.path=C:\collector\lib\sigar#- 
>> >Dfile.encoding=UTF-8#-Xms12m#-Xmx64m --StartPath "C:\collector" --Startup 
>> >auto --StartMode jvm --StartClass org.graylog.collector. 
>> >cli.Main --StartMethod main --StartParams 
>> >"run;-f;C:\collector\config\collector.conf" --StopMode jvm --StopClass 
>> >org.graylog.colle 
>> >ctor.cli.Main --StopMethod stop --StopTimeout 0 --PidFile ""GC.pid"" 
>> >--DisplayName "Graylog Collector (GC)" --Description "Graylog 
>> > Collector 0.2.1 service. See http://www.graylog.org/ for details." 
>> >--LogPath ""C:\collector\logs"" --LogPrefix "graylog-collector 
>> >" --StdError auto --StdOutput auto 
>> >ERROR: Failed to install service: GC 
>> > 
>> >=== 
>> > 
>> >Annoying. Trying to find a system requirement to install this on 2003 TS 
>> > 
>> >Arie 
>> > 
>> > 
>> > 
>> >On Friday, May 29, 2015 at 1:10:56 PM UTC+2, Bernd Ahlers wrote: 
>> >> 
>> >> Arie, 
>> >> 
>> >> the following command works for me on Windows 7. 
>> >> 
>> >> ## 
>> >> 
>> C:\Users\IEUser\Desktop\graylog-collector-0.2.1-20150529103656>bin\graylog-collector-service.bat
>>  
>>
>> >> install GC 
>> >> Installing service for Graylog Collector 
>> >> 
>> >> Service name: "GC" 
>> >> JAVA_HOME:"C:\Program Files\Java\jre7\" 
>> >> ARCH: "x86" 
>> >> 
>> >> WARNING: JAVA_HOME points to a JRE and not JDK installation; a client 
>> (not 
>> >> a server) JVM will be used... 
>> >> 
>> >> 
>> ""C:\Users\IEUser\Desktop\graylog-collector-0.2.1-20150529103656\bin\\windows\graylog-collector-service-x86.exe""
>>  
>>
>> >> //IS//GC --Classpath 
>> >> 
>> ""C:\Users\IEUser\Desktop\graylog-collector-0.2.1-20150529103656\graylog-collector.jar""
>>  
>>
>> >> --Jvm "C:\Program Files\Java\jre7\\bin\client\jvm.dll" --JvmMs 12m 
>> --JvmMx 
>> >> 64m --JvmOptions 
>> >> 
>> -Djava.library.path=C:\Users\IEUser\Desktop\graylog-collector-0.2.1-20150529103656\lib\sigar#-Dfile.encoding=UTF-8#-Xms12m#-Xmx64m
>>  
>>
>> >> --StartPath 
>> >> "C:\Users\IEUser\Desktop\graylog-collector-0.2.1-20150529103656" 
>> --Startup 
>> >> auto --StartMode jvm --StartClass org.graylog.collector.cli.Main 
>> >> --StartMethod main --StartParams 
>> >> 
>> "run;-f;C:\Users\IEUser\Desktop\graylog-collector-0.2.1-20150529103656\config\collector.conf"
>>  
>>
>> >> --StopMode jvm --StopClass org.graylog.collector.cli.Main --StopMethod 
>> stop 
>> >> --StopTimeout 0 --PidFile ""GC.pid"" --DisplayName "Graylog Collector 
>> (GC)" 
>> >> --Description "Graylog Collector 0.2.1 service. See 
>> >> http://www.graylog.org/ for details." --LogPath 
>> >> ""C:\Users\IEUser\Desktop\graylog-collector-0.2.1-20150529103656\logs"" 
>> >> --LogPrefix "graylog-collector" --StdError auto --StdOutput auto 
>> >> 
>> >> Service 'GC' has been installed 
>> >> ## 
>> >> 
>> >> The script replaces the whitespace with "#" characters already. 
>> >> 
>> >> Bernd 
>> >> 
>> >> Arie [Fri, May 29, 2015 at 03:20:21AM -0700] wrote: 
>> >> >Bernd 
>> >> > 
>> >> >Working on it, no promises. All new to me he