[graylog2] Where are my GELF messages going?

[Cody]

Hi,

I've been trying to setup a graylog2 server with clients sending in windows 
logs via GELF tcp, the issue i'm hitting is that the input on the server 
seems to be processing messages, see screenshot below where it says 1 
minute average rate: 9/msgs But when I click on show received messages I 
get no messages received. Anyone have any thoughts on what could be causing 
this?







-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/4e569efd-81af-43a0-b340-fbb8e2383372%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: Graylog IO Exception Error

[Ariel Godinez]

After further investigation I think this was due to elasticsearch and 
graylog being overloaded. I have increased their heap sizes accordingly and 
will see how the system performs.

Ariel

On Wednesday, July 6, 2016 at 12:21:11 PM UTC-5, Ariel Godinez wrote:
>
> Hello,
>
> I've been using graylog for a couple weeks now and started to notice some 
> unusual behavior today. I am currently running a single node setup.
>
> The Issue:
>
> Every once in awhile I start to notice that that graylog is dragging quite 
> a bit (the loading spinner is persisting much longer than usual) so I go 
> check the logs and find the following error message. 
>
> ERROR [ServerRuntime$Responder] An I/O error has occurred while writing a 
> response message entity to the container output stream.
> org.glassfish.jersey.server.internal.process.MappableException: 
> java.io.IOException: Connection closed
> at 
> org.glassfish.jersey.server.internal.MappableExceptionWrapperInterceptor.aroundWriteTo(MappableExceptionWrapperInterceptor.java:92)
>  
> ~[graylog.jar:?]
> at 
> org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:162)
>  
> ~[graylog.jar:?]
> at 
> org.glassfish.jersey.message.internal.MessageBodyFactory.writeTo(MessageBodyFactory.java:1130)
>  
> ~[graylog.jar:?]
> at 
> org.glassfish.jersey.server.ServerRuntime$Responder.writeResponse(ServerRuntime.java:711)
>  
> [graylog.jar:?]
> at 
> org.glassfish.jersey.server.ServerRuntime$Responder.processResponse(ServerRuntime.java:444)
>  
> [graylog.jar:?]
> at 
> org.glassfish.jersey.server.ServerRuntime$Responder.process(ServerRuntime.java:434)
>  
> [graylog.jar:?]
> at 
> org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:329) 
> [graylog.jar:?]
> at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271) 
> [graylog.jar:?]
> at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267) 
> [graylog.jar:?]
> at org.glassfish.jersey.internal.Errors.process(Errors.java:315) 
> [graylog.jar:?]
> at org.glassfish.jersey.internal.Errors.process(Errors.java:297) 
> [graylog.jar:?]
> at org.glassfish.jersey.internal.Errors.process(Errors.java:267) 
> [graylog.jar:?]
> at 
> org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317)
>  
> [graylog.jar:?]
> at 
> org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305) 
> [graylog.jar:?]
> at 
> org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154)
>  
> [graylog.jar:?]
> at 
> org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer.service(GrizzlyHttpContainer.java:384)
>  
> [graylog.jar:?]
> at 
> org.glassfish.grizzly.http.server.HttpHandler$1.run(HttpHandler.java:224) 
> [graylog.jar:?]
> at 
> com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176)
>  
> [graylog.jar:?]
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  
> [?:1.8.0_91]
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  
> [?:1.8.0_91]
> at java.lang.Thread.run(Thread.java:745) [?:1.8.0_91]
> Caused by: java.io.IOException: Connection closed
> at 
> org.glassfish.grizzly.asyncqueue.TaskQueue.onClose(TaskQueue.java:317) 
> ~[graylog.jar:?]
> at 
> org.glassfish.grizzly.nio.AbstractNIOAsyncQueueWriter.onClose(AbstractNIOAsyncQueueWriter.java:501)
>  
> ~[graylog.jar:?]
> at 
> org.glassfish.grizzly.nio.transport.TCPNIOTransport.closeConnection(TCPNIOTransport.java:412)
>  
> ~[graylog.jar:?]
> at 
> org.glassfish.grizzly.nio.NIOConnection.doClose(NIOConnection.java:604) 
> ~[graylog.jar:?]
> at 
> org.glassfish.grizzly.nio.NIOConnection$5.run(NIOConnection.java:570) 
> ~[graylog.jar:?]
> at 
> org.glassfish.grizzly.nio.DefaultSelectorHandler.execute(DefaultSelectorHandler.java:235)
>  
> ~[graylog.jar:?]
> at 
> org.glassfish.grizzly.nio.NIOConnection.terminate0(NIOConnection.java:564) 
> ~[graylog.jar:?]
> at 
> org.glassfish.grizzly.nio.transport.TCPNIOConnection.terminate0(TCPNIOConnection.java:291)
>  
> ~[graylog.jar:?]
> at 
> org.glassfish.grizzly.nio.transport.TCPNIOAsyncQueueWriter.writeCompositeRecord(TCPNIOAsyncQueueWriter.java:197)
>  
> ~[graylog.jar:?]
> at 
> org.glassfish.grizzly.nio.transport.TCPNIOAsyncQueueWriter.write0(TCPNIOAsyncQueueWriter.java:92)
>  
> ~[graylog.jar:?]
> at 
> org.glassfish.grizzly.nio.AbstractNIOAsyncQueueWriter.processAsync(AbstractNIOAsyncQueueWriter.java:344)
>  
> ~[graylog.jar:?]
> at 
> org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:107)
>  
> ~[graylog.jar:?]
> at 
> 

[graylog2] Re: Has any one successfully set up SSL on Graylog 2.0?

[Dave C.]

Jochen, 

Thank you, again, for all the help looking into this problem for me. 

Here is the output of the head -n1 command: 

==> /etc/graylog/graylog-ssl/CERT.pem <==
-BEGIN CERTIFICATE-

==> /etc/graylog/graylog-ssl/KEY.pem <==
-BEGIN ENCRYPTED PRIVATE KEY-


I looked over the log file and these errors are not the same as what I was 
receiving before adding the quotes. The previous error had text stating 
Graylog couldn't access the files, I may have fixed that with file 
permission and mistakenly assumed it was the quotes that fixed that error. 
Either way for the sake of thoroughness here are the errors when I removed 
the quotes around the password in the server.conf for both web and the rest 
api and the file permissions. 

2016-07-08T10:46:00.781-05:00 ERROR [ServiceManager] Service 
WebInterfaceService [FAILED] has failed in the STARTING state.
java.io.IOException: ObjectIdentifier() -- data isn't an object ID (tag = 
48)
at 
sun.security.util.ObjectIdentifier.(ObjectIdentifier.java:253) 
~[?:1.8.0_92]
at sun.security.util.DerInputStream.getOID(DerInputStream.java:281) 
~[?:1.8.0_92]
at 
com.sun.crypto.provider.PBES2Parameters.engineInit(PBES2Parameters.java:267) 
~[sunjce_provider.jar:1.8.0_71]
at 
java.security.AlgorithmParameters.init(AlgorithmParameters.java:293) 
~[?:1.8.0_92]
at sun.security.x509.AlgorithmId.decodeParams(AlgorithmId.java:132) 
~[?:1.8.0_92]
at sun.security.x509.AlgorithmId.(AlgorithmId.java:114) 
~[?:1.8.0_92]
at sun.security.x509.AlgorithmId.parse(AlgorithmId.java:372) 
~[?:1.8.0_92]
at 
javax.crypto.EncryptedPrivateKeyInfo.(EncryptedPrivateKeyInfo.java:95) 
~[?:1.8.0_71]
at 
org.graylog2.shared.security.tls.PemKeyStore.generateKeySpec(PemKeyStore.java:69)
 
~[graylog.jar:?]
at 
org.graylog2.shared.security.tls.PemKeyStore.buildKeyStore(PemKeyStore.java:96) 
~[graylog.jar:?]
at 
org.graylog2.shared.initializers.AbstractJerseyService.buildSslEngineConfigurator(AbstractJerseyService.java:187)
 
~[graylog.jar:?]
at 
org.graylog2.shared.initializers.AbstractJerseyService.setUp(AbstractJerseyService.java:158)
 
~[graylog.jar:?]
at 
org.graylog2.initializers.WebInterfaceService.startUp(WebInterfaceService.java:46)
 
~[graylog.jar:?]
at 
com.google.common.util.concurrent.AbstractIdleService$DelegateService$1.run(AbstractIdleService.java:60)
 
[graylog.jar:?]
at 
com.google.common.util.concurrent.Callables$3.run(Callables.java:100) 
[graylog.jar:?]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_92]
2016-07-08T10:46:00.817-05:00 ERROR [InputSetupService] Not starting any 
inputs because lifecycle is: Uninitialized [LB:DEAD]


2016-07-08T10:46:01.165-05:00 ERROR [ServiceManager] Service RestApiService 
[FAILED] has failed in the STOPPING state.
java.io.IOException: ObjectIdentifier() -- data isn't an object ID (tag = 
48)
at 
sun.security.util.ObjectIdentifier.(ObjectIdentifier.java:253) 
~[?:1.8.0_92]
at sun.security.util.DerInputStream.getOID(DerInputStream.java:281) 
~[?:1.8.0_92]
at 
com.sun.crypto.provider.PBES2Parameters.engineInit(PBES2Parameters.java:267) 
~[sunjce_provider.jar:1.8.0_71]
at 
java.security.AlgorithmParameters.init(AlgorithmParameters.java:293) 
~[?:1.8.0_92]
at sun.security.x509.AlgorithmId.decodeParams(AlgorithmId.java:132) 
~[?:1.8.0_92]
at sun.security.x509.AlgorithmId.(AlgorithmId.java:114) 
~[?:1.8.0_92]
at sun.security.x509.AlgorithmId.parse(AlgorithmId.java:372) 
~[?:1.8.0_92]
at 
javax.crypto.EncryptedPrivateKeyInfo.(EncryptedPrivateKeyInfo.java:95) 
~[?:1.8.0_71]
at 
org.graylog2.shared.security.tls.PemKeyStore.generateKeySpec(PemKeyStore.java:69)
 
~[graylog.jar:?]
at 
org.graylog2.shared.security.tls.PemKeyStore.buildKeyStore(PemKeyStore.java:96) 
~[graylog.jar:?]
at 
org.graylog2.shared.initializers.AbstractJerseyService.buildSslEngineConfigurator(AbstractJerseyService.java:187)
 
~[graylog.jar:?]
at 
org.graylog2.shared.initializers.AbstractJerseyService.setUp(AbstractJerseyService.java:158)
 
~[graylog.jar:?]
at 
org.graylog2.shared.initializers.RestApiService.startUp(RestApiService.java:65) 
~[graylog.jar:?]
at 
com.google.common.util.concurrent.AbstractIdleService$DelegateService$1.run(AbstractIdleService.java:60)
 
[graylog.jar:?]
at 
com.google.common.util.concurrent.Callables$3.run(Callables.java:100) 
[graylog.jar:?]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_92]


2016-07-08T10:46:03.784-05:00 ERROR [ServiceManager] Service 
IndexerSetupService [FAILED] has failed in the STOPPING state.
java.lang.IllegalStateException: Can't move to started state when closed
at 
org.elasticsearch.common.component.Lifecycle.canMoveToStarted(Lifecycle.java:114)
 
~[graylog.jar:?]
at 

[graylog2] Graylog slow processing.

[Hema Kumar]

Hi,
   I am using graylog 1.3.3 with ES 1.7.5, from yesterday we are seeing the 
process buffer filled up on the master node and the outgoing process is too 
slow than normal, I have tried restarting GL and ES but did not fix the 
issue, below are the log warn and errors we see that repeats continuously. 

We have 4 graylog server and 7 elasticsearch nodes, Only the Master graylog 
is processing slow and sometimes the 3rd node, rest of the nodes are 
working fine. 

Could you please help me on this, i have been breaking my head since 
yesterday. 


2016-07-08T01:53:21.355-06:00 WARN  [GelfChunkAggregator] Error while 
expiring GELF chunk entries
java.util.NoSuchElementException
at 
java.util.concurrent.ConcurrentSkipListMap.firstKey(ConcurrentSkipListMap.java:2036)
at 
java.util.concurrent.ConcurrentSkipListSet.first(ConcurrentSkipListSet.java:396)
at 
org.graylog2.inputs.codecs.GelfChunkAggregator$ChunkEvictionTask.run(GelfChunkAggregator.java:288)
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

2016-07-08T05:37:47.803-06:00 ERROR [AnyExceptionClassMapper] Unhandled 
exception in REST resource
org.elasticsearch.action.search.ReduceSearchPhaseException: Failed to 
execute phase [fetch], [reduce]
at 
org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction$AsyncAction$2.onFailure(TransportSearchQueryThenFetchAction.java:159)
at 
org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:41)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.ClassCastException: 
org.elasticsearch.search.aggregations.bucket.terms.LongTerms$Bucket cannot 
be cast to 
org.elasticsearch.search.aggregations.bucket.terms.StringTerms$Bucket
at 
org.elasticsearch.search.aggregations.bucket.terms.StringTerms$Bucket.compareTerm(StringTerms.java:85)
at 
org.elasticsearch.search.aggregations.bucket.terms.InternalOrder$4.compare(InternalOrder.java:87)
at 
org.elasticsearch.search.aggregations.bucket.terms.InternalOrder$4.compare(InternalOrder.java:83)
at 
org.elasticsearch.search.aggregations.bucket.terms.InternalOrder$CompoundOrder$CompoundOrderComparator.compare(InternalOrder.java:284)
at 
org.elasticsearch.search.aggregations.bucket.terms.InternalOrder$CompoundOrder$CompoundOrderComparator.compare(InternalOrder.java:270)
at 
org.elasticsearch.search.aggregations.bucket.terms.support.BucketPriorityQueue.lessThan(BucketPriorityQueue.java:37)
at 
org.elasticsearch.search.aggregations.bucket.terms.support.BucketPriorityQueue.lessThan(BucketPriorityQueue.java:26)
at 
org.apache.lucene.util.PriorityQueue.upHeap(PriorityQueue.java:225)
at org.apache.lucene.util.PriorityQueue.add(PriorityQueue.java:133)
at 
org.apache.lucene.util.PriorityQueue.insertWithOverflow(PriorityQueue.java:149)
at 
org.elasticsearch.search.aggregations.bucket.terms.InternalTerms.reduce(InternalTerms.java:195)
at 
org.elasticsearch.search.aggregations.InternalAggregations.reduce(InternalAggregations.java:140)
  at 
org.elasticsearch.search.aggregations.bucket.InternalSingleBucketAggregation.reduce(InternalSingleBucketAggregation.java:79)
at 
org.elasticsearch.search.aggregations.InternalAggregations.reduce(InternalAggregations.java:140)
at 
org.elasticsearch.search.controller.SearchPhaseController.merge(SearchPhaseController.java:407)
at 
org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction$AsyncAction$2.doRun(TransportSearchQueryThenFetchAction.java:147)
at 
org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:36)

2016-07-08T05:43:30.757-06:00 WARN  [DefaultExceptionMonitor] Unexpected 
exception.
java.nio.channels.ClosedSelectorException
at sun.nio.ch.SelectorImpl.keys(SelectorImpl.java:68)
at 
org.apache.mina.transport.socket.nio.NioProcessor.isSelectorEmpty(NioProcessor.java:107)
at 
org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.run(AbstractPollingIoProcessor.java:1139)
at 
org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:64)
at 

[graylog2] Graylog Training Courses

[Bruce Givens]

Hi there!

Is anyone aware of any Graylog training courses in Germany?

I've done a bit of searching, but the offerings do not seem to be 
overwhelming.


Ideally, I'd be looking for a course on administration of Graylog, 
obviously including Elasticsearch and MongoDB, as well as sizing and HA 
architecture.


Thanks!
Bruce

--
You received this message because you are subscribed to the Google Groups "Graylog 
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/577F8D69.9020501%40cipsoft.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: Graylog Collector Sidecar - no logs being shipped

[Kev Johnson]

After going through some further testing I've filed this 
at https://github.com/Graylog2/collector-sidecar/issues/37 - it looks to be 
an issue with NXlog CE handling the input and output modules as GUIDs.

On Thursday, 7 July 2016 17:11:41 UTC+1, Kev Johnson wrote:
>
> Firstly: I love the idea of being able to push out updated configuration 
> files to my collectors. That said: I'm having issues getting logs to my 
> Graylog box (deployed from the OVA)
>
> Steps taken so far are as follows
>
>
>- Installed NXlogCE
>- Uninstalled the NXlog service
>- Installed the Graylog Collector Sidecar
>- Edited the sidecar_collector.yml file to point to my Graylog server, 
>and remove the reference to IIS
>- Installed the Graylog Collector Sidecar service
>- Started the Graylog Collector Sidecar service
>- Created a configuration (Windows Logs, ship to the UDP GELF Input 
>defined on my Graylog box)
>- Created a tag called Windows and applied it to this configuration
>
>
> I see the nxlog.conf get created on the Windows server, I see nxlog.exe 
> start up on server, but nothing is sent. TCPDump on the Graylog server 
> shows only the TCP connections in on port 12900 from the Windows server.
>
> Any advice on troubleshooting this would be much appreciated!
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/e230b2cf-e477-41e2-928a-74b42a76463a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: Graylog is not processing Messages from one input anymore

[Jochen Schalanda]

Hi Keamas,

please refer 
to 
https://www.elastic.co/guide/en/elasticsearch/reference/2.3/setup-configuration.html
 
and https://www.elastic.co/guide/en/elasticsearch/guide/2.x/heap-sizing.html 
for information about sizing Elasticsearch and changing its memory 
configuration.

Elasticsearch should at least have 4 GB of memory (and of course way more 
if you ingest more messages).


Cheers,
Jochen

On Friday, 8 July 2016 12:36:04 UTC+2, Keamas M wrote:
>
> Hey, here are the details of the system:
> I installes de dpkg files like in here on the Ubuntu 16.04 LTS 
> http://docs.graylog.org/en/2.0/pages/installation/os/ubuntu.html
>
> Everything is running on one single VM Graylog and Elasticsearch.
>
>
> VMware 
>
> 1 Virtual Socket
>
> 2 Cores
>
> Memory: 8GB RAM
> HDD 800 GB 
>
> root@ATLOG001:/home/ladmin# uname -a
> Linux ATLOG001 4.4.0-28-generic #47-Ubuntu SMP Fri Jun 24 10:09:13 UTC 
> 2016 x86_64 x86_64 x86_64 GNU/Linux
>
>
> root@ATLOG001:/home/ladmin# lsb_release -a
> No LSB modules are available.
> Distributor ID: Ubuntu
> Description:Ubuntu 16.04 LTS
> Release:16.04
> Codename:   xenial
>
> SystemHostname:ATLOG001Node ID:e2b97d26-f84a-4a82-99ba-3bedfbb5b207
> Version:2.0.3 (f07c170), codename *Rothaus*JVM:PID 1014, Oracle 
> Corporation 1.8.0_91 on Linux 4.4.0-28-generic
>
>
> I can easily add more system ressources please let me know hoch much.
>
> How can I give Elastisearch more memory ? How much should I give?
>
>
> The Input ist UDP Syslog. I am receiving the Syslog messages of a Firewall 
> which are quite a lot of messages.
>
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/145ac890-e092-434e-8c7e-fee2c3cb43e6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: Out of memory - Java heap space in Graylog-Server

[Rumen Tashev]

I have a similar problem on my Graylog2 configuration. I have a cluster 
with two nodes. The problem is with my slave node, where we capture NetFlow 
data from our routers. The incoming messages are about 30 - 50 per second. 
I have allowed up to 4g of heap memory for the graylog-server. With a fresh 
start, the node uses up to 972.8 MB and this starts to grow over time. It 
takes approximately 24 hours until the node reaches the full 4g (shown as 
3.8 GB) and then constantly stops and re-starts. A re-start on the node 
(graylog-ctl stop && shutdown -r now) rectifies the problem, but then again 
just temporary. The graylog slave node is configured as "backend".

We have the 100% same configuration on the master node, where this problem 
does not appear. The master node runs for weeks now, processing about 10 - 
30 messages per second and uses 1.1GB of heap space. It never reaches any 
close to 3.8 GB, which would be the maximum configured. The only difference 
is, that it does not accept any NetFlow messages.

Previously we had the NetFlow messages go to the master node. Then the 
exact same behaviour would appear there as well - the node gradually 
consumes more and more memory, until it reaches a state where it constantly 
crashes and restarts. Moving the NetFlow messages to the slave seems to 
have rectified the problem on the master. Both nodes run the latest version 
of Graylog2 - 2.0.3.

Do you also run NetFlow inputs on your node? Any help is greatly 
appreciated!

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/3e213832-a4c2-4671-b08f-5a9b863b274f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: Graylog is not processing Messages from one input anymore

[Keamas M]

Hey, here are the details of the system:
I installes de dpkg files like in here on the Ubuntu 16.04 LTS 
http://docs.graylog.org/en/2.0/pages/installation/os/ubuntu.html

Everything is running on one single VM Graylog and Elasticsearch.


VMware 

1 Virtual Socket

2 Cores

Memory: 8GB RAM
HDD 800 GB 

root@ATLOG001:/home/ladmin# uname -a
Linux ATLOG001 4.4.0-28-generic #47-Ubuntu SMP Fri Jun 24 10:09:13 UTC 2016 
x86_64 x86_64 x86_64 GNU/Linux


root@ATLOG001:/home/ladmin# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:Ubuntu 16.04 LTS
Release:16.04
Codename:   xenial

SystemHostname:ATLOG001Node 
ID:e2b97d26-f84a-4a82-99ba-3bedfbb5b207Version:2.0.3 
(f07c170), codename *Rothaus*JVM:PID 1014, Oracle Corporation 1.8.0_91 on 
Linux 4.4.0-28-generic


I can easily add more system ressources please let me know hoch much.

How can I give Elastisearch more memory ? How much should I give?


The Input ist UDP Syslog. I am receiving the Syslog messages of a Firewall 
which are quite a lot of messages.



-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/d0506e27-6fc0-417c-8513-388a6679ad58%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: Graylog is not processing Messages from one input anymore

[Keamas M]

Hey, here are the details of the system:
I installes de dpkg files like in here on the Ubuntu 16.04 LTS 
http://docs.graylog.org/en/2.0/pages/installation/os/ubuntu.html


VMware 

1 Virtual Socket

2 Cores

Memory: 8GB RAM
HDD 800 GB 

root@ATLOG001:/home/ladmin# uname -a
Linux ATLOG001 4.4.0-28-generic #47-Ubuntu SMP Fri Jun 24 10:09:13 UTC 2016 
x86_64 x86_64 x86_64 GNU/Linux


root@ATLOG001:/home/ladmin# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:Ubuntu 16.04 LTS
Release:16.04
Codename:   xenial

SystemHostname:ATLOG001Node 
ID:e2b97d26-f84a-4a82-99ba-3bedfbb5b207Version:2.0.3 
(f07c170), codename *Rothaus*JVM:PID 1014, Oracle Corporation 1.8.0_91 on 
Linux 4.4.0-28-generic


I can easily add more system ressources please let me know hoch much.

How can I give Elastisearch more memory ? How much should I give?


The Input ist UDP Syslog. I am receiving the Syslog messages of a Firewall 
which are quite a lot of messages.









-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/37f20213-80f4-4751-93be-a3b497054c33%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [graylog2] debugging pipelines is... difficult

[Edmundo Alvarez]

Hi Jason,

It's hard to tell what is wrong from here, since we can't exactly see how your 
messages look like. Could you share a couple of messages with us?

Please be aware that at the moment, the "regex" function needs to match the 
whole string: 
https://github.com/Graylog2/graylog-plugin-pipeline-processor/issues/35

We will release a Graylog 2.1.0 alpha really soon, and it would be really 
helpful to know what you think about the new pipeline simulator. It's still a 
work in progress, just as the pipeline processor, but hopefully will help you 
debug your pipelines and rules.

Regards,
Edmundo

> On 07 Jul 2016, at 23:31, Jason Haar  wrote:
> 
> 
> On Wed, Jul 6, 2016 at 9:50 PM, Jochen Schalanda  wrote:
> there's something coming up in Graylog 2.1.0 which will vastly simplify 
> testing pipeline rules.
> 
> That's great to hear. Any suggestions as to what's wrong with my rule?
> 
> Thanks
> 
> 
> -- 
> Cheers
> 
> Jason Haar
> Information Security Manager, Trimble Navigation Ltd.
> Phone: +1 408 481 8171
> PGP Fingerprint: 7A2E 0407 C9A6 CAF6 2B9F 8422 C063 5EBB FE1D 66D1
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Graylog Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to graylog2+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/graylog2/CAFChrgL7rcbe_rFpciwxs%3D5%3Dh%3D%3DXC7E3mdXWyO-skSP4ZjidCg%40mail.gmail.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/CE4236F5-A849-43F0-A8B2-BA9BD79A359C%40graylog.com.
For more options, visit https://groups.google.com/d/optout.

Re: [graylog2] Graylog Collector Sidecar - no logs being shipped

[Kev Johnson]

Wireshark on the test server shows no packets being sent other than the 
TCP12900 poll too, so we can be reasonably happy that there's nothing on 
the network eating them.

Config file has updated based on the snippet that I've added, but it's 
almost as if the nxlog process is running without a config file at all. If 
I edit the nxlog.conf file in the nxlog install directory to use the same 
criteria as the deployed nxlog.conf (other than the output name) and 
reinstall then start the nxlog service I see traffic as expected.


In conclusion then: it looks like while there is an nxlog.conf file 
deployed to the collector machine and the nxlog process is running this 
process doesn't seem to be leveraging the nxlog.conf file. Any ideas as to 
how I could further troubleshoot this? We know that the contents of the 
config are good, we know that there's no firewall interfering, we know that 
the collector sidecar service is running, and it's called the nxlog process.


On Friday, 8 July 2016 10:19:52 UTC+1, Kev Johnson wrote:
>
> Ok - so I've built a clean Windows Server 2012 R2, disabled the firewall 
> and run through the same process with the same result - the only traffic 
> back to the Graylog server is the tcp 12900 poll from the collector - I've 
> tried logging out/in and rebooting the server which all *should* generate 
> some log data. At this point I'm reasonably happy that it's not McAfee 
> causing the issue.
>
> Next port of call is going to be adding some snippets from NXlog.conf 
> files that I know work, let's see if that makes any difference!
>
> On Friday, 8 July 2016 07:03:27 UTC+1, Kev Johnson wrote:
>>
>> Thanks Marius - I'll give that a go today. Thanks for sense checking my 
>> config and confirming I've not done anything silly!
>>
>> On Thursday, 7 July 2016 22:30:29 UTC+1, Marius Sturm wrote:
>>>
>>> Yeah, sounds possible to me. All configurations look correct. So some 
>>> Windows firewall might be the root cause. Maybe you can try with a test 
>>> host with all firewalls disabled.
>>>
>>> On 7 July 2016 at 20:38, Kev Johnson  wrote:
>>>

 
 Does this help? Given that we're getting nothing but the Sidecar 
 checking traffic back from the servers I'm still leaning toward this being 
 an issue on the server rather than on the Graylog side. Any known issues 
 with McAfee VirusScan Enterprise (beyond the obvious!) - I can't remove 
 it, 
 but if I need to tweak it some I probably can. Not 100% certain that this 
 would be the case though, as if I just use NXlog to send syslog all works 
 fine.

 On Thursday, 7 July 2016 19:27:47 UTC+1, Marius Sturm wrote:
>
> The generated config looks fine, maybe a screenshot of the Graylog 
> input puts some light on this?
>
> On 7 July 2016 at 19:50, Kev Johnson  wrote:
>
>> Thanks Marius - I've double checked the input port (and that it's 
>> running!), but even if it were a mismatch I'd expect tcpdump to show the 
>> packets hitting the interface. I suspect that this has to be down to the 
>> generated config, so I'm pasting the contents of one of the servers' 
>> configs below - I'm afraid that I'm not really sure how I would 
>> troubleshoot that, so I'm happy to be told that I've done something 
>> stupid!
>>
>> define ROOT C:\Program Files (x86)\nxlog
>>>
>>>
 
>>>
>>>   Module xm_gelf
>>>
>>> 
>>>
>>>



 
>>>
>>> Module im_msvistalog
>>>
>>> PollInterval 10
>>>
>>> SavePos True
>>>
>>> ReadFromLast True
>>>
>>> 
>>>
>>>


 
>>>
>>> Module om_udp
>>>
>>> Host 192.168.21.12
>>>
>>> Port 5414
>>>
>>> OutputType  GELF
>>>
>>> Exec $short_message = $raw_event; # Avoids truncation of the 
 short_message field.
>>>
>>> Exec $gl2_source_collector = '28a3c8c7-bc02-44e0-98a5-e93e52b057e5';
>>>
>>> Exec $Hostname = hostname_fqdn();
>>>
>>> 
>>>
>>>


 
>>>
>>>   Path 577e5a4bc745f2099c054dd5 => 577e6c75c745f2099c0561b3
>>>
>>> 
>>>
>>>


>> On Thursday, 7 July 2016 18:41:36 UTC+1, Marius Sturm wrote:
>>>
>>> Hi,
>>> you could check if the Gelf port on the Graylog side is exactly the 
>>> same as on the Nxlog sender side, usually 12201. Go to System->Inputs 
>>> (the 
>>> input should have a green badge 'running') verify the port number with 
>>> the 
>>> one you configured for nxlog in the collector configuration.
>>> Another thing, Windows is not sending logs all the time so maybe you 
>>> just need to create an 

[graylog2] Re: Graylog search and sum fields

[Jochen Schalanda]

Hi Keamas,

aggregating or summing up different fields is currently not possible with 
Graylog.

Cheers,
Jochen

On Thursday, 7 July 2016 16:00:21 UTC+2, Keamas M wrote:
>
> Hey, 
> if I have multiple logs like this:
>
> type=FWD|proto=TCP|srcIF=port7.101|srcIP=10.244.130.102|srcPort=54610|srcMAC=00:00:00:00:00:00|dstIP=104.96.151.235|dstPort=80|dstService=|dstIF=port7.910|rule=|info=Normal
>  
> Operation|srcNAT=80.120.142.196|dstNAT=104.96.151.235|duration=0|count=1|receivedBytes=12|sentBytes=51|receivedPackets=125|sentPackets=12|user=n600724|protocol=HTTP
>  
> direct|application=Web browsing|target=www.microsoft.com
> |content=|urlcat=Computing/Technology
>
> I would like to know which User is creating the most traffic.
> For example I would like to see a Graph of: receivedBytes + sentByte for 
> HTTP and HTTPS Traffic for each user.
>
> Is this Possible with Graylog?
>
> In Splunk it lookes like this:
>
> index=main (dstPort=80 OR dstPort=443) | eval 
> totalbytes=receivedBytes+sentBytes | stats sum(totalbytes) as total by user 
> | sort -total | head 10 | top total by user showcount=false showperc=false
>
> In Graylog I tried to search:
>
> gl2_source_input:577e4cd717fd300404e5d7c8 AND (DST-PORT:80 OR DST-PORT:443)
>
> I added to Field Statistics RECEIVED-BYTES, SENT-BYTES  and USER
>
> Field Statistics
> Field TotalMeanMinimumMaximumStd. deviation
> VarianceSumCardinality
> RECEIVED-BYTES155,805NaNNaNNaNNaNNaNNaN
> 7,067
> SENT-BYTES155,739NaNNaNNaNNaNNaNNaN5,667
> USER49,031NaNNaNNaNNaNNaNNaN113
>
> But I am stucked here. Can anyone help me with this?
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/bba6b5aa-c3ea-4e96-bc45-818a7a17f76f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: Has any one successfully set up SSL on Graylog 2.0?

[Jochen Schalanda]

Hi Dave,

the quotes around the password shouldn't be necessary (and are, in fact, 
wrong). Could you please share the error message you've got when omitting 
these quotes?

Please also post the output of the following command (it doesn't contain 
any sensitive information, just the header of the private key and 
certificate file):

head -n1 /etc/graylog/graylog-ssl/CERT.pem /etc/graylog/graylog-ssl/KEY.pem



Cheers,
Jochen

On Thursday, 7 July 2016 20:11:03 UTC+2, Dave C. wrote:
>
> Jochen, 
>
> I ran the openssl command and it returned a single line with the text: RSA 
> key ok
>
> I did have some errors prior to the current ones with Graylog not being 
> able to access the key file. Those turned out to the an incorrect 
> formatting in the server.conf file, I had to put the password in quotes to 
> get passed that error. 
>
> These are the sections of the server.conf file you asked for with the 
> private info removed: 
>
> # Enable HTTPS support for the REST API. This secures the communication 
> with the REST API with
> # TLS to prevent request forgery and eavesdropping. This is disabled by 
> default. Uncomment the
> # next line to enable it.
> rest_enable_tls = true
>
> # The X.509 certificate chain file in PEM format to use for securing the 
> REST API.
> rest_tls_cert_file = /etc/graylog/graylog-ssl/CERT.pem
>
> # The PKCS#8 private key file in PEM format to use for securing the REST 
> API.
> rest_tls_key_file = /etc/graylog/graylog-ssl/KEY.pem
>
> # The password to unlock the private key used for securing the REST API.
> rest_tls_key_password ="PASSWORD"
>
>
> # Enable HTTPS support for the web interface. This secures the 
> communication of the web browser with the web interface
> # using TLS to prevent request forgery and eavesdropping.
> # This is disabled by default. Uncomment the next line to enable it and 
> see the other related configuration settings.
> web_enable_tls = true
>
> # The X.509 certificate chain file in PEM format to use for securing the 
> web interface.
> web_tls_cert_file = /etc/graylog/graylog-ssl/CERT.pem
>
> # The PKCS#8 private key file in PEM format to use for securing the web 
> interface.
> web_tls_key_file = /etc/graylog/graylog-ssl/KEY.pem
>
> # The password to unlock the private key used for securing the web 
> interface.
> web_tls_key_password ="PASSWORD"
>
> Thanks for the help. 
> --Dave C. 
>
> On Thursday, July 7, 2016 at 3:13:12 AM UTC-5, Jochen Schalanda wrote:
>>
>> Hi Dave,
>>
>> the error message looks like the private key is in an incompatible or 
>> invalid format which Graylog can't process.
>>
>> Could you please share your Graylog configuration (the rest_* and web_* 
>> settings should be sufficient) and the output of the following OpenSSL 
>> command:
>>
>> openssl rsa -noout -check -inform pem -in /path/to/private.key
>>
>>
>> Cheers,
>> Jochen
>>
>> On Wednesday, 6 July 2016 21:42:47 UTC+2, dave...@gmail.com wrote:
>>>
>>> All, 
>>>
>>> I have been working on setting up a test instance of Graylog 2.0 for 
>>> several weeks now and I can't seem to make any progress with implementing 
>>> SSL. I have seen a few other posts asking about converting java wallets to 
>>> the new set up of cert and key pair but that doesn't apply I have a new 
>>> cert from a CA. I am pretty sure I have the cert in the correct encoding 
>>> "X.509 certificate with PEM encoding" that the documentation 
>>> asks 
>>> for. I can use the command "openssl x509 -in cert.pem -text -noout" to 
>>> see the contents of the cert without issue. I can get Graylog 2.0 
>>> running with no SSL and with self generated certs but when I use the certs 
>>> from the CA I keep getting the errors below in 
>>> /var/log/graylog-server/server.log when I try to start Graylog 2.0, I can 
>>> send more of the log if needed. This is installed on Oracle Linux Server 
>>> release 6.7 with Graylog 2.0, Elasticsearch, and MongoDB installed from 
>>> their respective yum repos. Any advice would be greatly appreciated, I'm 
>>> just spinning my wheels at this point. 
>>>
>>>
>>> 2016-07-06T14:02:42.862-05:00 ERROR [ServiceManager] Service 
>>> WebInterfaceService [FAILED] has failed in the STARTING state.
>>> java.io.IOException: ObjectIdentifier() -- data isn't an object ID (tag 
>>> = 48)
>>> at 
>>> sun.security.util.ObjectIdentifier.(ObjectIdentifier.java:253) 
>>> ~[?:1.8.0_73]
>>> at 
>>> sun.security.util.DerInputStream.getOID(DerInputStream.java:281) 
>>> ~[?:1.8.0_73]
>>> at 
>>> com.sun.crypto.provider.PBES2Parameters.engineInit(PBES2Parameters.java:267)
>>>  
>>> ~[sunjce_provider.jar:1.8.0_71]
>>> at 
>>> java.security.AlgorithmParameters.init(AlgorithmParameters.java:293) 
>>> ~[?:1.8.0_73]
>>> at 
>>> sun.security.x509.AlgorithmId.decodeParams(AlgorithmId.java:132) 
>>> ~[?:1.8.0_73]
>>> at sun.security.x509.AlgorithmId.(AlgorithmId.java:114) 
>>> ~[?:1.8.0_73]
>>> at 

[graylog2] Re: Graylog is not processing Messages from one input anymore

[Jochen Schalanda]

Hi Keamas,

which version of Graylog are you using? What are the hardware specs of the 
machine(s) you're running Graylog and Elasticsearch on?
What kind of inputs are you talking about?

For Elasticsearch, 1 GB of heap memory is quite little and you should give 
it more memory (depending on how much memory that system has in total).


Cheers,
Jochen

On Friday, 8 July 2016 08:23:30 UTC+2, Keamas M wrote:
>
> Hey my Graylog just stoped processing messages from one input. But the 
> other Input is still working.
>
> Everything looks finde for me:
> I rebooted the Linux machine, Start Stop of the Input and so on. But 
> without success.
>
> root@ATLOG001:/var/log/graylog-server# top
> top - 08:14:49 up 16 min,  1 user,  load average: 2.69, 3.16, 2.63
> Tasks: 140 total,   1 running, 139 sleeping,   0 stopped,   0 zombie
> %Cpu(s): 59.8 us,  0.8 sy,  0.0 ni, 39.4 id,  0.0 wa,  0.0 hi,  0.0 si,  
> 0.0 st
> KiB Mem :  8175468 total,  5032380 free,  1899600 used,  1243488 buff/cache
> KiB Swap:  8386556 total,  8386556 free,0 used.  6002424 avail Mem
>
>
> root@ATLOG001:/var/log/graylog-server# sudo service mongodb status
> ● mongodb.service - An object/document-oriented database
>Loaded: loaded (/lib/systemd/system/mongodb.service; enabled; vendor 
> preset: enabled)
>Active: active (running) since Fri 2016-07-08 07:58:27 CEST; 13min ago
>  Docs: man:mongod(1)
>  Main PID: 765 (mongod)
> Tasks: 25
>Memory: 88.0M
>   CPU: 3.748s
>CGroup: /system.slice/mongodb.service
>└─765 /usr/bin/mongod --config /etc/mongodb.conf
>
> Jul 08 07:58:27 ATLOG001 systemd[1]: Started An object/document-oriented 
> database.
> root@ATLOG001:/var/log/graylog-server#  sudo service elasticsearch status
> ● elasticsearch.service - Elasticsearch
>Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service; enabled; 
> vendor preset: enabled)
>Active: active (running) since Fri 2016-07-08 07:58:30 CEST; 14min ago
>  Docs: http://www.elastic.co
>   Process: 994 
> ExecStartPre=/usr/share/elasticsearch/bin/elasticsearch-systemd-pre-exec 
> (code=exited, status=0/SUCCESS)
>  Main PID: 1011 (java)
> Tasks: 50
>Memory: 895.7M
>   CPU: 1min 45.260s
>CGroup: /system.slice/elasticsearch.service
>└─1011 /usr/bin/java -Xms256m -Xmx1g -Djava.awt.headless=true 
> -XX:+UseParNewGC -XX:+UseConcMarkSweepGC 
> -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly 
> -XX:+HeapD
>
> Jul 08 07:58:36 ATLOG001 elasticsearch[1011]: [2016-07-08 
> 07:58:36,790][INFO ][node ] [Jigsaw] initialized
> Jul 08 07:58:36 ATLOG001 elasticsearch[1011]: [2016-07-08 
> 07:58:36,791][INFO ][node ] [Jigsaw] starting ...
> Jul 08 07:58:36 ATLOG001 elasticsearch[1011]: [2016-07-08 
> 07:58:36,998][INFO ][transport] [Jigsaw] publish_address {
> 127.0.0.1:9300}, bound_addresses {[::1]:9300}, {127.0.0.1:9300}
> Jul 08 07:58:37 ATLOG001 elasticsearch[1011]: [2016-07-08 
> 07:58:37,003][INFO ][discovery] [Jigsaw] 
> graylog/aceP7uo1RTil41lYxtgDsA
> Jul 08 07:58:40 ATLOG001 elasticsearch[1011]: [2016-07-08 
> 07:58:40,067][INFO ][cluster.service  ] [Jigsaw] new_master 
> {Jigsaw}{aceP7uo1RTil41lYxtgDsA}{127.0.0.1}{127.0.0.1:9300}, reason: ze
> Jul 08 07:58:40 ATLOG001 elasticsearch[1011]: [2016-07-08 
> 07:58:40,114][INFO ][http ] [Jigsaw] publish_address {
> 127.0.0.1:9200}, bound_addresses {[::1]:9200}, {127.0.0.1:9200}
> Jul 08 07:58:40 ATLOG001 elasticsearch[1011]: [2016-07-08 
> 07:58:40,116][INFO ][node ] [Jigsaw] started
> Jul 08 07:58:40 ATLOG001 elasticsearch[1011]: [2016-07-08 
> 07:58:40,232][INFO ][gateway  ] [Jigsaw] recovered [2] 
> indices into cluster_state
> Jul 08 07:58:42 ATLOG001 elasticsearch[1011]: [2016-07-08 
> 07:58:42,433][INFO ][cluster.routing.allocation] [Jigsaw] Cluster health 
> status changed from [RED] to [YELLOW] (reason: [shards started [[g
> Jul 08 07:58:48 ATLOG001 elasticsearch[1011]: [2016-07-08 
> 07:58:48,460][INFO ][cluster.service  ] [Jigsaw] added 
> {{graylog-e2b97d26-f84a-4a82-99ba-3bedfbb5b207}{FsfnwA3nRcuTXGNBY9mEdA}{127.
> lines 1-22/22 (END)
> root@ATLOG001:/var/log/graylog-server# sudo service graylog-server status
> ● graylog-server.service - Graylog server
>Loaded: loaded (/usr/lib/systemd/system/graylog-server.service; 
> enabled; vendor preset: enabled)
>Active: active (running) since Fri 2016-07-08 07:58:30 CEST; 14min ago
>  Docs: http://docs.graylog.org/
>  Main PID: 996 (graylog-server)
> Tasks: 183
>Memory: 1.6G
>   CPU: 22min 46.246s
>CGroup: /system.slice/graylog-server.service
>├─ 996 /bin/sh /usr/share/graylog-server/bin/graylog-server
>└─1014 /usr/bin/java -Xms1g -Xmx1g -XX:NewRatio=1 -server 
> -XX:+ResizeTLAB -XX:+UseConcMarkSweepGC -XX:+CMSConcurrentMTEnabled 
> -XX:+CMSClassUnloadingEnabled -XX:+UseParNewGC 

Re: [graylog2] ERROR: org.graylog2.indexer.Indexer - Failed to index [34] messages.

[Tokhan T]

Thankyou Florent B. My issue was solved by "Manually cycle deflector" too.

เมื่อ วันพฤหัสบดีที่ 12 มิถุนายน ค.ศ. 2014 14 นาฬิกา 55 นาที 52 วินาที 
UTC+7, Florent B เขียนว่า:
>
> Hi
>
> Ok  I solved it doing a "Manually cycle deflector" ;-)
>
> Thank you.
>
> On 06/11/2014 07:14 PM, Kay Röpke wrote:
>
> These are usually fields that used to contain a number but now are a 
> string. Thus incompatible.
> Without looking at the index and the messages no one can tell what the 
> problem is :(
> On Jun 11, 2014 6:46 PM, "Florent B"  
> wrote:
>
>> Any idea for these errors ? :-)
>>
>> On 06/10/2014 06:32 PM, Florent B wrote:
>> > I am running 0.20.2 :)
>> >
>> > Server in debug mode is telling this :
>> >
>> > 2014-06-10 18:22:54,071 DEBUG:
>> > org.graylog2.outputs.BatchedElasticSearchOutput - Submitting new flush
>> > thread
>> > 2014-06-10 18:22:54,071 DEBUG:
>> > org.graylog2.outputs.BatchedElasticSearchOutput -
>> > [Thread[pool-9-thread-1,5,main]] Starting flushing 7 messages
>> > 2014-06-10 18:22:54,071 DEBUG: org.graylog2.outputs.ElasticSearchOutput
>> > - Writing <7> messages.
>> > 2014-06-10 18:22:54,073 DEBUG: org.graylog2.indexer.Indexer - Deflector
>> > index: Bulk indexed 7 messages, took 1 ms, failures: true
>> > 2014-06-10 18:22:54,073 ERROR: org.graylog2.indexer.Indexer - Failed to
>> > index [7] messages. Please check the index error log in your web
>> > interface for the reason.
>> >
>> >
>> > I added memory to ES nodes... restarted each node, waited cluster to be
>> > green... but still same messages...
>> >
>> > I did "Recalculate index ranges", still same errors...
>> >
>> > Nothing in ES logs (is debug needed?)
>>
>> --
>> You received this message because you are subscribed to the Google Groups 
>> "graylog2" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to graylog2+u...@googlegroups.com .
>> For more options, visit https://groups.google.com/d/optout.
>>
> -- 
> You received this message because you are subscribed to the Google Groups 
> "graylog2" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to graylog2+u...@googlegroups.com .
> For more options, visit https://groups.google.com/d/optout.
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/b7ac07ed-5f4b-4aeb-bbfb-08f94e2eff5c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [graylog2] Graylog Collector Sidecar - no logs being shipped

[Kev Johnson]

Ok - so I've built a clean Windows Server 2012 R2, disabled the firewall 
and run through the same process with the same result - the only traffic 
back to the Graylog server is the tcp 12900 poll from the collector - I've 
tried logging out/in and rebooting the server which all *should* generate 
some log data. At this point I'm reasonably happy that it's not McAfee 
causing the issue.

Next port of call is going to be adding some snippets from NXlog.conf files 
that I know work, let's see if that makes any difference!

On Friday, 8 July 2016 07:03:27 UTC+1, Kev Johnson wrote:
>
> Thanks Marius - I'll give that a go today. Thanks for sense checking my 
> config and confirming I've not done anything silly!
>
> On Thursday, 7 July 2016 22:30:29 UTC+1, Marius Sturm wrote:
>>
>> Yeah, sounds possible to me. All configurations look correct. So some 
>> Windows firewall might be the root cause. Maybe you can try with a test 
>> host with all firewalls disabled.
>>
>> On 7 July 2016 at 20:38, Kev Johnson  wrote:
>>
>>>
>>> 
>>> Does this help? Given that we're getting nothing but the Sidecar 
>>> checking traffic back from the servers I'm still leaning toward this being 
>>> an issue on the server rather than on the Graylog side. Any known issues 
>>> with McAfee VirusScan Enterprise (beyond the obvious!) - I can't remove it, 
>>> but if I need to tweak it some I probably can. Not 100% certain that this 
>>> would be the case though, as if I just use NXlog to send syslog all works 
>>> fine.
>>>
>>> On Thursday, 7 July 2016 19:27:47 UTC+1, Marius Sturm wrote:

 The generated config looks fine, maybe a screenshot of the Graylog 
 input puts some light on this?

 On 7 July 2016 at 19:50, Kev Johnson  wrote:

> Thanks Marius - I've double checked the input port (and that it's 
> running!), but even if it were a mismatch I'd expect tcpdump to show the 
> packets hitting the interface. I suspect that this has to be down to the 
> generated config, so I'm pasting the contents of one of the servers' 
> configs below - I'm afraid that I'm not really sure how I would 
> troubleshoot that, so I'm happy to be told that I've done something 
> stupid!
>
> define ROOT C:\Program Files (x86)\nxlog
>>
>>
>>> 
>>
>>   Module xm_gelf
>>
>> 
>>
>>
>>>
>>>
>>>
>>> 
>>
>> Module im_msvistalog
>>
>> PollInterval 10
>>
>> SavePos True
>>
>> ReadFromLast True
>>
>> 
>>
>>
>>>
>>>
>>> 
>>
>> Module om_udp
>>
>> Host 192.168.21.12
>>
>> Port 5414
>>
>> OutputType  GELF
>>
>> Exec $short_message = $raw_event; # Avoids truncation of the 
>>> short_message field.
>>
>> Exec $gl2_source_collector = '28a3c8c7-bc02-44e0-98a5-e93e52b057e5';
>>
>> Exec $Hostname = hostname_fqdn();
>>
>> 
>>
>>
>>>
>>>
>>> 
>>
>>   Path 577e5a4bc745f2099c054dd5 => 577e6c75c745f2099c0561b3
>>
>> 
>>
>>
>>>
>>>
> On Thursday, 7 July 2016 18:41:36 UTC+1, Marius Sturm wrote:
>>
>> Hi,
>> you could check if the Gelf port on the Graylog side is exactly the 
>> same as on the Nxlog sender side, usually 12201. Go to System->Inputs 
>> (the 
>> input should have a green badge 'running') verify the port number with 
>> the 
>> one you configured for nxlog in the collector configuration.
>> Another thing, Windows is not sending logs all the time so maybe you 
>> just need to create an event that is triggering a log e.g. opening the 
>> control panel?
>>
>> If that doesn't help please post the generated nxlog configuration, 
>> maybe there is something obvious.
>>
>> On 7 July 2016 at 18:11, Kev Johnson  wrote:
>>
>>> Firstly: I love the idea of being able to push out updated 
>>> configuration files to my collectors. That said: I'm having issues 
>>> getting 
>>> logs to my Graylog box (deployed from the OVA)
>>>
>>> Steps taken so far are as follows
>>>
>>>
>>>- Installed NXlogCE
>>>- Uninstalled the NXlog service
>>>- Installed the Graylog Collector Sidecar
>>>- Edited the sidecar_collector.yml file to point to my Graylog 
>>>server, and remove the reference to IIS
>>>- Installed the Graylog Collector Sidecar service
>>>- Started the Graylog Collector Sidecar service
>>>- Created a configuration (Windows Logs, ship to the UDP GELF 
>>>Input defined on my Graylog box)
>>>- Created a tag called Windows and applied it to this 
>>>configuration
>>>
>>>
>>> I 

[graylog2] Re: When is Graylog 2.1 releasing?

[Jochen Schalanda]

Hi Paul,

we're targeting August 2016 for releasing Graylog 2.1.0.

You can help by testing the alpha and beta versions until then.


Cheers,
Jochen

On Friday, 8 July 2016 03:07:53 UTC+2, Paul Mendoza wrote:
>
> When will I be able to use Graylog 2.1?
>
>  I'm waiting for the TCP TLS Graylog Collector Sidecar support which is 
> included with it. 
>
>
> Also, what new features will be included? 
>
> Is there a release notes page covering 2.1 yet? 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/357a99ca-d224-4d19-9725-d74c66943e0c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Graylog is not processing Messages from one input anymore

[Keamas M]

Hey my Graylog just stoped processing messages from one input. But the 
other Input is still working.

Everything looks finde for me:
I rebooted the Linux machine, Start Stop of the Input and so on. But 
without success.

root@ATLOG001:/var/log/graylog-server# top
top - 08:14:49 up 16 min,  1 user,  load average: 2.69, 3.16, 2.63
Tasks: 140 total,   1 running, 139 sleeping,   0 stopped,   0 zombie
%Cpu(s): 59.8 us,  0.8 sy,  0.0 ni, 39.4 id,  0.0 wa,  0.0 hi,  0.0 si,  
0.0 st
KiB Mem :  8175468 total,  5032380 free,  1899600 used,  1243488 buff/cache
KiB Swap:  8386556 total,  8386556 free,0 used.  6002424 avail Mem


root@ATLOG001:/var/log/graylog-server# sudo service mongodb status
● mongodb.service - An object/document-oriented database
   Loaded: loaded (/lib/systemd/system/mongodb.service; enabled; vendor 
preset: enabled)
   Active: active (running) since Fri 2016-07-08 07:58:27 CEST; 13min ago
 Docs: man:mongod(1)
 Main PID: 765 (mongod)
Tasks: 25
   Memory: 88.0M
  CPU: 3.748s
   CGroup: /system.slice/mongodb.service
   └─765 /usr/bin/mongod --config /etc/mongodb.conf

Jul 08 07:58:27 ATLOG001 systemd[1]: Started An object/document-oriented 
database.
root@ATLOG001:/var/log/graylog-server#  sudo service elasticsearch status
● elasticsearch.service - Elasticsearch
   Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service; enabled; 
vendor preset: enabled)
   Active: active (running) since Fri 2016-07-08 07:58:30 CEST; 14min ago
 Docs: http://www.elastic.co
  Process: 994 
ExecStartPre=/usr/share/elasticsearch/bin/elasticsearch-systemd-pre-exec 
(code=exited, status=0/SUCCESS)
 Main PID: 1011 (java)
Tasks: 50
   Memory: 895.7M
  CPU: 1min 45.260s
   CGroup: /system.slice/elasticsearch.service
   └─1011 /usr/bin/java -Xms256m -Xmx1g -Djava.awt.headless=true 
-XX:+UseParNewGC -XX:+UseConcMarkSweepGC 
-XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly 
-XX:+HeapD

Jul 08 07:58:36 ATLOG001 elasticsearch[1011]: [2016-07-08 
07:58:36,790][INFO ][node ] [Jigsaw] initialized
Jul 08 07:58:36 ATLOG001 elasticsearch[1011]: [2016-07-08 
07:58:36,791][INFO ][node ] [Jigsaw] starting ...
Jul 08 07:58:36 ATLOG001 elasticsearch[1011]: [2016-07-08 
07:58:36,998][INFO ][transport] [Jigsaw] publish_address 
{127.0.0.1:9300}, bound_addresses {[::1]:9300}, {127.0.0.1:9300}
Jul 08 07:58:37 ATLOG001 elasticsearch[1011]: [2016-07-08 
07:58:37,003][INFO ][discovery] [Jigsaw] 
graylog/aceP7uo1RTil41lYxtgDsA
Jul 08 07:58:40 ATLOG001 elasticsearch[1011]: [2016-07-08 
07:58:40,067][INFO ][cluster.service  ] [Jigsaw] new_master 
{Jigsaw}{aceP7uo1RTil41lYxtgDsA}{127.0.0.1}{127.0.0.1:9300}, reason: ze
Jul 08 07:58:40 ATLOG001 elasticsearch[1011]: [2016-07-08 
07:58:40,114][INFO ][http ] [Jigsaw] publish_address 
{127.0.0.1:9200}, bound_addresses {[::1]:9200}, {127.0.0.1:9200}
Jul 08 07:58:40 ATLOG001 elasticsearch[1011]: [2016-07-08 
07:58:40,116][INFO ][node ] [Jigsaw] started
Jul 08 07:58:40 ATLOG001 elasticsearch[1011]: [2016-07-08 
07:58:40,232][INFO ][gateway  ] [Jigsaw] recovered [2] 
indices into cluster_state
Jul 08 07:58:42 ATLOG001 elasticsearch[1011]: [2016-07-08 
07:58:42,433][INFO ][cluster.routing.allocation] [Jigsaw] Cluster health 
status changed from [RED] to [YELLOW] (reason: [shards started [[g
Jul 08 07:58:48 ATLOG001 elasticsearch[1011]: [2016-07-08 
07:58:48,460][INFO ][cluster.service  ] [Jigsaw] added 
{{graylog-e2b97d26-f84a-4a82-99ba-3bedfbb5b207}{FsfnwA3nRcuTXGNBY9mEdA}{127.
lines 1-22/22 (END)
root@ATLOG001:/var/log/graylog-server# sudo service graylog-server status
● graylog-server.service - Graylog server
   Loaded: loaded (/usr/lib/systemd/system/graylog-server.service; enabled; 
vendor preset: enabled)
   Active: active (running) since Fri 2016-07-08 07:58:30 CEST; 14min ago
 Docs: http://docs.graylog.org/
 Main PID: 996 (graylog-server)
Tasks: 183
   Memory: 1.6G
  CPU: 22min 46.246s
   CGroup: /system.slice/graylog-server.service
   ├─ 996 /bin/sh /usr/share/graylog-server/bin/graylog-server
   └─1014 /usr/bin/java -Xms1g -Xmx1g -XX:NewRatio=1 -server 
-XX:+ResizeTLAB -XX:+UseConcMarkSweepGC -XX:+CMSConcurrentMTEnabled 
-XX:+CMSClassUnloadingEnabled -XX:+UseParNewGC -XX:-OmitStac

Jul 08 07:58:30 ATLOG001 systemd[1]: Started Graylog server.
lines 1-13/13 (END)


I received the last messages at 07/08/2016 7:26
I found this in the log:
This happened now multiple times it seems if I delete the input and create 
a new one it works againg.
But why does this happen and how can I reactivate to get messages again 
without deleting the Input.

2016-07-08T07:01:30.077+02:00 INFO  [DashboardsResource] Deleted dashboard 
<577e30f817fd3004026bf6e6>. Reason: REST request.
2016-07-08T07:02:21.433+02:00 ERROR [SearchResource] Field histogram query 
failed. Make sure 

Re: [graylog2] Graylog Collector Sidecar - no logs being shipped

[Kev Johnson]

Thanks Marius - I'll give that a go today. Thanks for sense checking my 
config and confirming I've not done anything silly!

On Thursday, 7 July 2016 22:30:29 UTC+1, Marius Sturm wrote:
>
> Yeah, sounds possible to me. All configurations look correct. So some 
> Windows firewall might be the root cause. Maybe you can try with a test 
> host with all firewalls disabled.
>
> On 7 July 2016 at 20:38, Kev Johnson  > wrote:
>
>>
>> 
>> Does this help? Given that we're getting nothing but the Sidecar checking 
>> traffic back from the servers I'm still leaning toward this being an issue 
>> on the server rather than on the Graylog side. Any known issues with McAfee 
>> VirusScan Enterprise (beyond the obvious!) - I can't remove it, but if I 
>> need to tweak it some I probably can. Not 100% certain that this would be 
>> the case though, as if I just use NXlog to send syslog all works fine.
>>
>> On Thursday, 7 July 2016 19:27:47 UTC+1, Marius Sturm wrote:
>>>
>>> The generated config looks fine, maybe a screenshot of the Graylog input 
>>> puts some light on this?
>>>
>>> On 7 July 2016 at 19:50, Kev Johnson  wrote:
>>>
 Thanks Marius - I've double checked the input port (and that it's 
 running!), but even if it were a mismatch I'd expect tcpdump to show the 
 packets hitting the interface. I suspect that this has to be down to the 
 generated config, so I'm pasting the contents of one of the servers' 
 configs below - I'm afraid that I'm not really sure how I would 
 troubleshoot that, so I'm happy to be told that I've done something stupid!

 define ROOT C:\Program Files (x86)\nxlog
>
>
>> 
>
>   Module xm_gelf
>
> 
>
>
>>
>>
>>
>> 
>
> Module im_msvistalog
>
> PollInterval 10
>
> SavePos True
>
> ReadFromLast True
>
> 
>
>
>>
>>
>> 
>
> Module om_udp
>
> Host 192.168.21.12
>
> Port 5414
>
> OutputType  GELF
>
> Exec $short_message = $raw_event; # Avoids truncation of the 
>> short_message field.
>
> Exec $gl2_source_collector = '28a3c8c7-bc02-44e0-98a5-e93e52b057e5';
>
> Exec $Hostname = hostname_fqdn();
>
> 
>
>
>>
>>
>> 
>
>   Path 577e5a4bc745f2099c054dd5 => 577e6c75c745f2099c0561b3
>
> 
>
>
>>
>>
 On Thursday, 7 July 2016 18:41:36 UTC+1, Marius Sturm wrote:
>
> Hi,
> you could check if the Gelf port on the Graylog side is exactly the 
> same as on the Nxlog sender side, usually 12201. Go to System->Inputs 
> (the 
> input should have a green badge 'running') verify the port number with 
> the 
> one you configured for nxlog in the collector configuration.
> Another thing, Windows is not sending logs all the time so maybe you 
> just need to create an event that is triggering a log e.g. opening the 
> control panel?
>
> If that doesn't help please post the generated nxlog configuration, 
> maybe there is something obvious.
>
> On 7 July 2016 at 18:11, Kev Johnson  wrote:
>
>> Firstly: I love the idea of being able to push out updated 
>> configuration files to my collectors. That said: I'm having issues 
>> getting 
>> logs to my Graylog box (deployed from the OVA)
>>
>> Steps taken so far are as follows
>>
>>
>>- Installed NXlogCE
>>- Uninstalled the NXlog service
>>- Installed the Graylog Collector Sidecar
>>- Edited the sidecar_collector.yml file to point to my Graylog 
>>server, and remove the reference to IIS
>>- Installed the Graylog Collector Sidecar service
>>- Started the Graylog Collector Sidecar service
>>- Created a configuration (Windows Logs, ship to the UDP GELF 
>>Input defined on my Graylog box)
>>- Created a tag called Windows and applied it to this 
>>configuration
>>
>>
>> I see the nxlog.conf get created on the Windows server, I see 
>> nxlog.exe start up on server, but nothing is sent. TCPDump on the 
>> Graylog 
>> server shows only the TCP connections in on port 12900 from the Windows 
>> server.
>>
>> Any advice on troubleshooting this would be much appreciated!
>>
>> -- 
>> You received this message because you are subscribed to the Google 
>> Groups "Graylog Users" group.
>> To unsubscribe from this group and stop receiving emails from it, 
>> send an email to graylog2+u...@googlegroups.com.
>> To view this discussion on the web visit 
>>