Hey my Graylog just stoped processing messages from one input. But the other Input is still working.
Everything looks finde for me: I rebooted the Linux machine, Start Stop of the Input and so on. But without success. root@ATLOG001:/var/log/graylog-server# top top - 08:14:49 up 16 min, 1 user, load average: 2.69, 3.16, 2.63 Tasks: 140 total, 1 running, 139 sleeping, 0 stopped, 0 zombie %Cpu(s): 59.8 us, 0.8 sy, 0.0 ni, 39.4 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st KiB Mem : 8175468 total, 5032380 free, 1899600 used, 1243488 buff/cache KiB Swap: 8386556 total, 8386556 free, 0 used. 6002424 avail Mem root@ATLOG001:/var/log/graylog-server# sudo service mongodb status ● mongodb.service - An object/document-oriented database Loaded: loaded (/lib/systemd/system/mongodb.service; enabled; vendor preset: enabled) Active: active (running) since Fri 2016-07-08 07:58:27 CEST; 13min ago Docs: man:mongod(1) Main PID: 765 (mongod) Tasks: 25 Memory: 88.0M CPU: 3.748s CGroup: /system.slice/mongodb.service └─765 /usr/bin/mongod --config /etc/mongodb.conf Jul 08 07:58:27 ATLOG001 systemd[1]: Started An object/document-oriented database. root@ATLOG001:/var/log/graylog-server# sudo service elasticsearch status ● elasticsearch.service - Elasticsearch Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service; enabled; vendor preset: enabled) Active: active (running) since Fri 2016-07-08 07:58:30 CEST; 14min ago Docs: http://www.elastic.co Process: 994 ExecStartPre=/usr/share/elasticsearch/bin/elasticsearch-systemd-pre-exec (code=exited, status=0/SUCCESS) Main PID: 1011 (java) Tasks: 50 Memory: 895.7M CPU: 1min 45.260s CGroup: /system.slice/elasticsearch.service └─1011 /usr/bin/java -Xms256m -Xmx1g -Djava.awt.headless=true -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapD Jul 08 07:58:36 ATLOG001 elasticsearch[1011]: [2016-07-08 07:58:36,790][INFO ][node ] [Jigsaw] initialized Jul 08 07:58:36 ATLOG001 elasticsearch[1011]: [2016-07-08 07:58:36,791][INFO ][node ] [Jigsaw] starting ... Jul 08 07:58:36 ATLOG001 elasticsearch[1011]: [2016-07-08 07:58:36,998][INFO ][transport ] [Jigsaw] publish_address {127.0.0.1:9300}, bound_addresses {[::1]:9300}, {127.0.0.1:9300} Jul 08 07:58:37 ATLOG001 elasticsearch[1011]: [2016-07-08 07:58:37,003][INFO ][discovery ] [Jigsaw] graylog/aceP7uo1RTil41lYxtgDsA Jul 08 07:58:40 ATLOG001 elasticsearch[1011]: [2016-07-08 07:58:40,067][INFO ][cluster.service ] [Jigsaw] new_master {Jigsaw}{aceP7uo1RTil41lYxtgDsA}{127.0.0.1}{127.0.0.1:9300}, reason: ze Jul 08 07:58:40 ATLOG001 elasticsearch[1011]: [2016-07-08 07:58:40,114][INFO ][http ] [Jigsaw] publish_address {127.0.0.1:9200}, bound_addresses {[::1]:9200}, {127.0.0.1:9200} Jul 08 07:58:40 ATLOG001 elasticsearch[1011]: [2016-07-08 07:58:40,116][INFO ][node ] [Jigsaw] started Jul 08 07:58:40 ATLOG001 elasticsearch[1011]: [2016-07-08 07:58:40,232][INFO ][gateway ] [Jigsaw] recovered [2] indices into cluster_state Jul 08 07:58:42 ATLOG001 elasticsearch[1011]: [2016-07-08 07:58:42,433][INFO ][cluster.routing.allocation] [Jigsaw] Cluster health status changed from [RED] to [YELLOW] (reason: [shards started [[g Jul 08 07:58:48 ATLOG001 elasticsearch[1011]: [2016-07-08 07:58:48,460][INFO ][cluster.service ] [Jigsaw] added {{graylog-e2b97d26-f84a-4a82-99ba-3bedfbb5b207}{FsfnwA3nRcuTXGNBY9mEdA}{127. lines 1-22/22 (END) root@ATLOG001:/var/log/graylog-server# sudo service graylog-server status ● graylog-server.service - Graylog server Loaded: loaded (/usr/lib/systemd/system/graylog-server.service; enabled; vendor preset: enabled) Active: active (running) since Fri 2016-07-08 07:58:30 CEST; 14min ago Docs: http://docs.graylog.org/ Main PID: 996 (graylog-server) Tasks: 183 Memory: 1.6G CPU: 22min 46.246s CGroup: /system.slice/graylog-server.service ├─ 996 /bin/sh /usr/share/graylog-server/bin/graylog-server └─1014 /usr/bin/java -Xms1g -Xmx1g -XX:NewRatio=1 -server -XX:+ResizeTLAB -XX:+UseConcMarkSweepGC -XX:+CMSConcurrentMTEnabled -XX:+CMSClassUnloadingEnabled -XX:+UseParNewGC -XX:-OmitStac Jul 08 07:58:30 ATLOG001 systemd[1]: Started Graylog server. lines 1-13/13 (END) I received the last messages at 07/08/2016 7:26 I found this in the log: This happened now multiple times it seems if I delete the input and create a new one it works againg. But why does this happen and how can I reactivate to get messages again without deleting the Input. 2016-07-08T07:01:30.077+02:00 INFO [DashboardsResource] Deleted dashboard <577e30f817fd3004026bf6e6>. Reason: REST request. 2016-07-08T07:02:21.433+02:00 ERROR [SearchResource] Field histogram query failed. Make sure that field [Content_Length] is a numeric type. 2016-07-08T07:11:18.753+02:00 WARN [SearchResource] Unable to execute search: all shards failed 2016-07-08T07:13:31.818+02:00 WARN [SearchResource] Unable to execute search: all shards failed 2016-07-08T07:13:43.116+02:00 WARN [SearchResource] Unable to execute search: all shards failed 2016-07-08T07:16:23.267+02:00 WARN [SearchResource] Unable to execute search: all shards failed 2016-07-08T07:17:07.044+02:00 WARN [SearchResource] Unable to execute search: all shards failed 2016-07-08T07:31:38.253+02:00 WARN [SearchResource] Unable to execute search: all shards failed 2016-07-08T07:32:02.112+02:00 WARN [SearchResource] Unable to execute search: all shards failed 2016-07-08T07:32:02.113+02:00 ERROR [AnyExceptionClassMapper] Unhandled exception in REST resource java.lang.NullPointerException at org.apache.lucene.queryparser.classic.ParseException.initialise(ParseException.java:120) ~[graylog.jar:?] at org.apache.lucene.queryparser.classic.ParseException.<init>(ParseException.java:34) ~[graylog.jar:?] at org.apache.lucene.queryparser.classic.QueryParser.generateParseException(QueryParser.java:698) ~[graylog.jar:?] at org.graylog2.rest.resources.search.SearchResource.createRequestExceptionForParseFailure(SearchResource.java:248) ~[graylog.jar:?] at org.graylog2.rest.resources.search.RelativeSearchResource.searchRelative(RelativeSearchResource.java:107) ~[graylog.jar:?] at sun.reflect.GeneratedMethodAccessor214.invoke(Unknown Source) ~[?:?] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_91] at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_91] at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81) ~[graylog.jar:?] at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:144) ~[graylog.jar:?] at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:161) ~[graylog.jar:?] at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:205) ~[graylog.jar:?] at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:99) ~[graylog.jar:?] at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:389) ~[graylog.jar:?] at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:347) ~[graylog.jar:?] at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:102) ~[graylog.jar:?] at org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:326) [graylog.jar:?] at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271) [graylog.jar:?] at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267) [graylog.jar:?] at org.glassfish.jersey.internal.Errors.process(Errors.java:315) [graylog.jar:?] at org.glassfish.jersey.internal.Errors.process(Errors.java:297) [graylog.jar:?] at org.glassfish.jersey.internal.Errors.process(Errors.java:267) [graylog.jar:?] at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317) [graylog.jar:?] at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305) [graylog.jar:?] at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154) [graylog.jar:?] at org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer.service(GrizzlyHttpContainer.java:384) [graylog.jar:?] at org.glassfish.grizzly.http.server.HttpHandler$1.run(HttpHandler.java:224) [graylog.jar:?] at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176) [graylog.jar:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_91] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_91] at java.lang.Thread.run(Thread.java:745) [?:1.8.0_91] 2016-07-08T07:32:26.164+02:00 WARN [SearchResource] Unable to execute search: all shards failed 2016-07-08T07:32:26.164+02:00 ERROR [AnyExceptionClassMapper] Unhandled exception in REST resource java.lang.NullPointerException at org.apache.lucene.queryparser.classic.ParseException.initialise(ParseException.java:120) ~[graylog.jar:?] at org.apache.lucene.queryparser.classic.ParseException.<init>(ParseException.java:34) ~[graylog.jar:?] at org.apache.lucene.queryparser.classic.QueryParser.generateParseException(QueryParser.java:698) ~[graylog.jar:?] at org.graylog2.rest.resources.search.SearchResource.createRequestExceptionForParseFailure(SearchResource.java:248) ~[graylog.jar:?] at org.graylog2.rest.resources.search.RelativeSearchResource.searchRelative(RelativeSearchResource.java:107) ~[graylog.jar:?] at sun.reflect.GeneratedMethodAccessor214.invoke(Unknown Source) ~[?:?] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_91] at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_91] at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81) ~[graylog.jar:?] at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:144) ~[graylog.jar:?] at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:161) ~[graylog.jar:?] at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:205) ~[graylog.jar:?] at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:99) ~[graylog.jar:?] at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:389) ~[graylog.jar:?] at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:347) ~[graylog.jar:?] at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:102) ~[graylog.jar:?] at org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:326) [graylog.jar:?] at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271) [graylog.jar:?] at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267) [graylog.jar:?] at org.glassfish.jersey.internal.Errors.process(Errors.java:315) [graylog.jar:?] at org.glassfish.jersey.internal.Errors.process(Errors.java:297) [graylog.jar:?] at org.glassfish.jersey.internal.Errors.process(Errors.java:267) [graylog.jar:?] at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317) [graylog.jar:?] at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305) [graylog.jar:?] at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154) [graylog.jar:?] at org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer.service(GrizzlyHttpContainer.java:384) [graylog.jar:?] at org.glassfish.grizzly.http.server.HttpHandler$1.run(HttpHandler.java:224) [graylog.jar:?] at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176) [graylog.jar:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_91] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_91] at java.lang.Thread.run(Thread.java:745) [?:1.8.0_91] 2016-07-08T07:32:44.937+02:00 WARN [SearchResource] Unable to execute search: all shards failed 2016-07-08T07:32:44.937+02:00 ERROR [AnyExceptionClassMapper] Unhandled exception in REST resource java.lang.NullPointerException at org.apache.lucene.queryparser.classic.ParseException.initialise(ParseException.java:120) ~[graylog.jar:?] at org.apache.lucene.queryparser.classic.ParseException.<init>(ParseException.java:34) ~[graylog.jar:?] at org.apache.lucene.queryparser.classic.QueryParser.generateParseException(QueryParser.java:698) ~[graylog.jar:?] at org.graylog2.rest.resources.search.SearchResource.createRequestExceptionForParseFailure(SearchResource.java:248) ~[graylog.jar:?] at org.graylog2.rest.resources.search.RelativeSearchResource.searchRelative(RelativeSearchResource.java:107) ~[graylog.jar:?] at sun.reflect.GeneratedMethodAccessor214.invoke(Unknown Source) ~[?:?] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_91] at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_91] at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81) ~[graylog.jar:?] at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:144) ~[graylog.jar:?] at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:161) ~[graylog.jar:?] at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:205) ~[graylog.jar:?] at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:99) ~[graylog.jar:?] at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:389) ~[graylog.jar:?] at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:347) ~[graylog.jar:?] at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:102) ~[graylog.jar:?] at org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:326) [graylog.jar:?] at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271) [graylog.jar:?] at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267) [graylog.jar:?] at org.glassfish.jersey.internal.Errors.process(Errors.java:315) [graylog.jar:?] at org.glassfish.jersey.internal.Errors.process(Errors.java:297) [graylog.jar:?] at org.glassfish.jersey.internal.Errors.process(Errors.java:267) [graylog.jar:?] at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317) [graylog.jar:?] at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305) [graylog.jar:?] at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154) [graylog.jar:?] at org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer.service(GrizzlyHttpContainer.java:384) [graylog.jar:?] at org.glassfish.grizzly.http.server.HttpHandler$1.run(HttpHandler.java:224) [graylog.jar:?] at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176) [graylog.jar:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_91] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_91] at java.lang.Thread.run(Thread.java:745) [?:1.8.0_91] 2016-07-08T07:44:06.963+02:00 WARN [connection] Got socket exception on connection [connectionId{localValue:14, serverValue:12}] to localhost:27017. All connections to localhost:27017 will be closed. 2016-07-08T07:44:06.975+02:00 INFO [connection] Closed connection [connectionId{localValue:14, serverValue:12}] to localhost:27017 because there was a socket exception raised by this connection. 2016-07-08T07:44:06.975+02:00 ERROR [StreamRouter] Stream router engine update failed! com.mongodb.MongoSocketReadException: Prematurely reached end of stream at com.mongodb.connection.SocketStream.read(SocketStream.java:88) ~[graylog.jar:?] at com.mongodb.connection.InternalStreamConnection.receiveResponseBuffers(InternalStreamConnection.java:491) ~[graylog.jar:?] at com.mongodb.connection.InternalStreamConnection.receiveMessage(InternalStreamConnection.java:221) ~[graylog.jar:?] at com.mongodb.connection.UsageTrackingInternalConnection.receiveMessage(UsageTrackingInternalConnection.java:102) ~[graylog.jar:?] at com.mongodb.connection.DefaultConnectionPool$PooledConnection.receiveMessage(DefaultConnectionPool.java:435) ~[graylog.jar:?] at com.mongodb.connection.QueryProtocol.execute(QueryProtocol.java:297) ~[graylog.jar:?] at com.mongodb.connection.QueryProtocol.execute(QueryProtocol.java:54) ~[graylog.jar:?] at com.mongodb.connection.DefaultServer$DefaultServerProtocolExecutor.execute(DefaultServer.java:159) ~[graylog.jar:?] at com.mongodb.connection.DefaultServerConnection.executeProtocol(DefaultServerConnection.java:286) ~[graylog.jar:?] at com.mongodb.connection.DefaultServerConnection.query(DefaultServerConnection.java:209) ~[graylog.jar:?] at com.mongodb.operation.FindOperation$1.call(FindOperation.java:496) ~[graylog.jar:?] at com.mongodb.operation.FindOperation$1.call(FindOperation.java:482) ~[graylog.jar:?] at com.mongodb.operation.OperationHelper.withConnectionSource(OperationHelper.java:239) ~[graylog.jar:?] at com.mongodb.operation.OperationHelper.withConnection(OperationHelper.java:212) ~[graylog.jar:?] at com.mongodb.operation.FindOperation.execute(FindOperation.java:482) ~[graylog.jar:?] at com.mongodb.operation.FindOperation.execute(FindOperation.java:79) ~[graylog.jar:?] at com.mongodb.Mongo.execute(Mongo.java:772) ~[graylog.jar:?] at com.mongodb.Mongo$2.execute(Mongo.java:759) ~[graylog.jar:?] at com.mongodb.DBCursor.initializeCursor(DBCursor.java:851) ~[graylog.jar:?] at com.mongodb.DBCursor.hasNext(DBCursor.java:152) ~[graylog.jar:?] at org.graylog2.database.PersistedServiceImpl.cursorToList(PersistedServiceImpl.java:124) ~[graylog.jar:?] at org.graylog2.database.PersistedServiceImpl.query(PersistedServiceImpl.java:67) ~[graylog.jar:?] at org.graylog2.database.PersistedServiceImpl.query(PersistedServiceImpl.java:71) ~[graylog.jar:?] at org.graylog2.streams.StreamServiceImpl.loadAll(StreamServiceImpl.java:140) ~[graylog.jar:?] at org.graylog2.streams.StreamServiceImpl.loadAllEnabled(StreamServiceImpl.java:121) ~[graylog.jar:?] at org.graylog2.streams.StreamServiceImpl.loadAllEnabled(StreamServiceImpl.java:114) ~[graylog.jar:?] at org.graylog2.streams.StreamRouter$StreamRouterEngineUpdater.getNewEngine(StreamRouter.java:112) ~[graylog.jar:?] at org.graylog2.streams.StreamRouter$StreamRouterEngineUpdater.run(StreamRouter.java:97) [graylog.jar:?] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_91] at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [?:1.8.0_91] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [?:1.8.0_91] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [?:1.8.0_91] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_91] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_91] at java.lang.Thread.run(Thread.java:745) [?:1.8.0_91] 2016-07-08T07:44:07.039+02:00 INFO [Server] SIGNAL received. Shutting down. 2016-07-08T07:44:07.040+02:00 INFO [cluster] No server chosen by WritableServerSelector from cluster description ClusterDescription{type=UNKNOWN, connectionMode=SINGLE, all=[ServerDescription{address=localhost:27017, type=UNKNOWN, state=CONNECTING}]}. Waiting for 30000 ms before timing out 2016-07-08T07:44:07.133+02:00 INFO [zen] [graylog-e2b97d26-f84a-4a82-99ba-3bedfbb5b207] master_left [{Wong}{ZKvMBV5mSVGlG2HFGYQsYw}{127.0.0.1}{127.0.0.1:9300}], reason [transport disconnected] 2016-07-08T07:44:07.133+02:00 WARN [zen] [graylog-e2b97d26-f84a-4a82-99ba-3bedfbb5b207] master left (reason = transport disconnected), current nodes: {{graylog-e2b97d26-f84a-4a82-99ba-3bedfbb5b207}{Rbzi7_fvSDGN9EnLCdB0Eg}{127.0.0.1}{127.0.0.1:9350}{client=true, data=false, master=false},} 2016-07-08T07:44:07.134+02:00 INFO [service] [graylog-e2b97d26-f84a-4a82-99ba-3bedfbb5b207] removed {{Wong}{ZKvMBV5mSVGlG2HFGYQsYw}{127.0.0.1}{127.0.0.1:9300},}, reason: zen-disco-master_failed ({Wong}{ZKvMBV5mSVGlG2HFGYQsYw}{127.0.0.1}{127.0.0.1:9300}) 2016-07-08T07:44:07.134+02:00 WARN [ClusterStateMonitor] No Elasticsearch data nodes in cluster, cluster is completely offline. 2016-07-08T07:44:07.136+02:00 INFO [cluster] No server chosen by WritableServerSelector from cluster description ClusterDescription{type=UNKNOWN, connectionMode=SINGLE, all=[ServerDescription{address=localhost:27017, type=UNKNOWN, state=CONNECTING}]}. Waiting for 30000 ms before timing out 2016-07-08T07:44:07.173+02:00 INFO [cluster] No server chosen by ReadPreferenceServerSelector{readPreference=primary} from cluster description ClusterDescription{type=UNKNOWN, connectionMode=SINGLE, all=[ServerDescription{address=localhost:27017, type=UNKNOWN, state=CONNECTING}]}. Waiting for 30000 ms before timing out 2016-07-08T07:44:07.217+02:00 INFO [cluster] No server chosen by ReadPreferenceServerSelector{readPreference=primary} from cluster description ClusterDescription{type=UNKNOWN, connectionMode=SINGLE, all=[ServerDescription{address=localhost:27017, type=UNKNOWN, state=CONNECTING}]}. Waiting for 30000 ms before timing out 2016-07-08T07:44:07.217+02:00 INFO [cluster] No server chosen by ReadPreferenceServerSelector{readPreference=primary} from cluster description ClusterDescription{type=UNKNOWN, connectionMode=SINGLE, all=[ServerDescription{address=localhost:27017, type=UNKNOWN, state=CONNECTING}]}. Waiting for 30000 ms before timing out 2016-07-08T07:44:07.197+02:00 INFO [cluster] Exception in monitor thread while connecting to server localhost:27017 com.mongodb.MongoSocketOpenException: Exception opening socket at com.mongodb.connection.SocketStream.open(SocketStream.java:63) ~[graylog.jar:?] at com.mongodb.connection.InternalStreamConnection.open(InternalStreamConnection.java:114) ~[graylog.jar:?] at com.mongodb.connection.DefaultServerMonitor$ServerMonitorRunnable.run(DefaultServerMonitor.java:128) [graylog.jar:?] at java.lang.Thread.run(Thread.java:745) [?:1.8.0_91] Caused by: java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) ~[?:1.8.0_91] at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) ~[?:1.8.0_91] at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) ~[?:1.8.0_91] at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) ~[?:1.8.0_91] at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) ~[?:1.8.0_91] at java.net.Socket.connect(Socket.java:589) ~[?:1.8.0_91] at com.mongodb.connection.SocketStreamHelper.initialize(SocketStreamHelper.java:50) ~[graylog.jar:?] at com.mongodb.connection.SocketStream.open(SocketStream.java:58) ~[graylog.jar:?] ... 3 more -- You received this message because you are subscribed to the Google Groups "Graylog Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/graylog2/03379193-549f-477b-b0f7-42cc6dec70c5%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.