[graylog2] Re: Graylog IO Exception Error
Increasing the heap size on ES and Graylog respectively fixed the issue. On Friday, July 8, 2016 at 11:07:46 AM UTC-5, Ariel Godinez wrote: > > After further investigation I think this was due to elasticsearch and > graylog being overloaded. I have increased their heap sizes accordingly and > will see how the system performs. > > Ariel > > On Wednesday, July 6, 2016 at 12:21:11 PM UTC-5, Ariel Godinez wrote: >> >> Hello, >> >> I've been using graylog for a couple weeks now and started to notice some >> unusual behavior today. I am currently running a single node setup. >> >> The Issue: >> >> Every once in awhile I start to notice that that graylog is dragging >> quite a bit (the loading spinner is persisting much longer than usual) so I >> go check the logs and find the following error message. >> >> ERROR [ServerRuntime$Responder] An I/O error has occurred while writing a >> response message entity to the container output stream. >> org.glassfish.jersey.server.internal.process.MappableException: >> java.io.IOException: Connection closed >> at >> org.glassfish.jersey.server.internal.MappableExceptionWrapperInterceptor.aroundWriteTo(MappableExceptionWrapperInterceptor.java:92) >> >> ~[graylog.jar:?] >> at >> org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:162) >> >> ~[graylog.jar:?] >> at >> org.glassfish.jersey.message.internal.MessageBodyFactory.writeTo(MessageBodyFactory.java:1130) >> >> ~[graylog.jar:?] >> at >> org.glassfish.jersey.server.ServerRuntime$Responder.writeResponse(ServerRuntime.java:711) >> >> [graylog.jar:?] >> at >> org.glassfish.jersey.server.ServerRuntime$Responder.processResponse(ServerRuntime.java:444) >> >> [graylog.jar:?] >> at >> org.glassfish.jersey.server.ServerRuntime$Responder.process(ServerRuntime.java:434) >> >> [graylog.jar:?] >> at >> org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:329) >> [graylog.jar:?] >> at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271) >> [graylog.jar:?] >> at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267) >> [graylog.jar:?] >> at org.glassfish.jersey.internal.Errors.process(Errors.java:315) >> [graylog.jar:?] >> at org.glassfish.jersey.internal.Errors.process(Errors.java:297) >> [graylog.jar:?] >> at org.glassfish.jersey.internal.Errors.process(Errors.java:267) >> [graylog.jar:?] >> at >> org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317) >> >> [graylog.jar:?] >> at >> org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305) >> [graylog.jar:?] >> at >> org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154) >> >> [graylog.jar:?] >> at >> org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer.service(GrizzlyHttpContainer.java:384) >> >> [graylog.jar:?] >> at >> org.glassfish.grizzly.http.server.HttpHandler$1.run(HttpHandler.java:224) >> [graylog.jar:?] >> at >> com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176) >> >> [graylog.jar:?] >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) >> >> [?:1.8.0_91] >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) >> >> [?:1.8.0_91] >> at java.lang.Thread.run(Thread.java:745) [?:1.8.0_91] >> Caused by: java.io.IOException: Connection closed >> at >> org.glassfish.grizzly.asyncqueue.TaskQueue.onClose(TaskQueue.java:317) >> ~[graylog.jar:?] >> at >> org.glassfish.grizzly.nio.AbstractNIOAsyncQueueWriter.onClose(AbstractNIOAsyncQueueWriter.java:501) >> >> ~[graylog.jar:?] >> at >> org.glassfish.grizzly.nio.transport.TCPNIOTransport.closeConnection(TCPNIOTransport.java:412) >> >> ~[graylog.jar:?] >> at >> org.glassfish.grizzly.nio.NIOConnection.doClose(NIOConnection.java:604) >> ~[graylog.jar:?] >> at >> org.glassfish.grizzly.nio.NIOConnection$5.run(NIOConnection.java:570) >> ~[graylog.jar:?] >> at >> org.glassfish.grizzly.nio.DefaultSelectorHandler.execute(DefaultSelectorHandler.java:235) >> >> ~[graylog.jar:?] >> at >> org.glassfish.grizzly.nio.NIOConnection.terminate0(NIOConnection.java:564) >> ~[graylog.jar:?] >> at >> org.glassfish.grizzly.nio.transport.TCPNIOConnection.terminate0(TCPNIOConnection.java:291) >> >> ~[graylog.jar:?] >> at >> org.glassfish.grizzly.nio.transport.TCPNIOAsyncQueueWriter.writeCompositeRecord(TCPNIOAsyncQueueWriter.java:197) >> >> ~[graylog.jar:?] >> at >> org.glassfish.grizzly.nio.transport.TCPNIOAsyncQueueWriter.write0(TCPNIOAsyncQueueWriter.java:92) >> >> ~[graylog.jar:?] >> at >>
[graylog2] Re: Graylog IO Exception Error
After further investigation I think this was due to elasticsearch and graylog being overloaded. I have increased their heap sizes accordingly and will see how the system performs. Ariel On Wednesday, July 6, 2016 at 12:21:11 PM UTC-5, Ariel Godinez wrote: > > Hello, > > I've been using graylog for a couple weeks now and started to notice some > unusual behavior today. I am currently running a single node setup. > > The Issue: > > Every once in awhile I start to notice that that graylog is dragging quite > a bit (the loading spinner is persisting much longer than usual) so I go > check the logs and find the following error message. > > ERROR [ServerRuntime$Responder] An I/O error has occurred while writing a > response message entity to the container output stream. > org.glassfish.jersey.server.internal.process.MappableException: > java.io.IOException: Connection closed > at > org.glassfish.jersey.server.internal.MappableExceptionWrapperInterceptor.aroundWriteTo(MappableExceptionWrapperInterceptor.java:92) > > ~[graylog.jar:?] > at > org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:162) > > ~[graylog.jar:?] > at > org.glassfish.jersey.message.internal.MessageBodyFactory.writeTo(MessageBodyFactory.java:1130) > > ~[graylog.jar:?] > at > org.glassfish.jersey.server.ServerRuntime$Responder.writeResponse(ServerRuntime.java:711) > > [graylog.jar:?] > at > org.glassfish.jersey.server.ServerRuntime$Responder.processResponse(ServerRuntime.java:444) > > [graylog.jar:?] > at > org.glassfish.jersey.server.ServerRuntime$Responder.process(ServerRuntime.java:434) > > [graylog.jar:?] > at > org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:329) > [graylog.jar:?] > at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271) > [graylog.jar:?] > at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267) > [graylog.jar:?] > at org.glassfish.jersey.internal.Errors.process(Errors.java:315) > [graylog.jar:?] > at org.glassfish.jersey.internal.Errors.process(Errors.java:297) > [graylog.jar:?] > at org.glassfish.jersey.internal.Errors.process(Errors.java:267) > [graylog.jar:?] > at > org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317) > > [graylog.jar:?] > at > org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305) > [graylog.jar:?] > at > org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154) > > [graylog.jar:?] > at > org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer.service(GrizzlyHttpContainer.java:384) > > [graylog.jar:?] > at > org.glassfish.grizzly.http.server.HttpHandler$1.run(HttpHandler.java:224) > [graylog.jar:?] > at > com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176) > > [graylog.jar:?] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > > [?:1.8.0_91] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > > [?:1.8.0_91] > at java.lang.Thread.run(Thread.java:745) [?:1.8.0_91] > Caused by: java.io.IOException: Connection closed > at > org.glassfish.grizzly.asyncqueue.TaskQueue.onClose(TaskQueue.java:317) > ~[graylog.jar:?] > at > org.glassfish.grizzly.nio.AbstractNIOAsyncQueueWriter.onClose(AbstractNIOAsyncQueueWriter.java:501) > > ~[graylog.jar:?] > at > org.glassfish.grizzly.nio.transport.TCPNIOTransport.closeConnection(TCPNIOTransport.java:412) > > ~[graylog.jar:?] > at > org.glassfish.grizzly.nio.NIOConnection.doClose(NIOConnection.java:604) > ~[graylog.jar:?] > at > org.glassfish.grizzly.nio.NIOConnection$5.run(NIOConnection.java:570) > ~[graylog.jar:?] > at > org.glassfish.grizzly.nio.DefaultSelectorHandler.execute(DefaultSelectorHandler.java:235) > > ~[graylog.jar:?] > at > org.glassfish.grizzly.nio.NIOConnection.terminate0(NIOConnection.java:564) > ~[graylog.jar:?] > at > org.glassfish.grizzly.nio.transport.TCPNIOConnection.terminate0(TCPNIOConnection.java:291) > > ~[graylog.jar:?] > at > org.glassfish.grizzly.nio.transport.TCPNIOAsyncQueueWriter.writeCompositeRecord(TCPNIOAsyncQueueWriter.java:197) > > ~[graylog.jar:?] > at > org.glassfish.grizzly.nio.transport.TCPNIOAsyncQueueWriter.write0(TCPNIOAsyncQueueWriter.java:92) > > ~[graylog.jar:?] > at > org.glassfish.grizzly.nio.AbstractNIOAsyncQueueWriter.processAsync(AbstractNIOAsyncQueueWriter.java:344) > > ~[graylog.jar:?] > at > org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:107) > > ~[graylog.jar:?] > at >
