[ https://issues.apache.org/jira/browse/FLUME-3021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15932298#comment-15932298 ]
tycho_yang edited comment on FLUME-3021 at 3/20/17 8:29 AM: ------------------------------------------------------------ Use your binary package, not stable enough, 70000 data transmission through continuous testing, found that there are about 2 times in the exception of the 5, the same configuration of the same data, configuration data with no problem this can be determined before the same configuration in ES1.7 run OK at java.lang.Thread.run(Thread.java:745) 2017-03-20 15:08:32,799 (elasticsearch[_client_][generic][T#1]) [ERROR - org.elasticsearch.transport.netty4.Netty4InternalESLogger.error(Netty4InternalESLogger.java:184)] Failed to submit a listener notification task. Event loop shut down? java.util.concurrent.RejectedExecutionException: event executor terminated at io.netty.util.concurrent.SingleThreadEventExecutor.reject(SingleThreadEventExecutor.java:821) at io.netty.util.concurrent.SingleThreadEventExecutor.offerTask(SingleThreadEventExecutor.java:327) at io.netty.util.concurrent.SingleThreadEventExecutor.addTask(SingleThreadEventExecutor.java:320) at io.netty.util.concurrent.SingleThreadEventExecutor.execute(SingleThreadEventExecutor.java:746) at io.netty.util.concurrent.DefaultPromise.safeExecute(DefaultPromise.java:760) at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:428) at io.netty.util.concurrent.DefaultPromise.setFailure(DefaultPromise.java:113) at io.netty.channel.DefaultChannelPromise.setFailure(DefaultChannelPromise.java:87) at io.netty.channel.AbstractChannelHandlerContext.safeExecute(AbstractChannelHandlerContext.java:1011) at io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:826) at io.netty.channel.AbstractChannelHandlerContext.writeAndFlush(AbstractChannelHandlerContext.java:795) at io.netty.channel.AbstractChannelHandlerContext.writeAndFlush(AbstractChannelHandlerContext.java:832) at io.netty.channel.DefaultChannelPipeline.writeAndFlush(DefaultChannelPipeline.java:1032) at io.netty.channel.AbstractChannel.writeAndFlush(AbstractChannel.java:296) at org.elasticsearch.transport.netty4.Netty4Transport.sendMessage(Netty4Transport.java:422) at org.elasticsearch.transport.netty4.Netty4Transport.sendMessage(Netty4Transport.java:93) at org.elasticsearch.transport.TcpTransport$ScheduledPing.doRunInLifecycle(TcpTransport.java:315) at org.elasticsearch.common.util.concurrent.AbstractLifecycleRunnable.doRun(AbstractLifecycleRunnable.java:67) at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:596) at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 2017-03-20 15:08:32,871 (elasticsearch[_client_][generic][T#1]) [ERROR - org.elasticsearch.transport.netty4.Netty4InternalESLogger.error(Netty4InternalESLogger.java:184)] Failed to submit a listener notification task. Event loop shut down? java.util.concurrent.RejectedExecutionException: event executor terminated at io.netty.util.concurrent.SingleThreadEventExecutor.reject(SingleThreadEventExecutor.java:821) at io.netty.util.concurrent.SingleThreadEventExecutor.offerTask(SingleThreadEventExecutor.java:327) at io.netty.util.concurrent.SingleThreadEventExecutor.addTask(SingleThreadEventExecutor.java:320) at io.netty.util.concurrent.SingleThreadEventExecutor.execute(SingleThreadEventExecutor.java:746) at io.netty.util.concurrent.DefaultPromise.safeExecute(DefaultPromise.java:760) at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:428) at io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:163) at io.netty.channel.DefaultChannelPromise.addListener(DefaultChannelPromise.java:93) at io.netty.channel.DefaultChannelPromise.addListener(DefaultChannelPromise.java:28) at org.elasticsearch.transport.netty4.Netty4Transport.sendMessage(Netty4Transport.java:423) at org.elasticsearch.transport.netty4.Netty4Transport.sendMessage(Netty4Transport.java:93) at org.elasticsearch.transport.TcpTransport$ScheduledPing.doRunInLifecycle(TcpTransport.java:315) at org.elasticsearch.common.util.concurrent.AbstractLifecycleRunnable.doRun(AbstractLifecycleRunnable.java:67) at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:596) at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) ========================================================================== The same test error 2017-03-20 16:05:14,900 (SinkRunner-PollingRunner-DefaultSinkProcessor) [ERROR - org.apache.flume.sink.elasticsearch.ElasticSearchSink.process(ElasticSearchSink.java:221)] Failed to commit transaction. Transaction rolled back. NoNodeAvailableException[None of the configured nodes were available: [{node-1}{Xb_O5doNTMuS2u4pbC4ILQ}{O4cUzwIPQ9uVMWkygbwhtw}{192.168.3.63}{192.168.3.63:9300}]]; nested: NodeNotConnectedException[[node-1][192.168.3.63:9300] Node not connected]; at org.elasticsearch.client.transport.TransportClientNodesService$RetryListener.onFailure(TransportClientNodesService.java:288) at org.elasticsearch.action.ActionListenerResponseHandler.handleException(ActionListenerResponseHandler.java:51) at org.elasticsearch.transport.TransportService.sendRequest(TransportService.java:476) at org.elasticsearch.action.TransportActionNodeProxy.execute(TransportActionNodeProxy.java:51) at org.elasticsearch.client.transport.TransportProxyClient.lambda$execute$0(TransportProxyClient.java:59) at org.elasticsearch.client.transport.TransportClientNodesService.execute(TransportClientNodesService.java:247) at org.elasticsearch.client.transport.TransportProxyClient.execute(TransportProxyClient.java:59) at org.elasticsearch.client.transport.TransportClient.doExecute(TransportClient.java:356) at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:403) at org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:80) at org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:54) at org.apache.flume.sink.elasticsearch.client.ElasticSearchTransportClient.execute(ElasticSearchTransportClient.java:184) at org.apache.flume.sink.elasticsearch.ElasticSearchSink.process(ElasticSearchSink.java:205) at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:67) at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:145) at java.lang.Thread.run(Thread.java:745) Caused by: NodeNotConnectedException[[node-1][192.168.3.63:9300] Node not connected] at org.elasticsearch.transport.TcpTransport.getConnection(TcpTransport.java:590) at org.elasticsearch.transport.TcpTransport.getConnection(TcpTransport.java:113) at org.elasticsearch.transport.TransportService.getConnection(TransportService.java:496) at org.elasticsearch.transport.TransportService.sendRequest(TransportService.java:472) ... 13 more 2017-03-20 16:05:14,954 (SinkRunner-PollingRunner-DefaultSinkProcessor) [ERROR - org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:158)] Unable to deliver event. Exception follows. NoNodeAvailableException[None of the configured nodes were available: [{node-1}{Xb_O5doNTMuS2u4pbC4ILQ}{O4cUzwIPQ9uVMWkygbwhtw}{192.168.3.63}{192.168.3.63:9300}]]; nested: NodeNotConnectedException[[node-1][192.168.3.63:9300] Node not connected]; at org.elasticsearch.client.transport.TransportClientNodesService$RetryListener.onFailure(TransportClientNodesService.java:288) at org.elasticsearch.action.ActionListenerResponseHandler.handleException(ActionListenerResponseHandler.java:51) at org.elasticsearch.transport.TransportService.sendRequest(TransportService.java:476) at org.elasticsearch.action.TransportActionNodeProxy.execute(TransportActionNodeProxy.java:51) at org.elasticsearch.client.transport.TransportProxyClient.lambda$execute$0(TransportProxyClient.java:59) at org.elasticsearch.client.transport.TransportClientNodesService.execute(TransportClientNodesService.java:247) at org.elasticsearch.client.transport.TransportProxyClient.execute(TransportProxyClient.java:59) at org.elasticsearch.client.transport.TransportClient.doExecute(TransportClient.java:356) at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:403) at org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:80) at org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:54) at org.apache.flume.sink.elasticsearch.client.ElasticSearchTransportClient.execute(ElasticSearchTransportClient.java:184) at org.apache.flume.sink.elasticsearch.ElasticSearchSink.process(ElasticSearchSink.java:205) at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:67) at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:145) at java.lang.Thread.run(Thread.java:745) Caused by: NodeNotConnectedException[[node-1][192.168.3.63:9300] Node not connected] at org.elasticsearch.transport.TcpTransport.getConnection(TcpTransport.java:590) at org.elasticsearch.transport.TcpTransport.getConnection(TcpTransport.java:113) at org.elasticsearch.transport.TransportService.getConnection(TransportService.java:496) at org.elasticsearch.transport.TransportService.sendRequest(TransportService.java:472) ... 13 more was (Author: ytc301): Use your binary package, not stable enough, 70000 data transmission through continuous testing, found that there are about 2 times in the exception of the 5, the same configuration of the same data, configuration data with no problem this can be determined before the same configuration in ES1.7 run OK at java.lang.Thread.run(Thread.java:745) 2017-03-20 15:08:32,799 (elasticsearch[_client_][generic][T#1]) [ERROR - org.elasticsearch.transport.netty4.Netty4InternalESLogger.error(Netty4InternalESLogger.java:184)] Failed to submit a listener notification task. Event loop shut down? java.util.concurrent.RejectedExecutionException: event executor terminated at io.netty.util.concurrent.SingleThreadEventExecutor.reject(SingleThreadEventExecutor.java:821) at io.netty.util.concurrent.SingleThreadEventExecutor.offerTask(SingleThreadEventExecutor.java:327) at io.netty.util.concurrent.SingleThreadEventExecutor.addTask(SingleThreadEventExecutor.java:320) at io.netty.util.concurrent.SingleThreadEventExecutor.execute(SingleThreadEventExecutor.java:746) at io.netty.util.concurrent.DefaultPromise.safeExecute(DefaultPromise.java:760) at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:428) at io.netty.util.concurrent.DefaultPromise.setFailure(DefaultPromise.java:113) at io.netty.channel.DefaultChannelPromise.setFailure(DefaultChannelPromise.java:87) at io.netty.channel.AbstractChannelHandlerContext.safeExecute(AbstractChannelHandlerContext.java:1011) at io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:826) at io.netty.channel.AbstractChannelHandlerContext.writeAndFlush(AbstractChannelHandlerContext.java:795) at io.netty.channel.AbstractChannelHandlerContext.writeAndFlush(AbstractChannelHandlerContext.java:832) at io.netty.channel.DefaultChannelPipeline.writeAndFlush(DefaultChannelPipeline.java:1032) at io.netty.channel.AbstractChannel.writeAndFlush(AbstractChannel.java:296) at org.elasticsearch.transport.netty4.Netty4Transport.sendMessage(Netty4Transport.java:422) at org.elasticsearch.transport.netty4.Netty4Transport.sendMessage(Netty4Transport.java:93) at org.elasticsearch.transport.TcpTransport$ScheduledPing.doRunInLifecycle(TcpTransport.java:315) at org.elasticsearch.common.util.concurrent.AbstractLifecycleRunnable.doRun(AbstractLifecycleRunnable.java:67) at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:596) at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 2017-03-20 15:08:32,871 (elasticsearch[_client_][generic][T#1]) [ERROR - org.elasticsearch.transport.netty4.Netty4InternalESLogger.error(Netty4InternalESLogger.java:184)] Failed to submit a listener notification task. Event loop shut down? java.util.concurrent.RejectedExecutionException: event executor terminated at io.netty.util.concurrent.SingleThreadEventExecutor.reject(SingleThreadEventExecutor.java:821) at io.netty.util.concurrent.SingleThreadEventExecutor.offerTask(SingleThreadEventExecutor.java:327) at io.netty.util.concurrent.SingleThreadEventExecutor.addTask(SingleThreadEventExecutor.java:320) at io.netty.util.concurrent.SingleThreadEventExecutor.execute(SingleThreadEventExecutor.java:746) at io.netty.util.concurrent.DefaultPromise.safeExecute(DefaultPromise.java:760) at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:428) at io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:163) at io.netty.channel.DefaultChannelPromise.addListener(DefaultChannelPromise.java:93) at io.netty.channel.DefaultChannelPromise.addListener(DefaultChannelPromise.java:28) at org.elasticsearch.transport.netty4.Netty4Transport.sendMessage(Netty4Transport.java:423) at org.elasticsearch.transport.netty4.Netty4Transport.sendMessage(Netty4Transport.java:93) at org.elasticsearch.transport.TcpTransport$ScheduledPing.doRunInLifecycle(TcpTransport.java:315) at org.elasticsearch.common.util.concurrent.AbstractLifecycleRunnable.doRun(AbstractLifecycleRunnable.java:67) at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:596) at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) The same test error 2017-03-20 16:05:14,900 (SinkRunner-PollingRunner-DefaultSinkProcessor) [ERROR - org.apache.flume.sink.elasticsearch.ElasticSearchSink.process(ElasticSearchSink.java:221)] Failed to commit transaction. Transaction rolled back. NoNodeAvailableException[None of the configured nodes were available: [{node-1}{Xb_O5doNTMuS2u4pbC4ILQ}{O4cUzwIPQ9uVMWkygbwhtw}{192.168.3.63}{192.168.3.63:9300}]]; nested: NodeNotConnectedException[[node-1][192.168.3.63:9300] Node not connected]; at org.elasticsearch.client.transport.TransportClientNodesService$RetryListener.onFailure(TransportClientNodesService.java:288) at org.elasticsearch.action.ActionListenerResponseHandler.handleException(ActionListenerResponseHandler.java:51) at org.elasticsearch.transport.TransportService.sendRequest(TransportService.java:476) at org.elasticsearch.action.TransportActionNodeProxy.execute(TransportActionNodeProxy.java:51) at org.elasticsearch.client.transport.TransportProxyClient.lambda$execute$0(TransportProxyClient.java:59) at org.elasticsearch.client.transport.TransportClientNodesService.execute(TransportClientNodesService.java:247) at org.elasticsearch.client.transport.TransportProxyClient.execute(TransportProxyClient.java:59) at org.elasticsearch.client.transport.TransportClient.doExecute(TransportClient.java:356) at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:403) at org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:80) at org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:54) at org.apache.flume.sink.elasticsearch.client.ElasticSearchTransportClient.execute(ElasticSearchTransportClient.java:184) at org.apache.flume.sink.elasticsearch.ElasticSearchSink.process(ElasticSearchSink.java:205) at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:67) at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:145) at java.lang.Thread.run(Thread.java:745) Caused by: NodeNotConnectedException[[node-1][192.168.3.63:9300] Node not connected] at org.elasticsearch.transport.TcpTransport.getConnection(TcpTransport.java:590) at org.elasticsearch.transport.TcpTransport.getConnection(TcpTransport.java:113) at org.elasticsearch.transport.TransportService.getConnection(TransportService.java:496) at org.elasticsearch.transport.TransportService.sendRequest(TransportService.java:472) ... 13 more 2017-03-20 16:05:14,954 (SinkRunner-PollingRunner-DefaultSinkProcessor) [ERROR - org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:158)] Unable to deliver event. Exception follows. NoNodeAvailableException[None of the configured nodes were available: [{node-1}{Xb_O5doNTMuS2u4pbC4ILQ}{O4cUzwIPQ9uVMWkygbwhtw}{192.168.3.63}{192.168.3.63:9300}]]; nested: NodeNotConnectedException[[node-1][192.168.3.63:9300] Node not connected]; at org.elasticsearch.client.transport.TransportClientNodesService$RetryListener.onFailure(TransportClientNodesService.java:288) at org.elasticsearch.action.ActionListenerResponseHandler.handleException(ActionListenerResponseHandler.java:51) at org.elasticsearch.transport.TransportService.sendRequest(TransportService.java:476) at org.elasticsearch.action.TransportActionNodeProxy.execute(TransportActionNodeProxy.java:51) at org.elasticsearch.client.transport.TransportProxyClient.lambda$execute$0(TransportProxyClient.java:59) at org.elasticsearch.client.transport.TransportClientNodesService.execute(TransportClientNodesService.java:247) at org.elasticsearch.client.transport.TransportProxyClient.execute(TransportProxyClient.java:59) at org.elasticsearch.client.transport.TransportClient.doExecute(TransportClient.java:356) at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:403) at org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:80) at org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:54) at org.apache.flume.sink.elasticsearch.client.ElasticSearchTransportClient.execute(ElasticSearchTransportClient.java:184) at org.apache.flume.sink.elasticsearch.ElasticSearchSink.process(ElasticSearchSink.java:205) at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:67) at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:145) at java.lang.Thread.run(Thread.java:745) Caused by: NodeNotConnectedException[[node-1][192.168.3.63:9300] Node not connected] at org.elasticsearch.transport.TcpTransport.getConnection(TcpTransport.java:590) at org.elasticsearch.transport.TcpTransport.getConnection(TcpTransport.java:113) at org.elasticsearch.transport.TransportService.getConnection(TransportService.java:496) at org.elasticsearch.transport.TransportService.sendRequest(TransportService.java:472) ... 13 more > flume Elasticsearch 5.0 not support > ----------------------------------- > > Key: FLUME-3021 > URL: https://issues.apache.org/jira/browse/FLUME-3021 > Project: Flume > Issue Type: Bug > Components: Sinks+Sources > Affects Versions: 1.7.0 > Reporter: tycho_yang > Assignee: Yonghao Zou > Fix For: 1.8.0 > > Attachments: FLUME-3021-0.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)