Re: Elasticsink sometimes gives NoClassDefFoundError
I don't see a particular reason why you see this behavior. Chesnay's explanation is the only plausible way that this behavior can happen. I fear that without a specific log, we cannot help further. On Fri, Jan 10, 2020 at 5:06 AM Jayant Ameta wrote: > Also, the ES version I'm using is 5.6.7 > > Jayant > > > On Thu, Jan 9, 2020 at 10:39 AM Jayant Ameta wrote: > >> Hi, >> The elastic connector is packaged in the uber jar that is submitted. >> There is only 1 version of the connector: >> flink-connector-elasticsearch5_2.11:1.7.1 >> I'm using Flink 1.7.1 >> >> I couldn't figure out whether this error causes the job to fail, or >> whether I see this error when the job is restarting after some other >> failure. >> But, the occurrence of this error and job restarts is correlated. >> >> >> Jayant Ameta >> >> >> On Wed, Jan 8, 2020 at 6:47 PM Arvid Heise wrote: >> >>> Hi Jayant, >>> >>> if you only see it sometimes that indicates that you have it in two >>> different versions of the connectors where class loader order is >>> non-deterministic. Could you post the classpath? >>> >>> Btw, it's always good to add which Flink version you use. >>> >>> Best, >>> >>> Arvid >>> >>> On Wed, Jan 8, 2020 at 12:20 PM Jayant Ameta >>> wrote: >>> Hi, I see the following error sometimes on my flink job, even though the class is present in my uber jar. java.lang.NoClassDefFoundError: org/apache/flink/streaming/connectors/elasticsearch5/shaded/org/jboss/netty/channel/socket/nio/NioClientSocketPipelineSink$1 at org.apache.flink.streaming.connectors.elasticsearch5.shaded.org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink.connect(NioClientSocketPipelineSink.java:111) ... 17 common frames omitted Wrapped by: org.elasticsearch.ElasticsearchException: java.lang.NoClassDefFoundError: org/apache/flink/streaming/connectors/elasticsearch5/shaded/org/jboss/netty/channel/socket/nio/NioClientSocketPipelineSink$1 at org.elasticsearch.transport.netty3.Netty3Transport.exceptionCaught(Netty3Transport.java:325) at org.elasticsearch.transport.netty3.Netty3MessageChannelHandler.exceptionCaught(Netty3MessageChannelHandler.java:83) at org.apache.flink.streaming.connectors.elasticsearch5.shaded.org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:112) at org.apache.flink.streaming.connectors.elasticsearch5.shaded.org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) at org.apache.flink.streaming.connectors.elasticsearch5.shaded.org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) ... 23 frames truncated Jayant >>>
Re: Elasticsink sometimes gives NoClassDefFoundError
Also, the ES version I'm using is 5.6.7 Jayant On Thu, Jan 9, 2020 at 10:39 AM Jayant Ameta wrote: > Hi, > The elastic connector is packaged in the uber jar that is submitted. There > is only 1 version of the connector: > flink-connector-elasticsearch5_2.11:1.7.1 > I'm using Flink 1.7.1 > > I couldn't figure out whether this error causes the job to fail, or > whether I see this error when the job is restarting after some other > failure. > But, the occurrence of this error and job restarts is correlated. > > > Jayant Ameta > > > On Wed, Jan 8, 2020 at 6:47 PM Arvid Heise wrote: > >> Hi Jayant, >> >> if you only see it sometimes that indicates that you have it in two >> different versions of the connectors where class loader order is >> non-deterministic. Could you post the classpath? >> >> Btw, it's always good to add which Flink version you use. >> >> Best, >> >> Arvid >> >> On Wed, Jan 8, 2020 at 12:20 PM Jayant Ameta >> wrote: >> >>> Hi, >>> I see the following error sometimes on my flink job, even though the >>> class is present in my uber jar. >>> >>> java.lang.NoClassDefFoundError: >>> org/apache/flink/streaming/connectors/elasticsearch5/shaded/org/jboss/netty/channel/socket/nio/NioClientSocketPipelineSink$1 >>> at >>> org.apache.flink.streaming.connectors.elasticsearch5.shaded.org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink.connect(NioClientSocketPipelineSink.java:111) >>> ... 17 common frames omitted Wrapped by: >>> org.elasticsearch.ElasticsearchException: java.lang.NoClassDefFoundError: >>> org/apache/flink/streaming/connectors/elasticsearch5/shaded/org/jboss/netty/channel/socket/nio/NioClientSocketPipelineSink$1 >>> at >>> org.elasticsearch.transport.netty3.Netty3Transport.exceptionCaught(Netty3Transport.java:325) >>> at >>> org.elasticsearch.transport.netty3.Netty3MessageChannelHandler.exceptionCaught(Netty3MessageChannelHandler.java:83) >>> at >>> org.apache.flink.streaming.connectors.elasticsearch5.shaded.org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:112) >>> at >>> org.apache.flink.streaming.connectors.elasticsearch5.shaded.org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) >>> at >>> org.apache.flink.streaming.connectors.elasticsearch5.shaded.org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) >>> ... 23 frames truncated >>> >>> >>> Jayant >>> >>
Re: Elasticsink sometimes gives NoClassDefFoundError
Hi, The elastic connector is packaged in the uber jar that is submitted. There is only 1 version of the connector: flink-connector-elasticsearch5_2.11:1.7.1 I'm using Flink 1.7.1 I couldn't figure out whether this error causes the job to fail, or whether I see this error when the job is restarting after some other failure. But, the occurrence of this error and job restarts is correlated. Jayant Ameta On Wed, Jan 8, 2020 at 6:47 PM Arvid Heise wrote: > Hi Jayant, > > if you only see it sometimes that indicates that you have it in two > different versions of the connectors where class loader order is > non-deterministic. Could you post the classpath? > > Btw, it's always good to add which Flink version you use. > > Best, > > Arvid > > On Wed, Jan 8, 2020 at 12:20 PM Jayant Ameta wrote: > >> Hi, >> I see the following error sometimes on my flink job, even though the >> class is present in my uber jar. >> >> java.lang.NoClassDefFoundError: >> org/apache/flink/streaming/connectors/elasticsearch5/shaded/org/jboss/netty/channel/socket/nio/NioClientSocketPipelineSink$1 >> at >> org.apache.flink.streaming.connectors.elasticsearch5.shaded.org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink.connect(NioClientSocketPipelineSink.java:111) >> ... 17 common frames omitted Wrapped by: >> org.elasticsearch.ElasticsearchException: java.lang.NoClassDefFoundError: >> org/apache/flink/streaming/connectors/elasticsearch5/shaded/org/jboss/netty/channel/socket/nio/NioClientSocketPipelineSink$1 >> at >> org.elasticsearch.transport.netty3.Netty3Transport.exceptionCaught(Netty3Transport.java:325) >> at >> org.elasticsearch.transport.netty3.Netty3MessageChannelHandler.exceptionCaught(Netty3MessageChannelHandler.java:83) >> at >> org.apache.flink.streaming.connectors.elasticsearch5.shaded.org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:112) >> at >> org.apache.flink.streaming.connectors.elasticsearch5.shaded.org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) >> at >> org.apache.flink.streaming.connectors.elasticsearch5.shaded.org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) >> ... 23 frames truncated >> >> >> Jayant >> >
Re: Elasticsink sometimes gives NoClassDefFoundError
Hi Jayant, if you only see it sometimes that indicates that you have it in two different versions of the connectors where class loader order is non-deterministic. Could you post the classpath? Btw, it's always good to add which Flink version you use. Best, Arvid On Wed, Jan 8, 2020 at 12:20 PM Jayant Ameta wrote: > Hi, > I see the following error sometimes on my flink job, even though the class > is present in my uber jar. > > java.lang.NoClassDefFoundError: > org/apache/flink/streaming/connectors/elasticsearch5/shaded/org/jboss/netty/channel/socket/nio/NioClientSocketPipelineSink$1 > at > org.apache.flink.streaming.connectors.elasticsearch5.shaded.org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink.connect(NioClientSocketPipelineSink.java:111) > ... 17 common frames omitted Wrapped by: > org.elasticsearch.ElasticsearchException: java.lang.NoClassDefFoundError: > org/apache/flink/streaming/connectors/elasticsearch5/shaded/org/jboss/netty/channel/socket/nio/NioClientSocketPipelineSink$1 > at > org.elasticsearch.transport.netty3.Netty3Transport.exceptionCaught(Netty3Transport.java:325) > at > org.elasticsearch.transport.netty3.Netty3MessageChannelHandler.exceptionCaught(Netty3MessageChannelHandler.java:83) > at > org.apache.flink.streaming.connectors.elasticsearch5.shaded.org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:112) > at > org.apache.flink.streaming.connectors.elasticsearch5.shaded.org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) > at > org.apache.flink.streaming.connectors.elasticsearch5.shaded.org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) > ... 23 frames truncated > > > Jayant >
Re: Elasticsink sometimes gives NoClassDefFoundError
Could you clarify under what circumstances you see this issue? You say "sometimes"; is the job running normally and then failing due to this error? Does it happen when submitting/canceling/restarting a job etc. On 08/01/2020 12:20, Jayant Ameta wrote: Hi, I see the following error sometimes on my flink job, even though the class is present in my uber jar. java.lang.NoClassDefFoundError: org/apache/flink/streaming/connectors/elasticsearch5/shaded/org/jboss/netty/channel/socket/nio/NioClientSocketPipelineSink$1 at org.apache.flink.streaming.connectors.elasticsearch5.shaded.org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink.connect(NioClientSocketPipelineSink.java:111) ... 17 common frames omitted Wrapped by: org.elasticsearch.ElasticsearchException: java.lang.NoClassDefFoundError: org/apache/flink/streaming/connectors/elasticsearch5/shaded/org/jboss/netty/channel/socket/nio/NioClientSocketPipelineSink$1 at org.elasticsearch.transport.netty3.Netty3Transport.exceptionCaught(Netty3Transport.java:325) at org.elasticsearch.transport.netty3.Netty3MessageChannelHandler.exceptionCaught(Netty3MessageChannelHandler.java:83) at org.apache.flink.streaming.connectors.elasticsearch5.shaded.org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:112) at org.apache.flink.streaming.connectors.elasticsearch5.shaded.org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) at org.apache.flink.streaming.connectors.elasticsearch5.shaded.org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) ... 23 frames truncated Jayant
Elasticsink sometimes gives NoClassDefFoundError
Hi, I see the following error sometimes on my flink job, even though the class is present in my uber jar. java.lang.NoClassDefFoundError: org/apache/flink/streaming/connectors/elasticsearch5/shaded/org/jboss/netty/channel/socket/nio/NioClientSocketPipelineSink$1 at org.apache.flink.streaming.connectors.elasticsearch5.shaded.org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink.connect(NioClientSocketPipelineSink.java:111) ... 17 common frames omitted Wrapped by: org.elasticsearch.ElasticsearchException: java.lang.NoClassDefFoundError: org/apache/flink/streaming/connectors/elasticsearch5/shaded/org/jboss/netty/channel/socket/nio/NioClientSocketPipelineSink$1 at org.elasticsearch.transport.netty3.Netty3Transport.exceptionCaught(Netty3Transport.java:325) at org.elasticsearch.transport.netty3.Netty3MessageChannelHandler.exceptionCaught(Netty3MessageChannelHandler.java:83) at org.apache.flink.streaming.connectors.elasticsearch5.shaded.org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:112) at org.apache.flink.streaming.connectors.elasticsearch5.shaded.org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) at org.apache.flink.streaming.connectors.elasticsearch5.shaded.org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) ... 23 frames truncated Jayant
