Yes pretty much - we use sbt to run the job in a local environment, not Intellij, but should be the same thing. We were also seeing that exception running unit tests locally. We did not see the exception when assembling a fat jar and submitting to a remote Flink cluster.
It seems like the flink-connector-elasticsearch jar should not have shaded classes in it. Maybe that jar in maven central was built incorrectly? We worked around this by just not depending on that elasticsearch connector at all, since we wrote our own connector for Elasticsearch 2.x. -Zach On Mon, Mar 14, 2016 at 2:03 PM Andrew Whitaker < andrew.whita...@braintreepayments.com> wrote: > We're having the same issue (we also have a dependency on > flink-connector-elasticsearch). It's only happening to us in IntelliJ > though. Is this the case for you as well? > > On Thu, Mar 10, 2016 at 3:20 PM, Zach Cox <zcox...@gmail.com> wrote: > >> After some poking around I noticed >> that flink-connector-elasticsearch_2.10-1.0.0.jar contains shaded asm >> classes. If I remove that dependency from my project then I do not get the >> IllegalArgumentException. >> >> >> On Thu, Mar 10, 2016 at 11:51 AM Zach Cox <zcox...@gmail.com> wrote: >> >>> Here are the jars on the classpath when I try to run our Flink job in a >>> local environment (via `sbt run`): >>> >>> >>> https://gist.githubusercontent.com/zcox/0992aba1c517b51dc879/raw/7136ec034c2beef04bd65de9f125ce3796db511f/gistfile1.txt >>> >>> There are many transitive dependencies pulled in from internal library >>> projects that probably need to be cleaned out. Maybe we are including >>> something that conflicts? Or maybe something important is being excluded? >>> >>> Are the asm classes included in Flink jars in some shaded form? >>> >>> Thanks, >>> Zach >>> >>> >>> On Thu, Mar 10, 2016 at 5:06 AM Stephan Ewen <se...@apache.org> wrote: >>> >>>> Dependency shading changed a bit between RC4 and RC5 - maybe a >>>> different minor ASM version is now included in the "test" scope. >>>> >>>> Can you share the dependencies of the problematic project? >>>> >>>> On Thu, Mar 10, 2016 at 12:26 AM, Zach Cox <zcox...@gmail.com> wrote: >>>> >>>>> I also noticed when I try to run this application in a local >>>>> environment, I get the same IllegalArgumentException. >>>>> >>>>> When I assemble this application into a fat jar and run it on a Flink >>>>> cluster using the CLI tools, it seems to run fine. >>>>> >>>>> Maybe my local classpath is missing something that is provided on the >>>>> Flink task managers? >>>>> >>>>> -Zach >>>>> >>>>> >>>>> On Wed, Mar 9, 2016 at 5:16 PM Zach Cox <zcox...@gmail.com> wrote: >>>>> >>>>>> Hi - after upgrading to 1.0.0, I'm getting this exception now in a >>>>>> unit test: >>>>>> >>>>>> IllegalArgumentException: (null:-1) >>>>>> org.apache.flink.shaded.org.objectweb.asm.ClassVisitor.<init>(Unknown >>>>>> Source) >>>>>> org.apache.flink.shaded.org.objectweb.asm.ClassVisitor.<init>(Unknown >>>>>> Source) >>>>>> >>>>>> org.apache.flink.api.scala.InnerClosureFinder.<init>(ClosureCleaner.scala:279) >>>>>> >>>>>> org.apache.flink.api.scala.ClosureCleaner$.getInnerClasses(ClosureCleaner.scala:95) >>>>>> >>>>>> org.apache.flink.api.scala.ClosureCleaner$.clean(ClosureCleaner.scala:115) >>>>>> >>>>>> org.apache.flink.streaming.api.scala.StreamExecutionEnvironment.scalaClean(StreamExecutionEnvironment.scala:568) >>>>>> >>>>>> org.apache.flink.streaming.api.scala.StreamExecutionEnvironment.addSource(StreamExecutionEnvironment.scala:498) >>>>>> >>>>>> The line that causes that exception is just adding >>>>>> a FlinkKafkaConsumer08 source. >>>>>> >>>>>> ClassVisitor [1] seems to throw that IllegalArgumentException when it >>>>>> is not given a valid api version number, but InnerClosureFinder [2] looks >>>>>> fine to me. >>>>>> >>>>>> Any idea what might be causing this? This unit test worked fine with >>>>>> 1.0.0-rc0 jars. >>>>>> >>>>>> Thanks, >>>>>> Zach >>>>>> >>>>>> [1] >>>>>> http://websvn.ow2.org/filedetails.php?repname=asm&path=%2Ftrunk%2Fasm%2Fsrc%2Forg%2Fobjectweb%2Fasm%2FClassVisitor.java >>>>>> [2] >>>>>> https://github.com/apache/flink/blob/master/flink-scala/src/main/scala/org/apache/flink/api/scala/ClosureCleaner.scala#L279 >>>>>> >>>>>> >>>>>> >>>> > > > -- > Andrew Whitaker | andrew.whita...@braintreepayments.com > -- > Note: this information is confidential. It is prohibited to share, post > online or otherwise publicize without Braintree's prior written consent. >
