Hi Robert, I was not aware of this big change (I know it's my fault) but I am not sure if I agree with the rationale.
I read through the JIRA and it seems that this is mostly a convenience change that we dont need to copy jars and mess with the classloading that much. On the other hand if user jars can conflict with frontend/backend classes that can lead to very serious (and hard to fix) problems, especially in larger scale deployments. What do you think about this? Gyula Robert Metzger <rmetz...@apache.org> ezt írta (időpont: 2017. febr. 23., Cs, 22:10): > Mh. The user jar is put into every classpath. So the jobmanager / > taskmanagers are potentially affected by this as well. > Probably the data transfer between the TMs doesn't call the same methods > as the UI on the JobManager :) > > The simplest solution is to shade your netty in the user jar into a > different location. > > > On Thu, Feb 23, 2017 at 10:01 PM, Gyula Fóra <gyula.f...@gmail.com> wrote: > > Hi Robert, > It definitely explains the behaviour. > > This only applies to the frontend right? > If so what is the rationale behind it, and how should I handle the > dependency conflict? > > Thanks, > Gyula > > Robert Metzger <rmetz...@apache.org> ezt írta (időpont: 2017. febr. 23., > Cs, 21:44): > > Hi, > Since Flink 1.2 "per job yarn applications" (when you do "-m > yarn-cluster") include the job jar into the classpath as well. > Does this change explain the behavior? > > On Thu, Feb 23, 2017 at 4:59 PM, Gyula Fóra <gyf...@apache.org> wrote: > > Hi, > > I have a problem that the frontend somehow seems to have the user jar on > the classpath and it leads to a netty conflict: > > https://gist.github.com/gyfora/4ec2c8a8a6b33adb80d411460432ce8d > > So in the jobmanager logs I can see that my job started (running on YARN), > but can't access the frontend, it gives internal server error with the > previous exception. So I dont have the same jar problem on the actual > running job. > > I haven't really seen this before, is this something that happened to > somebody else as well? > > Thank you! > Gyula > > > >
