Re: Re: Re: Re: Flink and Akka version incompatibility ..
Thanks for the pointer .. yes, we need this .. On Thu, Jul 25, 2019 at 9:08 AM Haibo Sun wrote: > > The following JIRA is about the problem you encounter. I think you should > be very interested in its comments.There does seem to be a problem with > shading Akka, and Flink is considering isolating the classloader that > contain Akka and Scala to allow the applications and Flink to use different > Akka versions. > > https://issues.apache.org/jira/browse/FLINK-10903 > > Best, > Haibo > > At 2019-07-25 00:07:27, "Debasish Ghosh" wrote: > > Also wanted to check if anyone has ventured into this exercise of shading > Akka in Flink .. > Is this something that qualifies as one of the roadmap items in Flink ? > > regards. > > On Wed, Jul 24, 2019 at 3:44 PM Debasish Ghosh > wrote: > >> Hi Haibo - Thanks for the clarification .. >> >> regards. >> >> On Wed, Jul 24, 2019 at 2:58 PM Haibo Sun wrote: >> >>> Hi Debasish Ghosh, >>> >>> I agree that Flink should shade its Akka. >>> >>> Maybe you misunderstood me. I mean, in the absence of official shading >>> Akka in Flink, the relatively conservative way is to shade Akka of your >>> application (I concern Flink won't work well after shading its Akka). >>> >>> Best, >>> Haibo >>> >>> At 2019-07-24 16:43:28, "Debasish Ghosh" >>> wrote: >>> >>> For our application users are expected to work with Akka APIs - hence if >>> I shade Akka in my application users will need to work with shaded imports >>> which feels unnatural. With Flink, Akka is an implementation detail and >>> Flink users are not expected to use Akka APIs. Hence shading will not have >>> any user level impact. >>> >>> Hence the suggestion to shade Akka in Flink rather than the user >>> application. >>> >>> regards. >>> >>> On Wed, 24 Jul 2019 at 2:04 PM, Jeff Zhang wrote: >>> I think it is better to shade all the dependencies of flink so that all the projects that use flink won't hit this kind of issue. Haibo Sun 于2019年7月24日周三 下午4:07写道： > Hi, Debasish Ghosh > > I don't know why not shade Akka, maybe it can be shaded. Chesnay may > be able to answer that. > I recommend to shade Akka dependency of your application because it > don't be known what's wrong with shading Flink's Akka. > > CC @Chesnay Schepler > > Best, > Haibo > > At 2019-07-24 15:48:59, "Debasish Ghosh" > wrote: > > The problem that I am facing is with Akka serialization .. Why not > shade the whole of Akka ? > > java.lang.AbstractMethodError: >> akka.remote.RemoteActorRefProvider.serializationInformation()Lakka/serialization/Serialization$Information; >> at >> akka.serialization.Serialization.serializationInformation(Serialization.scala:166) > > > Akka 2.6 is just around the corner and I don't think Flink will > upgrade to Akka 2.6 that soon .. so somehow this problem is bound to recur > .. > > regards. > > On Wed, Jul 24, 2019 at 1:01 PM Zili Chen > wrote: > >> I can see that we relocate akka's netty, akka uncommon math but also >> be curious why Flink doesn't shaded all of akka dependencies... >> >> Best, >> tison. >> >> >> Debasish Ghosh 于2019年7月24日周三 下午3:15写道： >> >>> Hello Haibo - >>> >>> Yes, my application depends on Akka 2.5. >>> Just curious, why do you think it's recommended to shade Akka >>> version of my application instead of Flink ? >>> >>> regards. >>> >>> On Wed, Jul 24, 2019 at 12:42 PM Haibo Sun >>> wrote: >>> Hi Debasish Ghosh, Does your application have to depend on Akka 2.5? If not, it's a good idea to always keep the Akka version that the application depend on in line with Flink. If you want to try shading Akka dependency, I think that it is more recommended to shade Akka dependency of your application. Best, Haibo At 2019-07-24 14:31:29, "Debasish Ghosh" wrote: Hello - An application that uses Akka 2.5 and Flink 1.8.0 gives runtime errors because of version mismatch between Akka that we use and the one that Flink uses (which is Akka 2.4). Anyone tried shading Akka dependency with Flink ? Or is there any other alternative way to handle this issue ? I know Flink 1.9 has upgraded to Akka 2.5 but this is (I think) going to be a recurring problem down the line with mismatch between the new releases of Akka and Flink. regards. -- Debasish Ghosh http://manning.com/ghosh2 http://manning.com/ghosh Twttr: @debasishg Blog: http://debasishg.blogspot.com Code: http://github.com/debasishg >>> >>> -- >>>
Re:Re: Re: Re: Flink and Akka version incompatibility ..
The following JIRA is about the problem you encounter. I think you should be very interested in its comments.There does seem to be a problem with shading Akka, and Flink is considering isolating the classloader that contain Akka and Scala to allow the applications and Flink to use different Akka versions. https://issues.apache.org/jira/browse/FLINK-10903 Best, Haibo At 2019-07-25 00:07:27, "Debasish Ghosh" wrote: Also wanted to check if anyone has ventured into this exercise of shading Akka in Flink .. Is this something that qualifies as one of the roadmap items in Flink ? regards. On Wed, Jul 24, 2019 at 3:44 PM Debasish Ghosh wrote: Hi Haibo - Thanks for the clarification .. regards. On Wed, Jul 24, 2019 at 2:58 PM Haibo Sun wrote: Hi Debasish Ghosh, I agree that Flink should shade its Akka. Maybe you misunderstood me. I mean, in the absence of official shading Akka in Flink, the relatively conservative way is to shade Akka of your application (I concern Flink won't work well after shading its Akka). Best, Haibo At 2019-07-24 16:43:28, "Debasish Ghosh" wrote: For our application users are expected to work with Akka APIs - hence if I shade Akka in my application users will need to work with shaded imports which feels unnatural. With Flink, Akka is an implementation detail and Flink users are not expected to use Akka APIs. Hence shading will not have any user level impact. Hence the suggestion to shade Akka in Flink rather than the user application. regards. On Wed, 24 Jul 2019 at 2:04 PM, Jeff Zhang wrote: I think it is better to shade all the dependencies of flink so that all the projects that use flink won't hit this kind of issue. Haibo Sun 于2019年7月24日周三 下午4:07写道： Hi, Debasish Ghosh I don't know why not shade Akka, maybe it can be shaded. Chesnay may be able to answer that. I recommend to shade Akka dependency of your application because it don't be known what's wrong with shading Flink's Akka. CC @Chesnay Schepler Best, Haibo At 2019-07-24 15:48:59, "Debasish Ghosh" wrote: The problem that I am facing is with Akka serialization .. Why not shade the whole of Akka ? java.lang.AbstractMethodError: akka.remote.RemoteActorRefProvider.serializationInformation()Lakka/serialization/Serialization$Information; at akka.serialization.Serialization.serializationInformation(Serialization.scala:166) Akka 2.6 is just around the corner and I don't think Flink will upgrade to Akka 2.6 that soon .. so somehow this problem is bound to recur .. regards. On Wed, Jul 24, 2019 at 1:01 PM Zili Chen wrote: I can see that we relocate akka's netty, akka uncommon math but also be curious why Flink doesn't shaded all of akka dependencies... Best, tison. Debasish Ghosh 于2019年7月24日周三 下午3:15写道： Hello Haibo - Yes, my application depends on Akka 2.5. Just curious, why do you think it's recommended to shade Akka version of my application instead of Flink ? regards. On Wed, Jul 24, 2019 at 12:42 PM Haibo Sun wrote: Hi Debasish Ghosh, Does your application have to depend on Akka 2.5? If not, it's a good idea to always keep the Akka version that the application depend on in line with Flink. If you want to try shading Akka dependency, I think that it is more recommended to shade Akka dependency of your application. Best, Haibo At 2019-07-24 14:31:29, "Debasish Ghosh" wrote: Hello - An application that uses Akka 2.5 and Flink 1.8.0 gives runtime errors because of version mismatch between Akka that we use and the one that Flink uses (which is Akka 2.4). Anyone tried shading Akka dependency with Flink ? Or is there any other alternative way to handle this issue ? I know Flink 1.9 has upgraded to Akka 2.5 but this is (I think) going to be a recurring problem down the line with mismatch between the new releases of Akka and Flink. regards. -- Debasish Ghosh http://manning.com/ghosh2 http://manning.com/ghosh Twttr: @debasishg Blog: http://debasishg.blogspot.com Code: http://github.com/debasishg -- Debasish Ghosh http://manning.com/ghosh2 http://manning.com/ghosh Twttr: @debasishg Blog: http://debasishg.blogspot.com Code: http://github.com/debasishg -- Debasish Ghosh http://manning.com/ghosh2 http://manning.com/ghosh Twttr: @debasishg Blog: http://debasishg.blogspot.com Code: http://github.com/debasishg -- Best Regards Jeff Zhang -- Sent from my iPhone -- Debasish Ghosh http://manning.com/ghosh2 http://manning.com/ghosh Twttr: @debasishg Blog: http://debasishg.blogspot.com Code: http://github.com/debasishg -- Debasish Ghosh http://manning.com/ghosh2 http://manning.com/ghosh Twttr: @debasishg Blog: http://debasishg.blogspot.com Code: http://github.com/debasishg
Re: Re: Re: Flink and Akka version incompatibility ..
Also wanted to check if anyone has ventured into this exercise of shading Akka in Flink .. Is this something that qualifies as one of the roadmap items in Flink ? regards. On Wed, Jul 24, 2019 at 3:44 PM Debasish Ghosh wrote: > Hi Haibo - Thanks for the clarification .. > > regards. > > On Wed, Jul 24, 2019 at 2:58 PM Haibo Sun wrote: > >> Hi Debasish Ghosh, >> >> I agree that Flink should shade its Akka. >> >> Maybe you misunderstood me. I mean, in the absence of official shading >> Akka in Flink, the relatively conservative way is to shade Akka of your >> application (I concern Flink won't work well after shading its Akka). >> >> Best, >> Haibo >> >> At 2019-07-24 16:43:28, "Debasish Ghosh" >> wrote: >> >> For our application users are expected to work with Akka APIs - hence if >> I shade Akka in my application users will need to work with shaded imports >> which feels unnatural. With Flink, Akka is an implementation detail and >> Flink users are not expected to use Akka APIs. Hence shading will not have >> any user level impact. >> >> Hence the suggestion to shade Akka in Flink rather than the user >> application. >> >> regards. >> >> On Wed, 24 Jul 2019 at 2:04 PM, Jeff Zhang wrote: >> >>> I think it is better to shade all the dependencies of flink so that all >>> the projects that use flink won't hit this kind of issue. >>> >>> >>> Haibo Sun 于2019年7月24日周三 下午4:07写道： >>> Hi, Debasish Ghosh I don't know why not shade Akka, maybe it can be shaded. Chesnay may be able to answer that. I recommend to shade Akka dependency of your application because it don't be known what's wrong with shading Flink's Akka. CC @Chesnay Schepler Best, Haibo At 2019-07-24 15:48:59, "Debasish Ghosh" wrote: The problem that I am facing is with Akka serialization .. Why not shade the whole of Akka ? java.lang.AbstractMethodError: > akka.remote.RemoteActorRefProvider.serializationInformation()Lakka/serialization/Serialization$Information; > at > akka.serialization.Serialization.serializationInformation(Serialization.scala:166) Akka 2.6 is just around the corner and I don't think Flink will upgrade to Akka 2.6 that soon .. so somehow this problem is bound to recur .. regards. On Wed, Jul 24, 2019 at 1:01 PM Zili Chen wrote: > I can see that we relocate akka's netty, akka uncommon math but also > be curious why Flink doesn't shaded all of akka dependencies... > > Best, > tison. > > > Debasish Ghosh 于2019年7月24日周三 下午3:15写道： > >> Hello Haibo - >> >> Yes, my application depends on Akka 2.5. >> Just curious, why do you think it's recommended to shade Akka version >> of my application instead of Flink ? >> >> regards. >> >> On Wed, Jul 24, 2019 at 12:42 PM Haibo Sun >> wrote: >> >>> Hi Debasish Ghosh, >>> >>> Does your application have to depend on Akka 2.5? If not, it's a >>> good idea to always keep the Akka version that the application depend >>> on in >>> line with Flink. >>> If you want to try shading Akka dependency, I think that it is more >>> recommended to shade Akka dependency of your application. >>> >>> Best, >>> Haibo >>> >>> At 2019-07-24 14:31:29, "Debasish Ghosh" >>> wrote: >>> >>> Hello - >>> >>> An application that uses Akka 2.5 and Flink 1.8.0 gives runtime >>> errors because of version mismatch between Akka that we use and the one >>> that Flink uses (which is Akka 2.4). Anyone tried shading Akka >>> dependency >>> with Flink ? >>> >>> Or is there any other alternative way to handle this issue ? I know >>> Flink 1.9 has upgraded to Akka 2.5 but this is (I think) going to be a >>> recurring problem down the line with mismatch between the new releases >>> of >>> Akka and Flink. >>> >>> regards. >>> >>> -- >>> Debasish Ghosh >>> http://manning.com/ghosh2 >>> http://manning.com/ghosh >>> >>> Twttr: @debasishg >>> Blog: http://debasishg.blogspot.com >>> Code: http://github.com/debasishg >>> >>> >> >> -- >> Debasish Ghosh >> http://manning.com/ghosh2 >> http://manning.com/ghosh >> >> Twttr: @debasishg >> Blog: http://debasishg.blogspot.com >> Code: http://github.com/debasishg >> > -- Debasish Ghosh http://manning.com/ghosh2 http://manning.com/ghosh Twttr: @debasishg Blog: http://debasishg.blogspot.com Code: http://github.com/debasishg >>> >>> -- >>> Best Regards >>> >>> Jeff Zhang >>> >> -- >> Sent from my iPhone >> >> > > -- > Debasish Ghosh > http://manning.com/ghosh2 > http://manning.com/ghosh > > Twttr: @debasishg > Blog: http://debasishg.blogspot.com > Code: http://github.com/debasishg > -- Debasish Ghosh
Re: Re: Re: Flink and Akka version incompatibility ..
Hi Haibo - Thanks for the clarification .. regards. On Wed, Jul 24, 2019 at 2:58 PM Haibo Sun wrote: > Hi Debasish Ghosh, > > I agree that Flink should shade its Akka. > > Maybe you misunderstood me. I mean, in the absence of official shading > Akka in Flink, the relatively conservative way is to shade Akka of your > application (I concern Flink won't work well after shading its Akka). > > Best, > Haibo > > At 2019-07-24 16:43:28, "Debasish Ghosh" wrote: > > For our application users are expected to work with Akka APIs - hence if I > shade Akka in my application users will need to work with shaded imports > which feels unnatural. With Flink, Akka is an implementation detail and > Flink users are not expected to use Akka APIs. Hence shading will not have > any user level impact. > > Hence the suggestion to shade Akka in Flink rather than the user > application. > > regards. > > On Wed, 24 Jul 2019 at 2:04 PM, Jeff Zhang wrote: > >> I think it is better to shade all the dependencies of flink so that all >> the projects that use flink won't hit this kind of issue. >> >> >> Haibo Sun 于2019年7月24日周三 下午4:07写道： >> >>> Hi, Debasish Ghosh >>> >>> I don't know why not shade Akka, maybe it can be shaded. Chesnay may be >>> able to answer that. >>> I recommend to shade Akka dependency of your application because it >>> don't be known what's wrong with shading Flink's Akka. >>> >>> CC @Chesnay Schepler >>> >>> Best, >>> Haibo >>> >>> At 2019-07-24 15:48:59, "Debasish Ghosh" >>> wrote: >>> >>> The problem that I am facing is with Akka serialization .. Why not shade >>> the whole of Akka ? >>> >>> java.lang.AbstractMethodError: akka.remote.RemoteActorRefProvider.serializationInformation()Lakka/serialization/Serialization$Information; at akka.serialization.Serialization.serializationInformation(Serialization.scala:166) >>> >>> >>> Akka 2.6 is just around the corner and I don't think Flink will upgrade >>> to Akka 2.6 that soon .. so somehow this problem is bound to recur .. >>> >>> regards. >>> >>> On Wed, Jul 24, 2019 at 1:01 PM Zili Chen wrote: >>> I can see that we relocate akka's netty, akka uncommon math but also be curious why Flink doesn't shaded all of akka dependencies... Best, tison. Debasish Ghosh 于2019年7月24日周三 下午3:15写道： > Hello Haibo - > > Yes, my application depends on Akka 2.5. > Just curious, why do you think it's recommended to shade Akka version > of my application instead of Flink ? > > regards. > > On Wed, Jul 24, 2019 at 12:42 PM Haibo Sun wrote: > >> Hi Debasish Ghosh, >> >> Does your application have to depend on Akka 2.5? If not, it's a >> good idea to always keep the Akka version that the application depend on >> in >> line with Flink. >> If you want to try shading Akka dependency, I think that it is more >> recommended to shade Akka dependency of your application. >> >> Best, >> Haibo >> >> At 2019-07-24 14:31:29, "Debasish Ghosh" >> wrote: >> >> Hello - >> >> An application that uses Akka 2.5 and Flink 1.8.0 gives runtime >> errors because of version mismatch between Akka that we use and the one >> that Flink uses (which is Akka 2.4). Anyone tried shading Akka dependency >> with Flink ? >> >> Or is there any other alternative way to handle this issue ? I know >> Flink 1.9 has upgraded to Akka 2.5 but this is (I think) going to be a >> recurring problem down the line with mismatch between the new releases of >> Akka and Flink. >> >> regards. >> >> -- >> Debasish Ghosh >> http://manning.com/ghosh2 >> http://manning.com/ghosh >> >> Twttr: @debasishg >> Blog: http://debasishg.blogspot.com >> Code: http://github.com/debasishg >> >> > > -- > Debasish Ghosh > http://manning.com/ghosh2 > http://manning.com/ghosh > > Twttr: @debasishg > Blog: http://debasishg.blogspot.com > Code: http://github.com/debasishg > >>> >>> -- >>> Debasish Ghosh >>> http://manning.com/ghosh2 >>> http://manning.com/ghosh >>> >>> Twttr: @debasishg >>> Blog: http://debasishg.blogspot.com >>> Code: http://github.com/debasishg >>> >>> >> >> -- >> Best Regards >> >> Jeff Zhang >> > -- > Sent from my iPhone > > -- Debasish Ghosh http://manning.com/ghosh2 http://manning.com/ghosh Twttr: @debasishg Blog: http://debasishg.blogspot.com Code: http://github.com/debasishg
Re:Re: Re: Flink and Akka version incompatibility ..
Hi Debasish Ghosh, I agree that Flink should shade its Akka. Maybe you misunderstood me. I mean, in the absence of official shading Akka in Flink, the relatively conservative way is to shade Akka of your application (I concern Flink won't work well after shading its Akka). Best, Haibo At 2019-07-24 16:43:28, "Debasish Ghosh" wrote: For our application users are expected to work with Akka APIs - hence if I shade Akka in my application users will need to work with shaded imports which feels unnatural. With Flink, Akka is an implementation detail and Flink users are not expected to use Akka APIs. Hence shading will not have any user level impact. Hence the suggestion to shade Akka in Flink rather than the user application. regards. On Wed, 24 Jul 2019 at 2:04 PM, Jeff Zhang wrote: I think it is better to shade all the dependencies of flink so that all the projects that use flink won't hit this kind of issue. Haibo Sun 于2019年7月24日周三 下午4:07写道： Hi, Debasish Ghosh I don't know why not shade Akka, maybe it can be shaded. Chesnay may be able to answer that. I recommend to shade Akka dependency of your application because it don't be known what's wrong with shading Flink's Akka. CC @Chesnay Schepler Best, Haibo At 2019-07-24 15:48:59, "Debasish Ghosh" wrote: The problem that I am facing is with Akka serialization .. Why not shade the whole of Akka ? java.lang.AbstractMethodError: akka.remote.RemoteActorRefProvider.serializationInformation()Lakka/serialization/Serialization$Information; at akka.serialization.Serialization.serializationInformation(Serialization.scala:166) Akka 2.6 is just around the corner and I don't think Flink will upgrade to Akka 2.6 that soon .. so somehow this problem is bound to recur .. regards. On Wed, Jul 24, 2019 at 1:01 PM Zili Chen wrote: I can see that we relocate akka's netty, akka uncommon math but also be curious why Flink doesn't shaded all of akka dependencies... Best, tison. Debasish Ghosh 于2019年7月24日周三 下午3:15写道： Hello Haibo - Yes, my application depends on Akka 2.5. Just curious, why do you think it's recommended to shade Akka version of my application instead of Flink ? regards. On Wed, Jul 24, 2019 at 12:42 PM Haibo Sun wrote: Hi Debasish Ghosh, Does your application have to depend on Akka 2.5? If not, it's a good idea to always keep the Akka version that the application depend on in line with Flink. If you want to try shading Akka dependency, I think that it is more recommended to shade Akka dependency of your application. Best, Haibo At 2019-07-24 14:31:29, "Debasish Ghosh" wrote: Hello - An application that uses Akka 2.5 and Flink 1.8.0 gives runtime errors because of version mismatch between Akka that we use and the one that Flink uses (which is Akka 2.4). Anyone tried shading Akka dependency with Flink ? Or is there any other alternative way to handle this issue ? I know Flink 1.9 has upgraded to Akka 2.5 but this is (I think) going to be a recurring problem down the line with mismatch between the new releases of Akka and Flink. regards. -- Debasish Ghosh http://manning.com/ghosh2 http://manning.com/ghosh Twttr: @debasishg Blog: http://debasishg.blogspot.com Code: http://github.com/debasishg -- Debasish Ghosh http://manning.com/ghosh2 http://manning.com/ghosh Twttr: @debasishg Blog: http://debasishg.blogspot.com Code: http://github.com/debasishg -- Debasish Ghosh http://manning.com/ghosh2 http://manning.com/ghosh Twttr: @debasishg Blog: http://debasishg.blogspot.com Code: http://github.com/debasishg -- Best Regards Jeff Zhang -- Sent from my iPhone
Re: Re: Flink and Akka version incompatibility ..
For our application users are expected to work with Akka APIs - hence if I shade Akka in my application users will need to work with shaded imports which feels unnatural. With Flink, Akka is an implementation detail and Flink users are not expected to use Akka APIs. Hence shading will not have any user level impact. Hence the suggestion to shade Akka in Flink rather than the user application. regards. On Wed, 24 Jul 2019 at 2:04 PM, Jeff Zhang wrote: > I think it is better to shade all the dependencies of flink so that all > the projects that use flink won't hit this kind of issue. > > > Haibo Sun 于2019年7月24日周三 下午4:07写道： > >> Hi, Debasish Ghosh >> >> I don't know why not shade Akka, maybe it can be shaded. Chesnay may be >> able to answer that. >> I recommend to shade Akka dependency of your application because it don't >> be known what's wrong with shading Flink's Akka. >> >> CC @Chesnay Schepler >> >> Best, >> Haibo >> >> At 2019-07-24 15:48:59, "Debasish Ghosh" >> wrote: >> >> The problem that I am facing is with Akka serialization .. Why not shade >> the whole of Akka ? >> >> java.lang.AbstractMethodError: >>> akka.remote.RemoteActorRefProvider.serializationInformation()Lakka/serialization/Serialization$Information; >>> at >>> akka.serialization.Serialization.serializationInformation(Serialization.scala:166) >> >> >> Akka 2.6 is just around the corner and I don't think Flink will upgrade >> to Akka 2.6 that soon .. so somehow this problem is bound to recur .. >> >> regards. >> >> On Wed, Jul 24, 2019 at 1:01 PM Zili Chen wrote: >> >>> I can see that we relocate akka's netty, akka uncommon math but also >>> be curious why Flink doesn't shaded all of akka dependencies... >>> >>> Best, >>> tison. >>> >>> >>> Debasish Ghosh 于2019年7月24日周三 下午3:15写道： >>> Hello Haibo - Yes, my application depends on Akka 2.5. Just curious, why do you think it's recommended to shade Akka version of my application instead of Flink ? regards. On Wed, Jul 24, 2019 at 12:42 PM Haibo Sun wrote: > Hi Debasish Ghosh, > > Does your application have to depend on Akka 2.5? If not, it's a good > idea to always keep the Akka version that the application depend on in > line > with Flink. > If you want to try shading Akka dependency, I think that it is more > recommended to shade Akka dependency of your application. > > Best, > Haibo > > At 2019-07-24 14:31:29, "Debasish Ghosh" > wrote: > > Hello - > > An application that uses Akka 2.5 and Flink 1.8.0 gives runtime errors > because of version mismatch between Akka that we use and the one that > Flink > uses (which is Akka 2.4). Anyone tried shading Akka dependency with Flink > ? > > Or is there any other alternative way to handle this issue ? I know > Flink 1.9 has upgraded to Akka 2.5 but this is (I think) going to be a > recurring problem down the line with mismatch between the new releases of > Akka and Flink. > > regards. > > -- > Debasish Ghosh > http://manning.com/ghosh2 > http://manning.com/ghosh > > Twttr: @debasishg > Blog: http://debasishg.blogspot.com > Code: http://github.com/debasishg > > -- Debasish Ghosh http://manning.com/ghosh2 http://manning.com/ghosh Twttr: @debasishg Blog: http://debasishg.blogspot.com Code: http://github.com/debasishg >>> >> >> -- >> Debasish Ghosh >> http://manning.com/ghosh2 >> http://manning.com/ghosh >> >> Twttr: @debasishg >> Blog: http://debasishg.blogspot.com >> Code: http://github.com/debasishg >> >> > > -- > Best Regards > > Jeff Zhang > -- Sent from my iPhone
Re: Re: Flink and Akka version incompatibility ..
I think it is better to shade all the dependencies of flink so that all the projects that use flink won't hit this kind of issue. Haibo Sun 于2019年7月24日周三 下午4:07写道： > Hi, Debasish Ghosh > > I don't know why not shade Akka, maybe it can be shaded. Chesnay may be > able to answer that. > I recommend to shade Akka dependency of your application because it don't > be known what's wrong with shading Flink's Akka. > > CC @Chesnay Schepler > > Best, > Haibo > > At 2019-07-24 15:48:59, "Debasish Ghosh" wrote: > > The problem that I am facing is with Akka serialization .. Why not shade > the whole of Akka ? > > java.lang.AbstractMethodError: >> akka.remote.RemoteActorRefProvider.serializationInformation()Lakka/serialization/Serialization$Information; >> at >> akka.serialization.Serialization.serializationInformation(Serialization.scala:166) > > > Akka 2.6 is just around the corner and I don't think Flink will upgrade to > Akka 2.6 that soon .. so somehow this problem is bound to recur .. > > regards. > > On Wed, Jul 24, 2019 at 1:01 PM Zili Chen wrote: > >> I can see that we relocate akka's netty, akka uncommon math but also >> be curious why Flink doesn't shaded all of akka dependencies... >> >> Best, >> tison. >> >> >> Debasish Ghosh 于2019年7月24日周三 下午3:15写道： >> >>> Hello Haibo - >>> >>> Yes, my application depends on Akka 2.5. >>> Just curious, why do you think it's recommended to shade Akka version of >>> my application instead of Flink ? >>> >>> regards. >>> >>> On Wed, Jul 24, 2019 at 12:42 PM Haibo Sun wrote: >>> Hi Debasish Ghosh, Does your application have to depend on Akka 2.5? If not, it's a good idea to always keep the Akka version that the application depend on in line with Flink. If you want to try shading Akka dependency, I think that it is more recommended to shade Akka dependency of your application. Best, Haibo At 2019-07-24 14:31:29, "Debasish Ghosh" wrote: Hello - An application that uses Akka 2.5 and Flink 1.8.0 gives runtime errors because of version mismatch between Akka that we use and the one that Flink uses (which is Akka 2.4). Anyone tried shading Akka dependency with Flink ? Or is there any other alternative way to handle this issue ? I know Flink 1.9 has upgraded to Akka 2.5 but this is (I think) going to be a recurring problem down the line with mismatch between the new releases of Akka and Flink. regards. -- Debasish Ghosh http://manning.com/ghosh2 http://manning.com/ghosh Twttr: @debasishg Blog: http://debasishg.blogspot.com Code: http://github.com/debasishg >>> >>> -- >>> Debasish Ghosh >>> http://manning.com/ghosh2 >>> http://manning.com/ghosh >>> >>> Twttr: @debasishg >>> Blog: http://debasishg.blogspot.com >>> Code: http://github.com/debasishg >>> >> > > -- > Debasish Ghosh > http://manning.com/ghosh2 > http://manning.com/ghosh > > Twttr: @debasishg > Blog: http://debasishg.blogspot.com > Code: http://github.com/debasishg > > -- Best Regards Jeff Zhang
Re: Flink and Akka version incompatibility ..
The problem that I am facing is with Akka serialization .. Why not shade the whole of Akka ? java.lang.AbstractMethodError: > akka.remote.RemoteActorRefProvider.serializationInformation()Lakka/serialization/Serialization$Information; > at > akka.serialization.Serialization.serializationInformation(Serialization.scala:166) Akka 2.6 is just around the corner and I don't think Flink will upgrade to Akka 2.6 that soon .. so somehow this problem is bound to recur .. regards. On Wed, Jul 24, 2019 at 1:01 PM Zili Chen wrote: > I can see that we relocate akka's netty, akka uncommon math but also > be curious why Flink doesn't shaded all of akka dependencies... > > Best, > tison. > > > Debasish Ghosh 于2019年7月24日周三 下午3:15写道： > >> Hello Haibo - >> >> Yes, my application depends on Akka 2.5. >> Just curious, why do you think it's recommended to shade Akka version of >> my application instead of Flink ? >> >> regards. >> >> On Wed, Jul 24, 2019 at 12:42 PM Haibo Sun wrote: >> >>> Hi Debasish Ghosh, >>> >>> Does your application have to depend on Akka 2.5? If not, it's a good >>> idea to always keep the Akka version that the application depend on in line >>> with Flink. >>> If you want to try shading Akka dependency, I think that it is more >>> recommended to shade Akka dependency of your application. >>> >>> Best, >>> Haibo >>> >>> At 2019-07-24 14:31:29, "Debasish Ghosh" >>> wrote: >>> >>> Hello - >>> >>> An application that uses Akka 2.5 and Flink 1.8.0 gives runtime errors >>> because of version mismatch between Akka that we use and the one that Flink >>> uses (which is Akka 2.4). Anyone tried shading Akka dependency with Flink ? >>> >>> Or is there any other alternative way to handle this issue ? I know >>> Flink 1.9 has upgraded to Akka 2.5 but this is (I think) going to be a >>> recurring problem down the line with mismatch between the new releases of >>> Akka and Flink. >>> >>> regards. >>> >>> -- >>> Debasish Ghosh >>> http://manning.com/ghosh2 >>> http://manning.com/ghosh >>> >>> Twttr: @debasishg >>> Blog: http://debasishg.blogspot.com >>> Code: http://github.com/debasishg >>> >>> >> >> -- >> Debasish Ghosh >> http://manning.com/ghosh2 >> http://manning.com/ghosh >> >> Twttr: @debasishg >> Blog: http://debasishg.blogspot.com >> Code: http://github.com/debasishg >> > -- Debasish Ghosh http://manning.com/ghosh2 http://manning.com/ghosh Twttr: @debasishg Blog: http://debasishg.blogspot.com Code: http://github.com/debasishg
Re: Flink and Akka version incompatibility ..
I can see that we relocate akka's netty, akka uncommon math but also be curious why Flink doesn't shaded all of akka dependencies... Best, tison. Debasish Ghosh 于2019年7月24日周三 下午3:15写道： > Hello Haibo - > > Yes, my application depends on Akka 2.5. > Just curious, why do you think it's recommended to shade Akka version of > my application instead of Flink ? > > regards. > > On Wed, Jul 24, 2019 at 12:42 PM Haibo Sun wrote: > >> Hi Debasish Ghosh, >> >> Does your application have to depend on Akka 2.5? If not, it's a good >> idea to always keep the Akka version that the application depend on in line >> with Flink. >> If you want to try shading Akka dependency, I think that it is more >> recommended to shade Akka dependency of your application. >> >> Best, >> Haibo >> >> At 2019-07-24 14:31:29, "Debasish Ghosh" >> wrote: >> >> Hello - >> >> An application that uses Akka 2.5 and Flink 1.8.0 gives runtime errors >> because of version mismatch between Akka that we use and the one that Flink >> uses (which is Akka 2.4). Anyone tried shading Akka dependency with Flink ? >> >> Or is there any other alternative way to handle this issue ? I know Flink >> 1.9 has upgraded to Akka 2.5 but this is (I think) going to be a recurring >> problem down the line with mismatch between the new releases of Akka and >> Flink. >> >> regards. >> >> -- >> Debasish Ghosh >> http://manning.com/ghosh2 >> http://manning.com/ghosh >> >> Twttr: @debasishg >> Blog: http://debasishg.blogspot.com >> Code: http://github.com/debasishg >> >> > > -- > Debasish Ghosh > http://manning.com/ghosh2 > http://manning.com/ghosh > > Twttr: @debasishg > Blog: http://debasishg.blogspot.com > Code: http://github.com/debasishg >
Re: Flink and Akka version incompatibility ..
Hello Haibo - Yes, my application depends on Akka 2.5. Just curious, why do you think it's recommended to shade Akka version of my application instead of Flink ? regards. On Wed, Jul 24, 2019 at 12:42 PM Haibo Sun wrote: > Hi Debasish Ghosh, > > Does your application have to depend on Akka 2.5? If not, it's a good > idea to always keep the Akka version that the application depend on in line > with Flink. > If you want to try shading Akka dependency, I think that it is more > recommended to shade Akka dependency of your application. > > Best, > Haibo > > At 2019-07-24 14:31:29, "Debasish Ghosh" wrote: > > Hello - > > An application that uses Akka 2.5 and Flink 1.8.0 gives runtime errors > because of version mismatch between Akka that we use and the one that Flink > uses (which is Akka 2.4). Anyone tried shading Akka dependency with Flink ? > > Or is there any other alternative way to handle this issue ? I know Flink > 1.9 has upgraded to Akka 2.5 but this is (I think) going to be a recurring > problem down the line with mismatch between the new releases of Akka and > Flink. > > regards. > > -- > Debasish Ghosh > http://manning.com/ghosh2 > http://manning.com/ghosh > > Twttr: @debasishg > Blog: http://debasishg.blogspot.com > Code: http://github.com/debasishg > > -- Debasish Ghosh http://manning.com/ghosh2 http://manning.com/ghosh Twttr: @debasishg Blog: http://debasishg.blogspot.com Code: http://github.com/debasishg
