Apex is prone to class leak so even application developers have to use the guava version that underlying Hadoop uses. Platform developers have the least say other than fixing the class leak and free application developers to use whatever they want. The solution is said to be super complicated and there are no takers motivated enough to implement it.
‹ Chetan On 11/25/15, 5:01 AM, "Ted Dunning" <[email protected]> wrote: >Flink definitely hit the problem. > >The issue is that people use different versions of guava in their >applications and they aren't compatible. > >You are likely to want to use a more advanced version than the Hadoop >systems use, also. > > > >On Wed, Nov 25, 2015 at 3:13 PM, Timothy Farkas <[email protected]> >wrote: > >> I haven't seen an issue :) >> >> On Tue, Nov 24, 2015 at 10:57 PM, Thomas Weise <[email protected]> >> wrote: >> >> > That certainly may be the case. But have we seen problems sticking to >>the >> > Hadoop 2.2 compatible subset? AFAIK Apex runs on all Hadoop versions >> > starting from 2.2 across all major distros. >> > >> > On Tue, Nov 24, 2015 at 10:36 PM, Timothy Farkas <[email protected]> >> > wrote: >> > >> > > I was under the impression that the guava version varies for >>different >> > > hadoop distributions and versions, also guava is not strictly >>backwards >> > > compatible. Is that not the case? >> > > >> > > On Tue, Nov 24, 2015 at 10:30 PM, Thomas Weise >><[email protected] >> > >> > > wrote: >> > > >> > > > We do it for ASM. But why do we need to use a different guava >> version? >> > > > >> > > > >> > > > On Tue, Nov 24, 2015 at 10:27 PM, Timothy Farkas < >> [email protected]> >> > > > wrote: >> > > > >> > > > > Flink shades the guava dependency so that Flink can use a >>version >> of >> > > > guava >> > > > > that is different from the one provided by the hadoop >>distribution. >> > Is >> > > > this >> > > > > something that we should be doing as well? >> > > > > >> > > > > >> > > >> https://github.com/apache/flink/blob/master/flink-shaded-hadoop/pom.xml >> > > > > >> > > > >> > > >> > >>
