Chetan, Could you elaborate on what you mean by "class leak" and why the issue is complicated ?
Thanks. Ram On Wed, Nov 25, 2015 at 12:02 PM, Chetan Narsude (cnarsude) < [email protected]> wrote: > Apex is prone to class leak so even application developers have to use the > guava version that underlying Hadoop uses. Platform developers have the > least say other than fixing the class leak and free application developers > to use whatever they want. The solution is said to be super complicated > and there are no takers motivated enough to implement it. > > ‹ > Chetan > > > > On 11/25/15, 5:01 AM, "Ted Dunning" <[email protected]> wrote: > > >Flink definitely hit the problem. > > > >The issue is that people use different versions of guava in their > >applications and they aren't compatible. > > > >You are likely to want to use a more advanced version than the Hadoop > >systems use, also. > > > > > > > >On Wed, Nov 25, 2015 at 3:13 PM, Timothy Farkas <[email protected]> > >wrote: > > > >> I haven't seen an issue :) > >> > >> On Tue, Nov 24, 2015 at 10:57 PM, Thomas Weise <[email protected]> > >> wrote: > >> > >> > That certainly may be the case. But have we seen problems sticking to > >>the > >> > Hadoop 2.2 compatible subset? AFAIK Apex runs on all Hadoop versions > >> > starting from 2.2 across all major distros. > >> > > >> > On Tue, Nov 24, 2015 at 10:36 PM, Timothy Farkas <[email protected] > > > >> > wrote: > >> > > >> > > I was under the impression that the guava version varies for > >>different > >> > > hadoop distributions and versions, also guava is not strictly > >>backwards > >> > > compatible. Is that not the case? > >> > > > >> > > On Tue, Nov 24, 2015 at 10:30 PM, Thomas Weise > >><[email protected] > >> > > >> > > wrote: > >> > > > >> > > > We do it for ASM. But why do we need to use a different guava > >> version? > >> > > > > >> > > > > >> > > > On Tue, Nov 24, 2015 at 10:27 PM, Timothy Farkas < > >> [email protected]> > >> > > > wrote: > >> > > > > >> > > > > Flink shades the guava dependency so that Flink can use a > >>version > >> of > >> > > > guava > >> > > > > that is different from the one provided by the hadoop > >>distribution. > >> > Is > >> > > > this > >> > > > > something that we should be doing as well? > >> > > > > > >> > > > > > >> > > > >> https://github.com/apache/flink/blob/master/flink-shaded-hadoop/pom.xml > >> > > > > > >> > > > > >> > > > >> > > >> > >
