Re: [DISCUSS] [Java] Private shaded dependency uber jars

well we can disagree on the code - it is fine ;), but the needed part of it by beam is not huge and in any case it can be forked without requiring 10 classes - if so we'll use another impl than the guava one ;). This is the whole point. Romain Manni-Bucau @rmannibucau

Re: [DISCUSS] [Java] Private shaded dependency uber jars

TypeToken is not trivial. I've written code to do what TypeToken does before (figuring out generic ancestor types). It's actually somewhat tricky, and the code I wrote had subtle bugs in it; eventually we removed this code in favor of Guava's implementation :) On Fri, Feb 2, 2018 at 7:47 AM,

Re: [DISCUSS] [Java] Private shaded dependency uber jars

Yep, note I never said to reinvent the wheel, we can copy it from guava, openwebbeans or any other impl. Point was more to avoid to depend on something we don't own for that which is after all not that much code. I also think we can limit it a lot to align it on what is supported by beam (I'm

Re: [DISCUSS] [Java] Private shaded dependency uber jars

On Fri, Feb 2, 2018 at 7:18 AM, Romain Manni-Bucau wrote: > Don't forget beam doesn't support much behind it (mainly only a few > ParameterizedType due the usage code path) so it is mainly only about > handling parameterized types and typevariables recursively. Not a lot

Re: [DISCUSS] [Java] Private shaded dependency uber jars

Don't forget beam doesn't support much behind it (mainly only a few ParameterizedType due the usage code path) so it is mainly only about handling parameterized types and typevariables recursively. Not a lot of work. Main concern being it is in the API so using a shade as an API is never a good

Re: [DISCUSS] [Java] Private shaded dependency uber jars

On Fri, Feb 2, 2018 at 6:41 AM, Romain Manni-Bucau wrote: > > > 2018-02-02 15:37 GMT+01:00 Kenneth Knowles : > >> Another couple: >> >> - User-facing TypeDescriptor is a very thin wrapper on Guava's TypeToken >> > > Technically reflect Type is enough >

Re: [DISCUSS] [Java] Private shaded dependency uber jars

2018-02-02 15:37 GMT+01:00 Kenneth Knowles : > Another couple: > > - User-facing TypeDescriptor is a very thin wrapper on Guava's TypeToken > Technically reflect Type is enough > - ImmutableList and friends and their builders are very widely used and > IMO still add a lot

Re: [DISCUSS] [Java] Private shaded dependency uber jars

Nice. This sounds like a great idea (or two?) and goes along with what I just started for futures. Guava: filed https://issues.apache.org/jira/browse/BEAM-3606 and assigned to Ismaël :-) and converted my futures thing to a subtask. Specific things for our micro guava: - checkArgumentNotNull

Re: [DISCUSS] [Java] Private shaded dependency uber jars

Huge +1 to get rid of Guava! This solves annoying dependency issues for some IOs and allow us to get rid of the shading that makes current jars bigger than they should be. We can create our own 'micro guava' package with some classes for things that are hard to migrate, or that we prefer to

Re: [DISCUSS] [Java] Private shaded dependency uber jars

a map of list is fine and not a challenge we'll face long I hope ;) Romain Manni-Bucau @rmannibucau | Blog | Old Blog | Github | LinkedIn

Re: [DISCUSS] [Java] Private shaded dependency uber jars

Not sure we'll be able to replace them all. Things like guava Table and Multimap don't have great replacements in Java8. On Wed, Jan 31, 2018 at 10:11 PM, Jean-Baptiste Onofré wrote: > +1, it was on my TODO for a while waiting the Java8 update. > > Regards > JB > > On

Re: [DISCUSS] [Java] Private shaded dependency uber jars

+1, it was on my TODO for a while waiting the Java8 update. Regards JB On 02/01/2018 06:56 AM, Romain Manni-Bucau wrote: > Why not dropping guava for all beam codebase? With java 8 it is quite easy to > do > it and avoid a bunch of conflicts. Did it in 2 projects with quite a good > result. >

Re: [DISCUSS] [Java] Private shaded dependency uber jars

That is an even better idea. A lot of guava constructs have been superseded by stuff that was added to Java 8. On Wed, Jan 31, 2018 at 9:56 PM, Romain Manni-Bucau wrote: > Why not dropping guava for all beam codebase? With java 8 it is quite easy > to do it and avoid a

Re: [DISCUSS] [Java] Private shaded dependency uber jars

Why not dropping guava for all beam codebase? With java 8 it is quite easy to do it and avoid a bunch of conflicts. Did it in 2 projects with quite a good result. Le 1 févr. 2018 06:50, "Lukasz Cwik" a écrit : > Make sure to include the guava version in the artifact name so

Re: [DISCUSS] [Java] Private shaded dependency uber jars

Make sure to include the guava version in the artifact name so that we can have multiple vendored versions. On Wed, Jan 31, 2018 at 9:16 PM, Kenneth Knowles wrote: > I didn't have time for this, but it just bit me. We definitely have Guava > on the API surface of runner support

Re: [DISCUSS] [Java] Private shaded dependency uber jars

I didn't have time for this, but it just bit me. We definitely have Guava on the API surface of runner support code in ways that get incompatibly shaded. I will probably start "1a" by making a shaded library org.apache.beam:vendored-guava and starting to use it. It sounds like there is generally

Re: [DISCUSS] [Java] Private shaded dependency uber jars

I would suggest that either we use: 1) A common deps package containing shaded dependencies allows for Pros * doesn't require the user to build an uber jar Risks * dependencies package will keep growing even if something is or isn't needed by all of Apache Beam leading to a large jar anyways

Re: [DISCUSS] [Java] Private shaded dependency uber jars

Thanks for bringing that back. Indeed guava is shaded in different uber-jar. Maybe we can have a common deps module that we include once (but the user will have to explicitly define the dep) ? Basically, what do you propose for protobuf (unfortunately, I don't see an obvious) ? Regards JB

Re: [DISCUSS] [Java] Private shaded dependency uber jars

Hello, I wanted to bring back this subject because I think we should take action on this and at least first have a shaded version of guava. I was playing with a toy project and I did the procedure we use to submit jars to a Hadoop cluster via Flink/Spark which involves creating an uber jar and I

Re: [DISCUSS] [Java] Private shaded dependency uber jars

+1 to the goal. I'm hugely in favor of not doing the same shading work every time for dependencies we know we'll use. This also means that if we end up pulling in transitive dependencies we don't want in any particular module we can avoid having to adjust our repackaging strategy for that module