On Fri, Jan 17, 2020 at 9:12 AM Aaron Dixon <atdi...@gmail.com> wrote:
> @Tomo Suzuki Thanks for looking at this and your thorough analysis// let
> me know if I can help with any regression testing, CRs, or more context
> anything else.
>
> ---
>
> @Elliotte I was a little confused at first re: vendoring vs shading but
> here is how I understand it (happy to be corrected if anything I say here
> is a mischaracterization); I tend to over-explain for my own clarity
> fwiw--using Guava as an example:
>
> With SHADING you have your project (e.g., Beam) depend directly on Guava
> and its public package names. Your build will then compile and link to
> these package names and only after-the-fact [1] rewrite the byte-code to
> rename the guava package. This entails both byte-code rewriting the Guava
> class files (to rename the packages of the Guava code itself) and byte-code
> rewriting all of your project class files that import Guava (to change the
> import to use the new package name).
>
> With VENDORING you release a renamed Guava ("vendor") JAR and depend on
> that directly from your project. So your project's source code itself
> imports the (repackated) vendor JAR and depends on the
> renamed packages directly. Your project build then has no need for shading
> or byte code rewriting of its own classes. I see vendoring making it very
> clear what is going on when looking at a project's source code and its
> dependencies, and there are likely other tangible brass-tacks benefits to
> vendoring besides the conceptual clarity over shading.
>
Exactly. Vendoring can also refer to just making your own copy of a
library. The conceptual clarity extends beyond humans into clarity for
tooling. IDEs understand it better, and it is much easier to manage builds.
And of course it avoids jar bloat and speeds builds. It isn't perfect. You
lose interop.
Kenn
> [1] You can see that Maven shade:shade goal binds by default to Maven's
> package phase (which is well after after compilation).
>
> On Fri, Jan 17, 2020 at 9:16 AM Tomo Suzuki <suzt...@google.com> wrote:
>
>> Ismaël and I think the short-term solution for Aaron's issue is to
>> define Beam's own class for Avro's TimestampConversion. Luckily it's
>> the only blocker for Avro 1.9 I found.
>> I created a ticket https://issues.apache.org/jira/browse/BEAM-9144 and
>> PR https://github.com/apache/beam/pull/10628 .
>>
>> Regards,
>> Tomo
>>
>> On Fri, Jan 17, 2020 at 7:32 AM Elliotte Rusty Harold
>> <elh...@ibiblio.org> wrote:
>> >
>> > On Thu, Jan 16, 2020 at 7:08 PM Kenneth Knowles <k...@apache.org>
>> wrote:
>> > >
>> > > Regarding Google's advice about shading: don't go to a "one version
>> rule" monorepo for advice about solving diamond dependencies in the wild.
>> >
>> > FWIW these docs came out of the open source, Github, multirepo, many
>> > versions part of Google Cloud, not the mono repo world of google3.
>> > Though I will say the more time I spend trying to deal with these
>> > issues the more I love the monorepo.
>> >
>> > > It is a useful description of the pitfalls. We (and Flink before us,
>> and likely many more) are already doing something that avoids many of them
>> or makes them less likely. Building a separate vendored library is simpler
>> and more robust than the "shade during build".
>> > >
>> >
>> > I've only encountered "vendoring" as distinct from shading in Beam.
>> > Are there more details about this anywhere? A quick Google search only
>> > turned up one Beam doc that pointed to Go docs and a cloud endpoints
>> > doc that seemed to use it as a synonym for shading.
>> >
>> >
>> > --
>> > Elliotte Rusty Harold
>> > elh...@ibiblio.org
>>
>>
>>
>> --
>> Regards,
>> Tomo
>>
>
