I'm not saying it is a best practice. I'm suggesting we should figure out whether it's a best practice, worst practice, or somewhere in between and document in which situations it is or is not a good idea. Up till now I haven't really considered it an alternative to shading so it seems worth thinking about more deeply.
On Fri, Jan 17, 2020 at 1:35 PM Kenneth Knowles <k...@apache.org> wrote: > > > > On Fri, Jan 17, 2020 at 10:21 AM Elliotte Rusty Harold <elh...@ibiblio.org> > wrote: >> >> Thanks. That's a very clear explanation. I should think about how to >> incorporate that into https://jlbp.dev > > > Might want to give it some time before calling it a best practice. Maybe > we'll come to regret it :-). But I'm willing to say it is a better > alternative to shading in most cases, with the major exception being > generated code (like proto libs) that depend on libraries (like > protobuf-java). Ideally, proto compiler would vendor the dep into each > generated lib so we don't have to deal with the issue at all. Instead we do > it ourselves. > > Kenn > >> >> >> >> On Fri, Jan 17, 2020 at 12:12 PM Aaron Dixon <atdi...@gmail.com> wrote: >> > >> > @Tomo Suzuki Thanks for looking at this and your thorough analysis// let >> > me know if I can help with any regression testing, CRs, or more context >> > anything else. >> > >> > --- >> > >> > @Elliotte I was a little confused at first re: vendoring vs shading but >> > here is how I understand it (happy to be corrected if anything I say here >> > is a mischaracterization); I tend to over-explain for my own clarity >> > fwiw--using Guava as an example: >> > >> > With SHADING you have your project (e.g., Beam) depend directly on Guava >> > and its public package names. Your build will then compile and link to >> > these package names and only after-the-fact [1] rewrite the byte-code to >> > rename the guava package. This entails both byte-code rewriting the Guava >> > class files (to rename the packages of the Guava code itself) and >> > byte-code rewriting all of your project class files that import Guava (to >> > change the import to use the new package name). >> > >> > With VENDORING you release a renamed Guava ("vendor") JAR and depend on >> > that directly from your project. So your project's source code itself >> > imports the (repackated) vendor JAR and depends on the renamed packages >> > directly. Your project build then has no need for shading or byte code >> > rewriting of its own classes. I see vendoring making it very clear what is >> > going on when looking at a project's source code and its dependencies, and >> > there are likely other tangible brass-tacks benefits to vendoring besides >> > the conceptual clarity over shading. >> > >> > [1] You can see that Maven shade:shade goal binds by default to Maven's >> > package phase (which is well after after compilation). >> > >> > On Fri, Jan 17, 2020 at 9:16 AM Tomo Suzuki <suzt...@google.com> wrote: >> >> >> >> Ismaël and I think the short-term solution for Aaron's issue is to >> >> define Beam's own class for Avro's TimestampConversion. Luckily it's >> >> the only blocker for Avro 1.9 I found. >> >> I created a ticket https://issues.apache.org/jira/browse/BEAM-9144 and >> >> PR https://github.com/apache/beam/pull/10628 . >> >> >> >> Regards, >> >> Tomo >> >> >> >> On Fri, Jan 17, 2020 at 7:32 AM Elliotte Rusty Harold >> >> <elh...@ibiblio.org> wrote: >> >> > >> >> > On Thu, Jan 16, 2020 at 7:08 PM Kenneth Knowles <k...@apache.org> wrote: >> >> > > >> >> > > Regarding Google's advice about shading: don't go to a "one version >> >> > > rule" monorepo for advice about solving diamond dependencies in the >> >> > > wild. >> >> > >> >> > FWIW these docs came out of the open source, Github, multirepo, many >> >> > versions part of Google Cloud, not the mono repo world of google3. >> >> > Though I will say the more time I spend trying to deal with these >> >> > issues the more I love the monorepo. >> >> > >> >> > > It is a useful description of the pitfalls. We (and Flink before us, >> >> > > and likely many more) are already doing something that avoids many of >> >> > > them or makes them less likely. Building a separate vendored library >> >> > > is simpler and more robust than the "shade during build". >> >> > > >> >> > >> >> > I've only encountered "vendoring" as distinct from shading in Beam. >> >> > Are there more details about this anywhere? A quick Google search only >> >> > turned up one Beam doc that pointed to Go docs and a cloud endpoints >> >> > doc that seemed to use it as a synonym for shading. >> >> > >> >> > >> >> > -- >> >> > Elliotte Rusty Harold >> >> > elh...@ibiblio.org >> >> >> >> >> >> >> >> -- >> >> Regards, >> >> Tomo >> >> >> >> -- >> Elliotte Rusty Harold >> elh...@ibiblio.org -- Elliotte Rusty Harold elh...@ibiblio.org
