aah, that makes sense On Sun, Sep 22, 2019 at 6:11 PM Vinayakumar B <vinayakum...@apache.org> wrote:
> Thanks Steve. > > Idea is not to shade all artifacts. > Instead maintain one artifact ( hadoop-thirdparty) which have all such > dependencies ( com.google.* may be), add this artifact as dependency in > hadoop modules. Use shaded classes directly in the code of hadoop modules > instead of shading at package phase. > > Hbase, ozone and ratis already following this way. The artifact ( > hadoop-thirdparty) with shaded dependencies can be maintained in a separate > repo as suggested by stack on HADOOP-13363 or could be maintained as a > separate module in Hadoop repo. If maintained in separate repo, need to > build this only when there are changes related to shaded dependencies. > > > -Vinay > > On Sun, 22 Sep 2019, 10:11 pm Steve Loughran, <ste...@cloudera.com> wrote: > > > > > > > On Sun, Sep 22, 2019 at 3:22 PM Vinayakumar B <vinayakum...@apache.org> > > wrote: > > > >> Protobuf provides Wire compatibility between releases.. but not > >> guarantees the source compatibility in generated sources. There will be > a > >> problem in compatibility if anyone uses generated protobuf message > outside > >> of Hadoop modules. Which ideally shouldn't be as generated sources are > not > >> public APIs. > >> > >> There should not be any compatibility problems between releases in > >> terms > >> of communication provided both uses same syntax (proto2) of proto > message. > >> This I have verified by communication between protobuf 2.5.0 client with > >> protobuf 3.7.1 server. > >> > >> To avoid the downstream transitive dependency classpath problem, who > >> might be using protobuf 2.5.0 classes, planning to shade the 3.7.1 > classes > >> and its usages in all hadoop modules.. and keep 2.5.0 jar back in hadoop > >> classpath. > >> > >> Hope I have answered your question. > >> > >> -Vinay > >> > >> > > While I support the move and CP isolation, this is going to (finally) > > force us to make shaded versions of all artifacts which we publish with > the > > intent of them being loaded on the classpath of other applications > > >
