Protobuf provides Wire compatibility between releases.. but not guarantees the source compatibility in generated sources. There will be a problem in compatibility if anyone uses generated protobuf message outside of Hadoop modules. Which ideally shouldn't be as generated sources are not public APIs.
There should not be any compatibility problems between releases in terms of communication provided both uses same syntax (proto2) of proto message. This I have verified by communication between protobuf 2.5.0 client with protobuf 3.7.1 server. To avoid the downstream transitive dependency classpath problem, who might be using protobuf 2.5.0 classes, planning to shade the 3.7.1 classes and its usages in all hadoop modules.. and keep 2.5.0 jar back in hadoop classpath. Hope I have answered your question. -Vinay On Sun, 22 Sep 2019, 7:38 pm Vinod Kumar Vavilapalli, <vino...@apache.org> wrote: > Quick question, being lazy here, lots of JIRA updates on HADOOP-13363 over > the years not helping either. > > Does anyone know what this upgrade will mean w.r.t compatibility for the > Hadoop releases themselves? Remember that trunk is still 3.x. > > Thanks > +Vinod > > > On Sep 21, 2019, at 9:55 AM, Vinayakumar B <vinayakum...@apache.org> > wrote: > > > > @Wangda Tan <wheele...@gmail.com> , > > Sorry for the confusion. HADOOP-13363 is umbrella jira to track multiple > > stages of protobuf upgrade in subtasks. (jar upgrade, Docker update, > plugin > > upgrade, shading, etc). > > Right now, first task of jar upgrade is done. So need to update the > protoc > > executable in the in build environments. > > > > @张铎(Duo Zhang) <palomino...@gmail.com> , > > Sorry for the inconvenience. Yes, indeed plugin update before jar upgrde > > was possible. Sorry I missed it. > > > > Plugin update needed to be done for whole project, for which precommit > > jenkins will need more time complete end-to-end runs. > > So plugin update is planned in stages in further subtasks. It could be > done > > in 2-3 days. > > > > -Vinay > > > > On Sat, 21 Sep 2019, 5:55 am 张铎(Duo Zhang), <palomino...@gmail.com> > wrote: > > > >> I think this one is alread in place so we have to upgrade... > >> > >> https://issues.apache.org/jira/browse/HADOOP-16557 > >> > >> Wangda Tan <wheele...@gmail.com> 于2019年9月21日周六 上午7:19写道： > >> > >>> Hi Vinay, > >>> > >>> A bit confused, I saw the HADOOP-13363 is still pending. Do we need to > >>> upgrade protobuf version to 3.7.1 NOW or once HADOOP-13363 is > completed? > >>> > >>> Thanks, > >>> Wangda > >>> > >>> On Fri, Sep 20, 2019 at 8:11 AM Vinayakumar B <vinayakum...@apache.org > > > >>> wrote: > >>> > >>>> Hi All, > >>>> > >>>> A very long pending task, protobuf upgrade is happening in > >> HADOOP-13363. > >>> As > >>>> part of that protobuf version is upgraded to 3.7.1. > >>>> > >>>> Please update your build environments to have 3.7.1 protobuf version. > >>>> > >>>> BUILIDING.txt has been updated with latest instructions. > >>>> > >>>> This pre-requisite to update protoc dependecy manually is required > >> until > >>>> 'hadoop-maven-plugin' is replaced with 'protobuf-mavem-plugin' to > >>>> dynamically resolve required protoc exe. > >>>> > >>>> Dockerfile is being updated to have latest 3.7.1 as default protoc for > >>> test > >>>> environments. > >>>> > >>>> Thanks, > >>>> -Vinay > >>>> > >>> > >> > >
