Thanks Matt, If branching is the only choice, then we would potentially have two *master* branches until spark-3 is vastly adopted. That will somehow increase the maintenance burden and lead to inconsistency. IMO I'm OK with the branching way, just think that we should have a clear way to keep tracking of two branches.
Best, Saisai Matt Cheah <mch...@palantir.com.invalid> 于2020年3月4日周三 上午9:50写道： > I think it’s generally dangerous and error-prone to try to support two > versions of the same library in the same build, in the same published > artifacts. This is the stance that Baseline > <https://github.com/palantir/gradle-baseline> + Gradle Consistent Versions > <https://github.com/palantir/gradle-consistent-versions> takes. Gradle > Consistent Versions is specifically opinionated towards building against > one version of a library across all modules in the build. > > > > I would think that branching would be the best way to build and publish > against multiple versions of a dependency. > > > > -Matt Cheah > > > > *From: *Saisai Shao <sai.sai.s...@gmail.com> > *Reply-To: *"dev@iceberg.apache.org" <dev@iceberg.apache.org> > *Date: *Tuesday, March 3, 2020 at 5:45 PM > *To: *Iceberg Dev List <dev@iceberg.apache.org> > *Cc: *Ryan Blue <rb...@netflix.com> > *Subject: *Re: [Discuss] Merge spark-3 branch into master > > > > I didn't realized that Gradle cannot support two different versions in one > build. I think I did such things for Livy to build scala 2.10 and 2.11 jars > simultaneously with Maven. I'm not so familiar with Gradle thing, I can > take a shot to see if there's some hacky ways to make it work. > > > > Besides, are we saying that we will move to spark-3 support after 0.8 > release in the master branch to replace Spark-2, or we maintain two > branches for both spark-2 and spark-3 and make two releases? From > my understanding, the adoption of spark-3 may not be so fast, and there > still has lots users who stick on spark-2. Ideally, it might be better to > support two versions in a near future. > > > > Thanks > > Saisai > > > > > > > > Mass Dosage <massdos...@gmail.com> 于2020年3月4日周三 上午1:33写道： > > +1 for a 0.8.0 release with Spark 2.4 and then move on for Spark 3.0 when > it's ready. > > > > On Tue, 3 Mar 2020 at 16:32, Ryan Blue <rb...@netflix.com.invalid> wrote: > > Thanks for bringing this up, Saisai. I tried to do this a couple of months > ago, but ran into a problem with dependency locks. I couldn't get two > different versions of Spark packages in the build with baseline, but maybe > I was missing something. If you can get it working, I think it's a great > idea to get this into master. > > > > Otherwise, I was thinking about proposing an 0.8.0 release in the next > month or so based on Spark 2.4. Then we could merge the branch into master > and do another release for Spark 3.0 when it's ready. > > > > rb > > > > On Tue, Mar 3, 2020 at 6:07 AM Saisai Shao <sai.sai.s...@gmail.com> wrote: > > Hi team, > > > > I was thinking of merging spark-3 branch into master, also per the > discussion before we could make spark-2 and spark-3 coexisted into 2 > different sub-modules. With this, one build could generate both spark-2 and > spark-3 runtime jars, user could pick either at preference. > > > > One concern is that they share lots of common code in read/write path, > this will increase the maintenance overhead to keep consistency of two > copies. > > > > So I'd like to hear your thoughts, any suggestions on it? > > > > Thanks > > Saisai > > > > > -- > > Ryan Blue > > Software Engineer > > Netflix > >
