Re: [Discuss] Merge spark-3 branch into master

2020-04-21 Thread Saisai Shao
; > >>>>>>>> different versions of same dependencies could be error prone. > But I think> > > >>>>>>>> the case here should not lead to issue:> > > >>>>>>>>> > > >>>>>&

Re: [Discuss] Merge spark-3 branch into master

2020-03-26 Thread Saisai Shao
>> Saisai >>>>>>>> >>>>>>>> Saisai Shao 于2020年3月4日周三 上午10:01写道: >>>>>>>> >>>>>>>>> Thanks Matt, >>>>>>>>> >>>>>>>>> If branching is the only

Re: [Discuss] Merge spark-3 branch into master

2020-03-26 Thread Ryan Blue
gt;>>>>>> *master* branches until spark-3 is vastly adopted. That will somehow >>>>>>>> increase the maintenance burden and lead to inconsistency. IMO I'm OK >>>>>>>> with >>>>>>>> the branching way, just

Re: [Discuss] Merge spark-3 branch into master

2020-03-26 Thread Mass Dosage
gt;>>>> the branching way, just think that we should have a clear way to keep >>>>>>> tracking of two branches. >>>>>>> >>>>>>> Best, >>>>>>> Saisai >>>>>>> >>>>>

Re: [Discuss] Merge spark-3 branch into master

2020-03-25 Thread Saisai Shao
50写道: >>>>>> >>>>>>> I think it’s generally dangerous and error-prone to try to support >>>>>>> two versions of the same library in the same build, in the same >>>>>>> published >>>>>>> artifacts. This i

Re: [Discuss] Merge spark-3 branch into master

2020-03-17 Thread Anton Okolnychyi
he best way to build and publish > against multiple versions of a dependency. > > > > -Matt Cheah > > > > From: Saisai Shao mailto:sai.sai.s...@gmail.com>> > Reply-To: "dev@iceberg.apache.org <mailto:dev@iceberg.apache.org>" > ma

Re: [Discuss] Merge spark-3 branch into master

2020-03-07 Thread RD
dangerous and error-prone to try to support >>>>>> two versions of the same library in the same build, in the same published >>>>>> artifacts. This is the stance that Baseline >>>>>> <https://github.com/palantir/gradle-baseline> + Gradle Consistent

Re: [Discuss] Merge spark-3 branch into master

2020-03-05 Thread Saisai Shao
e Consistent Versions is specifically opinionated towards >>>>> building against one version of a library across all modules in the build. >>>>> >>>>> >>>>> >>>>> I would think that branching would be the best way to build

Re: [Discuss] Merge spark-3 branch into master

2020-03-04 Thread Saisai Shao
gt;>>> building against one version of a library across all modules in the build. >>>> >>>> >>>> >>>> I would think that branching would be the best way to build and publish >>>> against multiple versions of a dependency. &

Re: [Discuss] Merge spark-3 branch into master

2020-03-04 Thread Ryan Blue
the build. >>> >>> >>> >>> I would think that branching would be the best way to build and publish >>> against multiple versions of a dependency. >>> >>> >>> >>> -Matt Cheah >>> >>> >>> >&g

Re: [Discuss] Merge spark-3 branch into master

2020-03-04 Thread Saisai Shao
*From: *Saisai Shao >> *Reply-To: *"dev@iceberg.apache.org" >> *Date: *Tuesday, March 3, 2020 at 5:45 PM >> *To: *Iceberg Dev List >> *Cc: *Ryan Blue >> *Subject: *Re: [Discuss] Merge spark-3 branch into master >> >> >> >> I didn't realiz

Re: [Discuss] Merge spark-3 branch into master

2020-03-03 Thread Saisai Shao
> > I would think that branching would be the best way to build and publish > against multiple versions of a dependency. > > > > -Matt Cheah > > > > *From: *Saisai Shao > *Reply-To: *"dev@iceberg.apache.org" > *Date: *Tuesday, March 3, 2020 at 5:45

Re: [Discuss] Merge spark-3 branch into master

2020-03-03 Thread Matt Cheah
-Matt Cheah From: Saisai Shao Reply-To: "dev@iceberg.apache.org" Date: Tuesday, March 3, 2020 at 5:45 PM To: Iceberg Dev List Cc: Ryan Blue Subject: Re: [Discuss] Merge spark-3 branch into master I didn't realized that Gradle cannot support two different versions in one build. I t

Re: [Discuss] Merge spark-3 branch into master

2020-03-03 Thread Saisai Shao
I didn't realized that Gradle cannot support two different versions in one build. I think I did such things for Livy to build scala 2.10 and 2.11 jars simultaneously with Maven. I'm not so familiar with Gradle thing, I can take a shot to see if there's some hacky ways to make it work. Besides,

Re: [Discuss] Merge spark-3 branch into master

2020-03-03 Thread Mass Dosage
+1 for a 0.8.0 release with Spark 2.4 and then move on for Spark 3.0 when it's ready. On Tue, 3 Mar 2020 at 16:32, Ryan Blue wrote: > Thanks for bringing this up, Saisai. I tried to do this a couple of months > ago, but ran into a problem with dependency locks. I couldn't get two > different

Re: [Discuss] Merge spark-3 branch into master

2020-03-03 Thread Matt Cheah
t;dev@iceberg.apache.org" , "rb...@netflix.com" Date: Tuesday, March 3, 2020 at 8:32 AM To: Iceberg Dev List Subject: Re: [Discuss] Merge spark-3 branch into master Thanks for bringing this up, Saisai. I tried to do this a couple of months ago, but ran into a problem with dependency locks

Re: [Discuss] Merge spark-3 branch into master

2020-03-03 Thread Ryan Blue
Thanks for bringing this up, Saisai. I tried to do this a couple of months ago, but ran into a problem with dependency locks. I couldn't get two different versions of Spark packages in the build with baseline, but maybe I was missing something. If you can get it working, I think it's a great idea

[Discuss] Merge spark-3 branch into master

2020-03-03 Thread Saisai Shao
Hi team, I was thinking of merging spark-3 branch into master, also per the discussion before we could make spark-2 and spark-3 coexisted into 2 different sub-modules. With this, one build could generate both spark-2 and spark-3 runtime jars, user could pick either at preference. One concern is