Re: [VOTE] Release Apache Spark 2.4.1 (RC6)

Which version of Parquet has this bug? Maybe we can downgrade it. On Mon, Mar 11, 2019 at 10:34 AM Mark Hamstra wrote: > It worked in 2.3. We broke it with 2.4.0 and were informed of that > regression late in the 2.4.0 release process. Since we didn't fix it before > the 2.4.0 release, it

Re: [VOTE] Release Apache Spark 2.4.1 (RC6)

It worked in 2.3. We broke it with 2.4.0 and were informed of that regression late in the 2.4.0 release process. Since we didn't fix it before the 2.4.0 release, it should have been noted as a known issue. To now claim that there is no regression from 2.4.0 is a circular argument denying the

Re: [VOTE] Release Apache Spark 2.4.1 (RC6)

I'm not saying that this issue should be a blocker for 2.4.1, rather I'm looking for help moving things along. I'm not a committer in any of the Spark, Parquet, or Avro projects. > On Mar 10, 2019, at 8:53 PM, Sean Owen wrote: > > From https://issues.apache.org/jira/browse/SPARK-25588, I'm

Re: [VOTE] Release Apache Spark 2.4.1 (RC6)

>From https://issues.apache.org/jira/browse/SPARK-25588, I'm reading that: - this is a Parquet-Avro version conflict thing - a downstream app wants different versions of Parquet and Avro than Spark uses, which triggers it - it doesn't work in 2.4.0 It's not a regression from 2.4.0, which is the

Re: [VOTE] Release Apache Spark 2.4.1 (RC6)

Now wait... we created a regression in 2.4.0. Arguably, we should have blocked that release until we had a fix; but the issue came up late in the release process and it looks to me like there wasn't an adequate fix immediately available, so we did something bad and released 2.4.0 with a known

Re: [VOTE] Release Apache Spark 2.4.1 (RC6)

As we have many important fixes in 2.4 branch which we want to release asap, and this is is not a regression from Spark 2.4; as a result, 2.4.1 will be not blocked by this. Sincerely, DB Tsai -- Web: https://www.dbtsai.com PGP Key ID:

Re: [VOTE] Release Apache Spark 2.4.1 (RC6)

Any chance we could get some movement on this for 2.4.1? https://issues.apache.org/jira/browse/SPARK-25588 https://github.com/apache/parquet-mr/pull/560 It would require a new Parquet release,

Re: [VOTE] Release Apache Spark 2.4.1 (RC6)

Since I can not find the commit of `Preparing development version 2.4.2-SNAPSHOT` after rc6 cut, it's very risky to fix the branch and do a force-push. I'll follow Marcelo's suggestion to have another rc7 cut. Thus, this vote fails. DB Tsai | Siri Open Source Technologies [not a

Re: [VOTE] Release Apache Spark 2.4.1 (RC6)

Okay, I see the problem. rc6 tag is not in the 2.4 branch. It's very weird. It must be overwritten by a force push. DB Tsai | Siri Open Source Technologies [not a contribution] |  Apple, Inc > On Mar 8, 2019, at 11:39 AM, DB Tsai wrote: > > I was using `./do-release-docker.sh` to create

Re: [VOTE] Release Apache Spark 2.4.1 (RC6)

I'd be more comfortable with an rc7. Either that or manually fix the branch with a force push, but that's a bit risky, it's easy to mess up force pushes (if we can even do that?). It's very possible that there is a bug in the script; IIRC it should create the commits in the right branch when you

Re: [VOTE] Release Apache Spark 2.4.1 (RC6)

I was using `./do-release-docker.sh` to create a release. But since the gpg validation fails couple times when the script tried to publish the jars into Nexus, I re-ran the scripts multiple times without creating a new rc. I was wondering if the script will overwrite the v.2.4.1-rc6 tag instead

Re: [VOTE] Release Apache Spark 2.4.1 (RC6)

I personally find it a little weird to not have the commit in branch-2.4. Not that this would happen, but if the v2.4.1-rc6 tag is overwritten (e.g. accidentally) then you lose the reference to that commit, and then the exact commit from which the rc was generated is lost. On Fri, Mar 8, 2019 at

Re: [VOTE] Release Apache Spark 2.4.1 (RC6)

That's weird. I see the commit but can't find it in the branch. Was it pushed, or lost in a force push of 2.4 along the way? The change is there, just under a different commit in the 2.4 branch. It doesn't necessarily invalidate the RC as it is a valid public tagged commit and all that. I just

Re: [VOTE] Release Apache Spark 2.4.1 (RC6)

Hi, I am not sure how problematic it is but v2.4.1-rc6 is not on branch-2.4. Release related commits I have seen so far were also part of the branch. I guess the "Preparing Spark release v2.4.1-rc6" and "Preparing development version 2.4.2-SNAPSHOT" commits were simply not pushed to spark-2.4

[VOTE] Release Apache Spark 2.4.1 (RC6)

Please vote on releasing the following candidate as Apache Spark version 2.4.1. The vote is open until March 11 PST and passes if a majority +1 PMC votes are cast, with a minimum of 3 +1 votes. [ ] +1 Release this package as Apache Spark 2.4.1 [ ] -1 Do not release this package because ... To