Re: [ANNOUNCE] Announcing Spark 1.5.1

I think the key is to vote a specific set of source tarballs without any binary artifacts. The specific binaries are useful but shouldn't be part of the voting process. Makes sense, we really cannot prove (and no need to) that the binaries do not contain malware, but the source can be proven to

Re: [ANNOUNCE] Announcing Spark 1.5.1

I know there are multiple things being talked about here, but  I agree with Patrick here, we vote on the source distribution - src tarball (and of course the tag should match).  Perhaps in principle we vote on all the other specific binary distributions since they are generated from source

Re: [ANNOUNCE] Announcing Spark 1.5.1

Still confused. Why are you saying we didn't vote on an archive? refer to the email I linked, which includes both the git tag and a link to all generated artifacts (also in my email). So, there are two things at play here: First, I am not sure what you mean that a source distro can't have binary

Re: [ANNOUNCE] Announcing Spark 1.5.1

Out of curiosity: How can you vote on a release that contains 34 binary files? Surely a source code release should only contain source code and not binaries, as you cannot verify the content of these. Looking forward to a response. With regards, Daniel. On 10/2/2015, 4:42:31 AM, Reynold Xin

Re: [ANNOUNCE] Announcing Spark 1.5.1

The Spark releases include a source distribution and several binary distributions. This is pretty normal for Apache projects. What are you referring to here? On Sun, Oct 11, 2015 at 3:26 PM, Daniel Gruno wrote: > Out of curiosity: How can you vote on a release that contains

Re: [ANNOUNCE] Announcing Spark 1.5.1

On 10/11/2015 05:12 PM, Sean Owen wrote: > The Spark releases include a source distribution and several binary > distributions. This is pretty normal for Apache projects. What are you > referring to here? Surely the _source_ distribution does not contain binaries? How else can you vote on a

Re: [ANNOUNCE] Announcing Spark 1.5.1

Daniel: we did not vote on a tag. Please again read the VOTE email I linked to you: http://apache-spark-developers-list.1001551.n3.nabble.com/VOTE-Release-Apache-Spark-1-5-1-RC1-tt14310.html#none among other things, it contains a link to the concrete source (and binary) distribution under vote:

Re: [ANNOUNCE] Announcing Spark 1.5.1

You can find the source tagged for release on GitHub , as was clearly linked to in the thread to vote on the release (titled "[VOTE] Release Apache Spark 1.5.1 (RC1)"). Is there something about that thread that was unclear? Nick On Sun, Oct

Re: [ANNOUNCE] Announcing Spark 1.5.1

Of course, but what's making you think this was a binary-only distribution? The downloads page points you directly to the source distro: http://spark.apache.org/downloads.html Look for the last vote, and you'll find it was of course a vote on source (and binary) artifacts:

Re: [ANNOUNCE] Announcing Spark 1.5.1

On 10/11/2015 05:29 PM, Sean Owen wrote: > Of course, but what's making you think this was a binary-only > distribution? I'm not saying binary-only, I am saying your source release contains binary programs, which would invalidate a release vote. Is there a release candidate package, that is

Re: [ANNOUNCE] Announcing Spark 1.5.1

Here's my issue: How am I to audit that the dependencies you bundle are in fact what you claim they are? How do I know they don't contain malware or - in light of recent events - emissions test rigging? ;) I am not interested in a git tag - that means nothing in the ASF voting process, you

Re: [ANNOUNCE] Announcing Spark 1.5.1

Agree, but we are talking about the build/ bit right? I don't agree that it invalidates the release, which is probably the more important idea. As a point of process, you would not want to modify and republish the artifact that was already released after being voted on - unless it was invalid in

Re: [ANNOUNCE] Announcing Spark 1.5.1

I think Daniel is correct here. The source artifact incorrectly includes jars. It is inadvertent and not part of our intended release process. This was something I noticed in Spark 1.5.0 and filed a JIRA and was fixed by updating our build scripts to fix it. However, our build environment was not

Re: [ANNOUNCE] Announcing Spark 1.5.1

*to not include binaries. On Sun, Oct 11, 2015 at 9:35 PM, Patrick Wendell wrote: > I think Daniel is correct here. The source artifact incorrectly includes > jars. It is inadvertent and not part of our intended release process. This > was something I noticed in Spark 1.5.0

Re: [ANNOUNCE] Announcing Spark 1.5.1

Oh I see - yes it's the build/. I always thought release votes related to a source tag rather than specific binaries. But maybe we can just fix it in 1.5.2 if there is concern about mutating binaries. It seems reasonable to me. For tests... in the past we've tried to avoid having jars inside of

Re: [ANNOUNCE] Announcing Spark 1.5.1

Yeah I mean I definitely think we're not violating the *spirit* of the "no binaries" policy, in that we do not include any binary code that is used at runtime. This is because the binaries we distribute relate only to build and testing. Whether we are violating the *letter* of the policy, I'm not

Re: [ANNOUNCE] Announcing Spark 1.5.1

No we are voting on the artifacts being released (too) in principle. Although of course the artifacts should be a deterministic function of the source at a certain point in time. I think the concern is about putting Spark binaries or its dependencies into a source release. That should not happen,

[ANNOUNCE] Announcing Spark 1.5.1

Hi All, Spark 1.5.1 is a maintenance release containing stability fixes. This release is based on the branch-1.5 maintenance branch of Spark. We *strongly recommend* all 1.5.0 users to upgrade to this release. The full list of bug fixes is here: http://s.apache.org/spark-1.5.1