*to not include binaries. On Sun, Oct 11, 2015 at 9:35 PM, Patrick Wendell <pwend...@gmail.com> wrote:
> I think Daniel is correct here. The source artifact incorrectly includes > jars. It is inadvertent and not part of our intended release process. This > was something I noticed in Spark 1.5.0 and filed a JIRA and was fixed by > updating our build scripts to fix it. However, our build environment was > not using the most current version of the build scripts. See related links: > > https://issues.apache.org/jira/browse/SPARK-10511 > https://github.com/apache/spark/pull/8774/files > > I can update our build environment and we can repackage the Spark 1.5.1 > source tarball. To not include sources. > > - Patrick > > On Sun, Oct 11, 2015 at 8:53 AM, Sean Owen <so...@cloudera.com> wrote: > >> Daniel: we did not vote on a tag. Please again read the VOTE email I >> linked to you: >> >> >> http://apache-spark-developers-list.1001551.n3.nabble.com/VOTE-Release-Apache-Spark-1-5-1-RC1-tt14310.html#none >> >> among other things, it contains a link to the concrete source (and >> binary) distribution under vote: >> >> http://people.apache.org/~pwendell/spark-releases/spark-1.5.1-rc1-bin/ >> >> You can still examine it, sure. >> >> Dependencies are *not* bundled in the source release. You're again >> misunderstanding what you are seeing. Read my email again. >> >> I am still pretty confused about what the problem is. This is entirely >> business as usual for ASF projects. I'll follow up with you offline if >> you have any more doubts. >> >> On Sun, Oct 11, 2015 at 4:49 PM, Daniel Gruno <humbed...@apache.org> >> wrote: >> > Here's my issue: >> > >> > How am I to audit that the dependencies you bundle are in fact what you >> > claim they are? How do I know they don't contain malware or - in light >> > of recent events - emissions test rigging? ;) >> > >> > I am not interested in a git tag - that means nothing in the ASF voting >> > process, you cannot vote on a tag, only on a release candidate. The VCS >> > in use is irrelevant in this issue. If you can point me to a release >> > candidate archive that was voted upon and does not contain binary >> > applications, all is well. >> > >> > If there is no such thing, and we cannot come to an understanding, I >> > will exercise my ASF Members' rights and bring this to the attention of >> > the board of directors and ask for a clarification of the legality of >> this. >> > >> > I find it highly irregular. Perhaps it is something some projects do in >> > the Java community, but that doesn't make it permissible in my view. >> > >> > With regards, >> > Daniel. >> > >> > >> > On 10/11/2015 05:42 PM, Sean Owen wrote: >> >> Still confused. Why are you saying we didn't vote on an archive? refer >> >> to the email I linked, which includes both the git tag and a link to >> >> all generated artifacts (also in my email). >> >> >> >> So, there are two things at play here: >> >> >> >> First, I am not sure what you mean that a source distro can't have >> >> binary files. It's supposed to have the source code of Spark, and >> >> shouldn't contain binary Spark. Nothing you listed are Spark binaries. >> >> However, a distribution might have a lot of things in it that support >> >> the source build, like copies of tools, test files, etc. That >> >> explains I think the first couple lines that you identified. >> >> >> >> Still, I am curious why you are saying that would invalidate a source >> >> release? I have never heard anything like that. >> >> >> >> Second, I do think there are some binaries in here that aren't >> >> supposed to be there, like the build/ directory stuff. IIRC these were >> >> included accidentally and won't be in the next release. At least, I >> >> don't see why they need to be bundled. These are just local copies of >> >> third party tools though, and don't really matter. As it happens, the >> >> licenses that get distributed with the source distro even cover all of >> >> this stuff. I think that's not supposed to be there, but, also don't >> >> see it's 'invalid' as a result. >> >> >> >> >> >> On Sun, Oct 11, 2015 at 4:33 PM, Daniel Gruno <humbed...@apache.org> >> wrote: >> >>> On 10/11/2015 05:29 PM, Sean Owen wrote: >> >>>> Of course, but what's making you think this was a binary-only >> >>>> distribution? >> >>> >> >>> I'm not saying binary-only, I am saying your source release contains >> >>> binary programs, which would invalidate a release vote. Is there a >> >>> release candidate package, that is voted on (saying you have a git tag >> >>> does not satisfy this criteria, you need to vote on an actual archive >> of >> >>> files, otherwise there is no cogent proof of the release being from >> that >> >>> specific git tag). >> >>> >> >>> Here's what I found in your source release: >> >>> >> >>> Binary application (application/jar; charset=binary) found in >> >>> spark-1.5.1/sql/hive/src/test/resources/data/files/TestSerDe.jar >> >>> >> >>> Binary application (application/jar; charset=binary) found in >> >>> >> spark-1.5.1/sql/hive/src/test/resources/regression-test-SPARK-8489/test.jar >> >>> >> >>> Binary application (application/jar; charset=binary) found in >> >>> spark-1.5.1/sql/hive/src/test/resources/TestUDTF.jar >> >>> >> >>> Binary application (application/jar; charset=binary) found in >> >>> spark-1.5.1/R/pkg/inst/test_support/sparktestjar_2.10-1.0.jar >> >>> >> >>> Binary application (application/jar; charset=binary) found in >> >>> spark-1.5.1/build/zinc-0.3.5.3/lib/scala-reflect.jar >> >>> >> >>> Binary application (application/jar; charset=binary) found in >> >>> spark-1.5.1/build/zinc-0.3.5.3/lib/sbt-interface.jar >> >>> >> >>> Binary application (application/jar; charset=binary) found in >> >>> spark-1.5.1/build/zinc-0.3.5.3/lib/compiler-interface-sources.jar >> >>> >> >>> Binary application (application/jar; charset=binary) found in >> >>> spark-1.5.1/build/zinc-0.3.5.3/lib/incremental-compiler.jar >> >>> >> >>> Binary application (application/jar; charset=binary) found in >> >>> spark-1.5.1/build/zinc-0.3.5.3/lib/scala-compiler.jar >> >>> >> >>> Binary application (application/jar; charset=binary) found in >> >>> spark-1.5.1/build/zinc-0.3.5.3/lib/zinc.jar >> >>> >> >>> Binary application (application/jar; charset=binary) found in >> >>> spark-1.5.1/build/zinc-0.3.5.3/lib/scala-library.jar >> >>> >> >>> Binary application (application/jar; charset=binary) found in >> >>> >> spark-1.5.1/build/scala-2.10.4/misc/scala-devel/plugins/continuations.jar >> >>> >> >>> Binary application (application/jar; charset=binary) found in >> >>> spark-1.5.1/build/scala-2.10.4/lib/scala-reflect.jar >> >>> >> >>> Binary application (application/jar; charset=binary) found in >> >>> spark-1.5.1/build/scala-2.10.4/lib/akka-actors.jar >> >>> >> >>> Binary application (application/jar; charset=binary) found in >> >>> spark-1.5.1/build/scala-2.10.4/lib/typesafe-config.jar >> >>> >> >>> Binary application (application/jar; charset=binary) found in >> >>> spark-1.5.1/build/scala-2.10.4/lib/scala-actors-migration.jar >> >>> >> >>> Binary application (application/jar; charset=binary) found in >> >>> spark-1.5.1/build/scala-2.10.4/lib/scala-actors.jar >> >>> >> >>> Binary application (application/jar; charset=binary) found in >> >>> spark-1.5.1/build/scala-2.10.4/lib/scalap.jar >> >>> >> >>> Binary application (application/jar; charset=binary) found in >> >>> spark-1.5.1/build/scala-2.10.4/lib/scala-swing.jar >> >>> >> >>> Binary application (application/jar; charset=binary) found in >> >>> spark-1.5.1/build/scala-2.10.4/lib/scala-compiler.jar >> >>> >> >>> Binary application (application/jar; charset=binary) found in >> >>> spark-1.5.1/build/scala-2.10.4/lib/scala-library.jar >> >>> >> >>> Binary application (application/jar; charset=binary) found in >> >>> spark-1.5.1/build/scala-2.10.4/src/scala-reflect-src.jar >> >>> >> >>> Binary application (application/jar; charset=binary) found in >> >>> spark-1.5.1/build/scala-2.10.4/src/scala-swing-src.jar >> >>> >> >>> Binary application (application/jar; charset=binary) found in >> >>> spark-1.5.1/build/scala-2.10.4/src/scalap-src.jar >> >>> >> >>> Binary application (application/jar; charset=binary) found in >> >>> spark-1.5.1/build/scala-2.10.4/src/scala-actors-src.jar >> >>> >> >>> Binary application (application/jar; charset=binary) found in >> >>> spark-1.5.1/build/scala-2.10.4/src/scala-partest-src.jar >> >>> >> >>> Binary application (application/jar; charset=binary) found in >> >>> spark-1.5.1/build/scala-2.10.4/src/scala-library-src.jar >> >>> >> >>> Binary application (application/jar; charset=binary) found in >> >>> spark-1.5.1/build/scala-2.10.4/src/fjbg-src.jar >> >>> >> >>> Binary application (application/jar; charset=binary) found in >> >>> spark-1.5.1/build/scala-2.10.4/src/scala-compiler-src.jar >> >>> >> >>> Binary application (application/jar; charset=binary) found in >> >>> spark-1.5.1/build/scala-2.10.4/src/msil-src.jar >> >>> >> >>> Binary application (application/jar; charset=binary) found in >> >>> spark-1.5.1/build/apache-maven-3.3.3/boot/plexus-classworlds-2.5.2.jar >> >>> >> >>> Binary application (application/jar; charset=binary) found in >> >>> spark-1.5.1/build/apache-maven-3.3.3/lib/guava-18.0.jar >> >>> >> >>> Binary application (application/jar; charset=binary) found in >> >>> spark-1.5.1/build/apache-maven-3.3.3/lib/wagon-http-2.9-shaded.jar >> >>> >> >>> Binary application (application/jar; charset=binary) found in >> >>> spark-1.5.1/build/apache-maven-3.3.3/lib/jsr250-api-1.0.jar >> >>> >> >>> Binary application (application/jar; charset=binary) found in >> >>> spark-1.5.1/build/apache-maven-3.3.3/lib/javax.inject-1.jar >> >>> >> >>> >> >>> >> >>> The downloads page points you directly to the source >> >>>> distro: http://spark.apache.org/downloads.html >> >>>> >> >>>> Look for the last vote, and you'll find it was of course a vote on >> >>>> source (and binary) artifacts: >> >>>> >> http://apache-spark-developers-list.1001551.n3.nabble.com/VOTE-Release-Apache-Spark-1-5-1-RC1-tt14310.html#none >> >>>> >> http://people.apache.org/~pwendell/spark-releases/spark-1.5.1-rc1-bin/ >> >>>> >> >>>> On Sun, Oct 11, 2015 at 4:23 PM, Daniel Gruno <humbed...@apache.org> >> wrote: >> >>>>> On 10/11/2015 05:12 PM, Sean Owen wrote: >> >>>>>> The Spark releases include a source distribution and several binary >> >>>>>> distributions. This is pretty normal for Apache projects. What are >> you >> >>>>>> referring to here? >> >>>>> >> >>>>> Surely the _source_ distribution does not contain binaries? How >> else can >> >>>>> you vote on a release if you don't know what it contains? >> >>>>> >> >>>>> You can produce convenience downloads that contain binary files, >> yes, >> >>>>> but surely you need a source-only package which is the one you vote >> on, >> >>>>> that does not contain any binaries. Do you have such a thing? And >> where >> >>>>> may I find it? >> >>>>> >> >>>>> With regards, >> >>>>> Daniel. >> >>>>> >> >>>>>> >> >>>>>> On Sun, Oct 11, 2015 at 3:26 PM, Daniel Gruno < >> humbed...@apache.org> wrote: >> >>>>>>> Out of curiosity: How can you vote on a release that contains 34 >> binary files? Surely a source code release should only contain source code >> and not binaries, as you cannot verify the content of these. >> >>>>>>> >> >>>>>>> Looking forward to a response. >> >>>>>>> >> >>>>>>> With regards, >> >>>>>>> Daniel. >> >>>>>>> >> >>>>>>> On 10/2/2015, 4:42:31 AM, Reynold Xin <r...@databricks.com> >> wrote: >> >>>>>>>> Hi All, >> >>>>>>>> >> >>>>>>>> Spark 1.5.1 is a maintenance release containing stability fixes. >> This >> >>>>>>>> release is based on the branch-1.5 maintenance branch of Spark. >> We >> >>>>>>>> *strongly recommend* all 1.5.0 users to upgrade to this release. >> >>>>>>>> >> >>>>>>>> The full list of bug fixes is here: >> http://s.apache.org/spark-1.5.1 >> >>>>>>>> >> >>>>>>>> http://spark.apache.org/releases/spark-release-1-5-1.html >> >>>>>>>> >> >>>>>>>> >> >>>>>>>> (note: it can take a few hours for everything to be propagated, >> so you >> >>>>>>>> might get 404 on some download links, but everything should be >> in maven >> >>>>>>>> central already) >> >>>>>>>> >> >>>>>>> >> >>>>>>> >> --------------------------------------------------------------------- >> >>>>>>> To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org >> >>>>>>> For additional commands, e-mail: dev-h...@spark.apache.org >> >>>>>>> >> >>>>> >> >>> >> > >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org >> For additional commands, e-mail: dev-h...@spark.apache.org >> >> >
