I'm cancelling this vote in favor of RC3.
On Sun, Dec 8, 2013 at 9:13 PM, Patrick Wendell <pwend...@gmail.com> wrote: > Okay Mark thanks for bringing this up, I'm going to cut a new RC with this > fix. > > On Sun, Dec 8, 2013 at 8:42 PM, Mark Hamstra <m...@clearstorydata.com> wrote: >> And I think 195 is sufficient to build something that works; but I haven't >> personally tested it. >> >> >> On Sun, Dec 8, 2013 at 8:41 PM, Mark Hamstra <m...@clearstorydata.com>wrote: >> >>> Well, what I've already done for ClearStory is very close to how Debian >>> packaging should be done for Apache Spark. That much can be put into a >>> pull request quickly. The only real issues are exactly how the packages >>> should be named, checking that the metadata of the packages are exactly >>> correct for an Apache release, and deciding whether we should be producing >>> a spark-examples package, spark-tools package, separate the Java and Python >>> APIs into their own packages, create a source package, etc. What we've >>> been doing up to now is essentially just the minimal packaging of a fat jar >>> that can be deployed by Chef or something similar. It's never really been >>> put together in a way appropriate to go into a Debian or Ubuntu >>> distribution, for instance. >>> >>> >>> On Sun, Dec 8, 2013 at 8:25 PM, Patrick Wendell <pwend...@gmail.com>wrote: >>> >>>> Hey Mark, >>>> >>>> Okay if 195 gets this in working order in the branch 0.8 let's just >>>> merge that to keep it consistent with our docs and the way this is >>>> done in 0.8.0 >>>> >>>> We can do a broader refactoring in 0.9. Would be great if you could >>>> kick off a JIRA discussion or submit a PR relating to that. >>>> >>>> - Patrick >>>> >>>> On Sun, Dec 8, 2013 at 8:07 PM, Mark Hamstra <m...@clearstorydata.com> >>>> wrote: >>>> > Well, 195 is sufficient to give you something that runs, but it doesn't >>>> run >>>> > the same way as Spark built/distributed by other means -- e.g., after >>>> 195 >>>> > the package still uses something equivalent to the old `run` script >>>> instead >>>> > of the current `spark-class` way. >>>> > >>>> > >>>> > On Sun, Dec 8, 2013 at 8:02 PM, Patrick Wendell <pwend...@gmail.com> >>>> wrote: >>>> > >>>> >> Hey Mark, >>>> >> >>>> >> What I'm asking is whether this patch is sufficient to have a working >>>> >> debian build in 0.8.1, or are there other outstanding issues to make >>>> >> it work? By working I mean, within the initial design that was >>>> >> contributed (with repl-bin) it works according to that approach. >>>> >> >>>> >> We can redesign this packaging in 0.9. That will require having a PR >>>> >> against Apache Spark, discussing, etc. But it doesn't need to be on >>>> >> the critical path for this release. >>>> >> >>>> >> - Patrick >>>> >> >>>> >> On Sun, Dec 8, 2013 at 7:54 PM, Mark Hamstra <m...@clearstorydata.com> >>>> >> wrote: >>>> >> > Whatever Debian package gets built has to work, so that's the first >>>> >> > requirement. I don't know how to decide whether a change is >>>> acceptable >>>> >> in >>>> >> > 0.8 or has to wait until 0.9, but the 0.9 packaging should definitely >>>> >> > leverage the assembly sub-project, making repl-bin unnecessary. >>>> >> > >>>> >> > >>>> >> > On Sun, Dec 8, 2013 at 7:46 PM, Patrick Wendell <pwend...@gmail.com> >>>> >> wrote: >>>> >> > >>>> >> >> Looked into this a bit more - I think removing repl-bin is something >>>> >> >> we should wait until 0.9 to do, because we've published it to maven >>>> in >>>> >> >> 0.8.0 and people might expect it to be there in 0.8.1. >>>> >> >> >>>> >> >> Merging the directly referenced pull request (195) seems like a good >>>> >> >> idea though since it fixes a bug in the script. >>>> >> >> >>>> >> >> Is that what you are suggesting? >>>> >> >> >>>> >> >> - Patrick >>>> >> >> >>>> >> >> On Sun, Dec 8, 2013 at 7:30 PM, Patrick Wendell <pwend...@gmail.com >>>> > >>>> >> >> wrote: >>>> >> >> > Hey Mark - ya this would be good to get in. >>>> >> >> > >>>> >> >> > Does merging that particular PR put this in sufficient shape for >>>> the >>>> >> >> > 0.8.1 release or are there other open patches we need to look at? >>>> >> >> > >>>> >> >> > - Patrick >>>> >> >> > >>>> >> >> > On Sun, Dec 8, 2013 at 6:05 PM, Mark Hamstra < >>>> m...@clearstorydata.com >>>> >> > >>>> >> >> wrote: >>>> >> >> >> SPARK-962 should be resolved before release. See also: >>>> >> >> >> https://github.com/apache/incubator-spark/pull/195 >>>> >> >> >> >>>> >> >> >> With the references to the way I changed Debian packaging for >>>> >> >> ClearStory, >>>> >> >> >> we should be at least 90% of the way toward doing it right for >>>> >> Apache. >>>> >> >> >> >>>> >> >> >> >>>> >> >> >> On Sun, Dec 8, 2013 at 5:29 PM, Patrick Wendell < >>>> pwend...@gmail.com> >>>> >> >> wrote: >>>> >> >> >> >>>> >> >> >>> For my own part I'll give a +1 to this RC. >>>> >> >> >>> >>>> >> >> >>> On Sun, Dec 8, 2013 at 4:30 PM, Taka Shinagawa < >>>> >> taka.epsi...@gmail.com >>>> >> >> > >>>> >> >> >>> wrote: >>>> >> >> >>> > OK. I will post the entire output via separate email. I just >>>> >> upgraded >>>> >> >> >>> > Hadoop to 2.2.0 recently. So there might be something I need >>>> to >>>> >> >> >>> > remove/clean up. >>>> >> >> >>> > >>>> >> >> >>> > >>>> >> >> >>> > On Sun, Dec 8, 2013 at 4:24 PM, Patrick Wendell < >>>> >> pwend...@gmail.com> >>>> >> >> >>> wrote: >>>> >> >> >>> > >>>> >> >> >>> >> Hey Take, >>>> >> >> >>> >> >>>> >> >> >>> >> Could you start a separate thread to debug your build issue? >>>> In >>>> >> that >>>> >> >> >>> >> thread, could you paste the exact build command and entire >>>> >> output? >>>> >> >> The >>>> >> >> >>> log >>>> >> >> >>> >> you posted here suggests the first build detected hadoop >>>> 1.0.4 >>>> >> not >>>> >> >> 2.2.0 >>>> >> >> >>> >> based on the assembly file name it is logging. >>>> >> >> >>> >> >>>> >> >> >>> >> --- >>>> >> >> >>> >> sent from my phone >>>> >> >> >>> >> On Dec 8, 2013 4:13 PM, "Taka Shinagawa" < >>>> taka.epsi...@gmail.com >>>> >> > >>>> >> >> >>> wrote: >>>> >> >> >>> >> >>>> >> >> >>> >> > With Hadoop 2.2.0 (& Java 1.7.0_45) installed, I'm having >>>> >> trouble >>>> >> >> >>> >> > completing the build process (sbt/sbt assembly) on >>>> Macbook. The >>>> >> >> sbt >>>> >> >> >>> >> command >>>> >> >> >>> >> > hangs at the last step. >>>> >> >> >>> >> > >>>> >> >> >>> >> > ... >>>> >> >> >>> >> > ... >>>> >> >> >>> >> > [info] SHA-1: ce8275f5841002164c4305c912a2892ec7c1d395 >>>> >> >> >>> >> > [info] Packaging >>>> >> >> >>> >> > >>>> >> >> >>> >> > >>>> >> >> >>> >> >>>> >> >> >>> >>>> >> >> >>>> >> >>>> /Users/taka/Documents/Spark/Releases/spark-0.8.1-incubating-rc2/tools/target/scala-2.9.3/spark-tools-assembly-0.8.1-incubating.jar >>>> >> >> >>> >> > ... >>>> >> >> >>> >> > [info] SHA-1: 0657a347240266230247693f265a5797d40c326a >>>> >> >> >>> >> > [info] Packaging >>>> >> >> >>> >> > >>>> >> >> >>> >> > >>>> >> >> >>> >> >>>> >> >> >>> >>>> >> >> >>>> >> >>>> /Users/taka/Documents/Spark/Releases/spark-0.8.1-incubating-rc2/assembly/target/scala-2.9.3/spark-assembly-0.8.1-incubating-hadoop1.0.4.jar >>>> >> >> >>> >> > ... >>>> >> >> >>> >> > (hangs here) >>>> >> >> >>> >> > -------------------------- >>>> >> >> >>> >> > >>>> >> >> >>> >> > >>>> >> >> >>> >> > On another Macbook with Hadoop 1.1.1 (& Java 1.7.0_45) >>>> >> installed, >>>> >> >> I >>>> >> >> >>> was >>>> >> >> >>> >> > able to build it successfully. >>>> >> >> >>> >> > .. >>>> >> >> >>> >> > .. >>>> >> >> >>> >> > [info] SHA-1: 77109cd085bd4f0d2b601b3451b35b961d357534 >>>> >> >> >>> >> > [info] Packaging >>>> >> >> >>> >> > >>>> >> >> >>> >> > >>>> >> >> >>> >> >>>> >> >> >>> >>>> >> >> >>>> >> >>>> /Users/tshinagawa/Documents/Spark/RCs/spark-0.8.1-incubating/examples/target/scala-2.9.3/spark-examples-assembly-0.8.1-incubating.jar >>>> >> >> >>> >> > ... >>>> >> >> >>> >> > [info] Done packaging. >>>> >> >> >>> >> > [success] Total time: 266 s, completed Dec 8, 2013 3:03:10 >>>> PM >>>> >> >> >>> >> > -------------------------- >>>> >> >> >>> >> > >>>> >> >> >>> >> > >>>> >> >> >>> >> > >>>> >> >> >>> >> > On Sun, Dec 8, 2013 at 12:41 PM, Patrick Wendell < >>>> >> >> pwend...@gmail.com> >>>> >> >> >>> >> > wrote: >>>> >> >> >>> >> > >>>> >> >> >>> >> > > Please vote on releasing the following candidate as >>>> Apache >>>> >> Spark >>>> >> >> >>> >> > > (incubating) version 0.8.1. >>>> >> >> >>> >> > > >>>> >> >> >>> >> > > The tag to be voted on is v0.8.1-incubating (commit >>>> >> bf23794a): >>>> >> >> >>> >> > > >>>> >> >> >>> >> > > >>>> >> >> >>> >> > >>>> >> >> >>> >> >>>> >> >> >>> >>>> >> >> >>>> >> >>>> https://git-wip-us.apache.org/repos/asf?p=incubator-spark.git;a=tag;h=e6ba91b5a7527316202797fc3dce469ff86cf203 >>>> >> >> >>> >> > > >>>> >> >> >>> >> > > The release files, including signatures, digests, etc >>>> can be >>>> >> >> found >>>> >> >> >>> at: >>>> >> >> >>> >> > > >>>> >> http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc2/ >>>> >> >> >>> >> > > >>>> >> >> >>> >> > > Release artifacts are signed with the following key: >>>> >> >> >>> >> > > https://people.apache.org/keys/committer/pwendell.asc >>>> >> >> >>> >> > > >>>> >> >> >>> >> > > The staging repository for this release can be found at: >>>> >> >> >>> >> > > >>>> >> >> >>> >>>> >> https://repository.apache.org/content/repositories/orgapachespark-024/ >>>> >> >> >>> >> > > >>>> >> >> >>> >> > > The documentation corresponding to this release can be >>>> found >>>> >> at: >>>> >> >> >>> >> > > >>>> >> >> http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc2-docs/ >>>> >> >> >>> >> > > >>>> >> >> >>> >> > > For information about the contents of this release see: >>>> >> >> >>> >> > > <attached> draft of release notes >>>> >> >> >>> >> > > <attached> draft of release credits >>>> >> >> >>> >> > > >>>> >> >> >>> >>>> >> https://github.com/apache/incubator-spark/blob/branch-0.8/CHANGES.txt >>>> >> >> >>> >> > > >>>> >> >> >>> >> > > Please vote on releasing this package as Apache Spark >>>> >> >> >>> 0.8.1-incubating! >>>> >> >> >>> >> > > >>>> >> >> >>> >> > > The vote is open until Wednesday, December 11th at 21:00 >>>> UTC >>>> >> and >>>> >> >> >>> >> > > passes if a majority of at least 3 +1 PPMC votes are >>>> cast. >>>> >> >> >>> >> > > >>>> >> >> >>> >> > > [ ] +1 Release this package as Apache Spark >>>> 0.8.1-incubating >>>> >> >> >>> >> > > [ ] -1 Do not release this package because ... >>>> >> >> >>> >> > > >>>> >> >> >>> >> > > To learn more about Apache Spark, please see >>>> >> >> >>> >> > > http://spark.incubator.apache.org/ >>>> >> >> >>> >> > > >>>> >> >> >>> >> > >>>> >> >> >>> >> >>>> >> >> >>> >>>> >> >> >>>> >> >>>> >>> >>>
