Hey Mark, What I'm asking is whether this patch is sufficient to have a working debian build in 0.8.1, or are there other outstanding issues to make it work? By working I mean, within the initial design that was contributed (with repl-bin) it works according to that approach.
We can redesign this packaging in 0.9. That will require having a PR against Apache Spark, discussing, etc. But it doesn't need to be on the critical path for this release. - Patrick On Sun, Dec 8, 2013 at 7:54 PM, Mark Hamstra <m...@clearstorydata.com> wrote: > Whatever Debian package gets built has to work, so that's the first > requirement. I don't know how to decide whether a change is acceptable in > 0.8 or has to wait until 0.9, but the 0.9 packaging should definitely > leverage the assembly sub-project, making repl-bin unnecessary. > > > On Sun, Dec 8, 2013 at 7:46 PM, Patrick Wendell <pwend...@gmail.com> wrote: > >> Looked into this a bit more - I think removing repl-bin is something >> we should wait until 0.9 to do, because we've published it to maven in >> 0.8.0 and people might expect it to be there in 0.8.1. >> >> Merging the directly referenced pull request (195) seems like a good >> idea though since it fixes a bug in the script. >> >> Is that what you are suggesting? >> >> - Patrick >> >> On Sun, Dec 8, 2013 at 7:30 PM, Patrick Wendell <pwend...@gmail.com> >> wrote: >> > Hey Mark - ya this would be good to get in. >> > >> > Does merging that particular PR put this in sufficient shape for the >> > 0.8.1 release or are there other open patches we need to look at? >> > >> > - Patrick >> > >> > On Sun, Dec 8, 2013 at 6:05 PM, Mark Hamstra <m...@clearstorydata.com> >> wrote: >> >> SPARK-962 should be resolved before release. See also: >> >> https://github.com/apache/incubator-spark/pull/195 >> >> >> >> With the references to the way I changed Debian packaging for >> ClearStory, >> >> we should be at least 90% of the way toward doing it right for Apache. >> >> >> >> >> >> On Sun, Dec 8, 2013 at 5:29 PM, Patrick Wendell <pwend...@gmail.com> >> wrote: >> >> >> >>> For my own part I'll give a +1 to this RC. >> >>> >> >>> On Sun, Dec 8, 2013 at 4:30 PM, Taka Shinagawa <taka.epsi...@gmail.com >> > >> >>> wrote: >> >>> > OK. I will post the entire output via separate email. I just upgraded >> >>> > Hadoop to 2.2.0 recently. So there might be something I need to >> >>> > remove/clean up. >> >>> > >> >>> > >> >>> > On Sun, Dec 8, 2013 at 4:24 PM, Patrick Wendell <pwend...@gmail.com> >> >>> wrote: >> >>> > >> >>> >> Hey Take, >> >>> >> >> >>> >> Could you start a separate thread to debug your build issue? In that >> >>> >> thread, could you paste the exact build command and entire output? >> The >> >>> log >> >>> >> you posted here suggests the first build detected hadoop 1.0.4 not >> 2.2.0 >> >>> >> based on the assembly file name it is logging. >> >>> >> >> >>> >> --- >> >>> >> sent from my phone >> >>> >> On Dec 8, 2013 4:13 PM, "Taka Shinagawa" <taka.epsi...@gmail.com> >> >>> wrote: >> >>> >> >> >>> >> > With Hadoop 2.2.0 (& Java 1.7.0_45) installed, I'm having trouble >> >>> >> > completing the build process (sbt/sbt assembly) on Macbook. The >> sbt >> >>> >> command >> >>> >> > hangs at the last step. >> >>> >> > >> >>> >> > ... >> >>> >> > ... >> >>> >> > [info] SHA-1: ce8275f5841002164c4305c912a2892ec7c1d395 >> >>> >> > [info] Packaging >> >>> >> > >> >>> >> > >> >>> >> >> >>> >> /Users/taka/Documents/Spark/Releases/spark-0.8.1-incubating-rc2/tools/target/scala-2.9.3/spark-tools-assembly-0.8.1-incubating.jar >> >>> >> > ... >> >>> >> > [info] SHA-1: 0657a347240266230247693f265a5797d40c326a >> >>> >> > [info] Packaging >> >>> >> > >> >>> >> > >> >>> >> >> >>> >> /Users/taka/Documents/Spark/Releases/spark-0.8.1-incubating-rc2/assembly/target/scala-2.9.3/spark-assembly-0.8.1-incubating-hadoop1.0.4.jar >> >>> >> > ... >> >>> >> > (hangs here) >> >>> >> > -------------------------- >> >>> >> > >> >>> >> > >> >>> >> > On another Macbook with Hadoop 1.1.1 (& Java 1.7.0_45) installed, >> I >> >>> was >> >>> >> > able to build it successfully. >> >>> >> > .. >> >>> >> > .. >> >>> >> > [info] SHA-1: 77109cd085bd4f0d2b601b3451b35b961d357534 >> >>> >> > [info] Packaging >> >>> >> > >> >>> >> > >> >>> >> >> >>> >> /Users/tshinagawa/Documents/Spark/RCs/spark-0.8.1-incubating/examples/target/scala-2.9.3/spark-examples-assembly-0.8.1-incubating.jar >> >>> >> > ... >> >>> >> > [info] Done packaging. >> >>> >> > [success] Total time: 266 s, completed Dec 8, 2013 3:03:10 PM >> >>> >> > -------------------------- >> >>> >> > >> >>> >> > >> >>> >> > >> >>> >> > On Sun, Dec 8, 2013 at 12:41 PM, Patrick Wendell < >> pwend...@gmail.com> >> >>> >> > wrote: >> >>> >> > >> >>> >> > > Please vote on releasing the following candidate as Apache Spark >> >>> >> > > (incubating) version 0.8.1. >> >>> >> > > >> >>> >> > > The tag to be voted on is v0.8.1-incubating (commit bf23794a): >> >>> >> > > >> >>> >> > > >> >>> >> > >> >>> >> >> >>> >> https://git-wip-us.apache.org/repos/asf?p=incubator-spark.git;a=tag;h=e6ba91b5a7527316202797fc3dce469ff86cf203 >> >>> >> > > >> >>> >> > > The release files, including signatures, digests, etc can be >> found >> >>> at: >> >>> >> > > http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc2/ >> >>> >> > > >> >>> >> > > Release artifacts are signed with the following key: >> >>> >> > > https://people.apache.org/keys/committer/pwendell.asc >> >>> >> > > >> >>> >> > > The staging repository for this release can be found at: >> >>> >> > > >> >>> https://repository.apache.org/content/repositories/orgapachespark-024/ >> >>> >> > > >> >>> >> > > The documentation corresponding to this release can be found at: >> >>> >> > > >> http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc2-docs/ >> >>> >> > > >> >>> >> > > For information about the contents of this release see: >> >>> >> > > <attached> draft of release notes >> >>> >> > > <attached> draft of release credits >> >>> >> > > >> >>> https://github.com/apache/incubator-spark/blob/branch-0.8/CHANGES.txt >> >>> >> > > >> >>> >> > > Please vote on releasing this package as Apache Spark >> >>> 0.8.1-incubating! >> >>> >> > > >> >>> >> > > The vote is open until Wednesday, December 11th at 21:00 UTC and >> >>> >> > > passes if a majority of at least 3 +1 PPMC votes are cast. >> >>> >> > > >> >>> >> > > [ ] +1 Release this package as Apache Spark 0.8.1-incubating >> >>> >> > > [ ] -1 Do not release this package because ... >> >>> >> > > >> >>> >> > > To learn more about Apache Spark, please see >> >>> >> > > http://spark.incubator.apache.org/ >> >>> >> > > >> >>> >> > >> >>> >> >> >>> >>
