Well, 195 is sufficient to give you something that runs, but it doesn't run the same way as Spark built/distributed by other means -- e.g., after 195 the package still uses something equivalent to the old `run` script instead of the current `spark-class` way.
On Sun, Dec 8, 2013 at 8:02 PM, Patrick Wendell <[email protected]> wrote: > Hey Mark, > > What I'm asking is whether this patch is sufficient to have a working > debian build in 0.8.1, or are there other outstanding issues to make > it work? By working I mean, within the initial design that was > contributed (with repl-bin) it works according to that approach. > > We can redesign this packaging in 0.9. That will require having a PR > against Apache Spark, discussing, etc. But it doesn't need to be on > the critical path for this release. > > - Patrick > > On Sun, Dec 8, 2013 at 7:54 PM, Mark Hamstra <[email protected]> > wrote: > > Whatever Debian package gets built has to work, so that's the first > > requirement. I don't know how to decide whether a change is acceptable > in > > 0.8 or has to wait until 0.9, but the 0.9 packaging should definitely > > leverage the assembly sub-project, making repl-bin unnecessary. > > > > > > On Sun, Dec 8, 2013 at 7:46 PM, Patrick Wendell <[email protected]> > wrote: > > > >> Looked into this a bit more - I think removing repl-bin is something > >> we should wait until 0.9 to do, because we've published it to maven in > >> 0.8.0 and people might expect it to be there in 0.8.1. > >> > >> Merging the directly referenced pull request (195) seems like a good > >> idea though since it fixes a bug in the script. > >> > >> Is that what you are suggesting? > >> > >> - Patrick > >> > >> On Sun, Dec 8, 2013 at 7:30 PM, Patrick Wendell <[email protected]> > >> wrote: > >> > Hey Mark - ya this would be good to get in. > >> > > >> > Does merging that particular PR put this in sufficient shape for the > >> > 0.8.1 release or are there other open patches we need to look at? > >> > > >> > - Patrick > >> > > >> > On Sun, Dec 8, 2013 at 6:05 PM, Mark Hamstra <[email protected] > > > >> wrote: > >> >> SPARK-962 should be resolved before release. See also: > >> >> https://github.com/apache/incubator-spark/pull/195 > >> >> > >> >> With the references to the way I changed Debian packaging for > >> ClearStory, > >> >> we should be at least 90% of the way toward doing it right for > Apache. > >> >> > >> >> > >> >> On Sun, Dec 8, 2013 at 5:29 PM, Patrick Wendell <[email protected]> > >> wrote: > >> >> > >> >>> For my own part I'll give a +1 to this RC. > >> >>> > >> >>> On Sun, Dec 8, 2013 at 4:30 PM, Taka Shinagawa < > [email protected] > >> > > >> >>> wrote: > >> >>> > OK. I will post the entire output via separate email. I just > upgraded > >> >>> > Hadoop to 2.2.0 recently. So there might be something I need to > >> >>> > remove/clean up. > >> >>> > > >> >>> > > >> >>> > On Sun, Dec 8, 2013 at 4:24 PM, Patrick Wendell < > [email protected]> > >> >>> wrote: > >> >>> > > >> >>> >> Hey Take, > >> >>> >> > >> >>> >> Could you start a separate thread to debug your build issue? In > that > >> >>> >> thread, could you paste the exact build command and entire > output? > >> The > >> >>> log > >> >>> >> you posted here suggests the first build detected hadoop 1.0.4 > not > >> 2.2.0 > >> >>> >> based on the assembly file name it is logging. > >> >>> >> > >> >>> >> --- > >> >>> >> sent from my phone > >> >>> >> On Dec 8, 2013 4:13 PM, "Taka Shinagawa" <[email protected] > > > >> >>> wrote: > >> >>> >> > >> >>> >> > With Hadoop 2.2.0 (& Java 1.7.0_45) installed, I'm having > trouble > >> >>> >> > completing the build process (sbt/sbt assembly) on Macbook. The > >> sbt > >> >>> >> command > >> >>> >> > hangs at the last step. > >> >>> >> > > >> >>> >> > ... > >> >>> >> > ... > >> >>> >> > [info] SHA-1: ce8275f5841002164c4305c912a2892ec7c1d395 > >> >>> >> > [info] Packaging > >> >>> >> > > >> >>> >> > > >> >>> >> > >> >>> > >> > /Users/taka/Documents/Spark/Releases/spark-0.8.1-incubating-rc2/tools/target/scala-2.9.3/spark-tools-assembly-0.8.1-incubating.jar > >> >>> >> > ... > >> >>> >> > [info] SHA-1: 0657a347240266230247693f265a5797d40c326a > >> >>> >> > [info] Packaging > >> >>> >> > > >> >>> >> > > >> >>> >> > >> >>> > >> > /Users/taka/Documents/Spark/Releases/spark-0.8.1-incubating-rc2/assembly/target/scala-2.9.3/spark-assembly-0.8.1-incubating-hadoop1.0.4.jar > >> >>> >> > ... > >> >>> >> > (hangs here) > >> >>> >> > -------------------------- > >> >>> >> > > >> >>> >> > > >> >>> >> > On another Macbook with Hadoop 1.1.1 (& Java 1.7.0_45) > installed, > >> I > >> >>> was > >> >>> >> > able to build it successfully. > >> >>> >> > .. > >> >>> >> > .. > >> >>> >> > [info] SHA-1: 77109cd085bd4f0d2b601b3451b35b961d357534 > >> >>> >> > [info] Packaging > >> >>> >> > > >> >>> >> > > >> >>> >> > >> >>> > >> > /Users/tshinagawa/Documents/Spark/RCs/spark-0.8.1-incubating/examples/target/scala-2.9.3/spark-examples-assembly-0.8.1-incubating.jar > >> >>> >> > ... > >> >>> >> > [info] Done packaging. > >> >>> >> > [success] Total time: 266 s, completed Dec 8, 2013 3:03:10 PM > >> >>> >> > -------------------------- > >> >>> >> > > >> >>> >> > > >> >>> >> > > >> >>> >> > On Sun, Dec 8, 2013 at 12:41 PM, Patrick Wendell < > >> [email protected]> > >> >>> >> > wrote: > >> >>> >> > > >> >>> >> > > Please vote on releasing the following candidate as Apache > Spark > >> >>> >> > > (incubating) version 0.8.1. > >> >>> >> > > > >> >>> >> > > The tag to be voted on is v0.8.1-incubating (commit > bf23794a): > >> >>> >> > > > >> >>> >> > > > >> >>> >> > > >> >>> >> > >> >>> > >> > https://git-wip-us.apache.org/repos/asf?p=incubator-spark.git;a=tag;h=e6ba91b5a7527316202797fc3dce469ff86cf203 > >> >>> >> > > > >> >>> >> > > The release files, including signatures, digests, etc can be > >> found > >> >>> at: > >> >>> >> > > > http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc2/ > >> >>> >> > > > >> >>> >> > > Release artifacts are signed with the following key: > >> >>> >> > > https://people.apache.org/keys/committer/pwendell.asc > >> >>> >> > > > >> >>> >> > > The staging repository for this release can be found at: > >> >>> >> > > > >> >>> > https://repository.apache.org/content/repositories/orgapachespark-024/ > >> >>> >> > > > >> >>> >> > > The documentation corresponding to this release can be found > at: > >> >>> >> > > > >> http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc2-docs/ > >> >>> >> > > > >> >>> >> > > For information about the contents of this release see: > >> >>> >> > > <attached> draft of release notes > >> >>> >> > > <attached> draft of release credits > >> >>> >> > > > >> >>> > https://github.com/apache/incubator-spark/blob/branch-0.8/CHANGES.txt > >> >>> >> > > > >> >>> >> > > Please vote on releasing this package as Apache Spark > >> >>> 0.8.1-incubating! > >> >>> >> > > > >> >>> >> > > The vote is open until Wednesday, December 11th at 21:00 UTC > and > >> >>> >> > > passes if a majority of at least 3 +1 PPMC votes are cast. > >> >>> >> > > > >> >>> >> > > [ ] +1 Release this package as Apache Spark 0.8.1-incubating > >> >>> >> > > [ ] -1 Do not release this package because ... > >> >>> >> > > > >> >>> >> > > To learn more about Apache Spark, please see > >> >>> >> > > http://spark.incubator.apache.org/ > >> >>> >> > > > >> >>> >> > > >> >>> >> > >> >>> > >> >
