Sorry all, I sent the wrong pull request to refer to Prashant's work: https://github.com/apache/spark/pull/772
On Thu, Jul 3, 2014 at 1:37 PM, Patrick Wendell <pwend...@gmail.com> wrote: > Just a reminder here - we'll soon be merging a patch that changes the > SBT build internals significantly. We've tried to make this fully > backwards compatible, but there may be issues (which we'll resolve as > they arrive). > > https://github.com/apache/spark/pull/77 > > - Patrick > > On Sun, Jun 22, 2014 at 10:27 AM, Mark Hamstra <m...@clearstorydata.com> > wrote: >> Just a couple of FYI notes: With Zinc and the scala-maven-plugin, repl and >> incremental builds are also available to those doing day-to-day development >> using Maven. As long as you don't have to delve into the extra boilerplate >> and verbosity of Maven's POMs relative to an SBT build file, there is >> little day-to-day functional difference between the two -- if anything, I >> find that Maven supports faster development cycles. >> >> >> On Sun, Jun 22, 2014 at 12:24 AM, Patrick Wendell <pwend...@gmail.com> >> wrote: >> >>> Hey All, >>> >>> 1. The original test infrastructure hosted by the AMPLab has been >>> fully restored and also expanded with many more executor slots for >>> tests. Thanks to Matt Massie at the Amplab for helping with this. >>> >>> 2. We now have a nightly build matrix across different Hadoop >>> versions. It appears that the Maven build is failing tests with some >>> of the newer Hadoop versions. If people from the community are >>> interested, diagnosing and fixing test issues would be welcome patches >>> (they are all dependency related). >>> >>> https://issues.apache.org/jira/browse/SPARK-2232 >>> >>> 3. Prashant Sharma has spent a lot of time to make it possible for our >>> sbt build to read dependencies from Maven. This will save us a huge >>> amount of headache keeping the builds consistent. I just wanted to >>> give a heads up to users about this - we should retain compatibility >>> with features of the sbt build, but if you are e.g. hooking into deep >>> internals of our build it may affect you. I'm hoping this can be >>> updated and merged in the next week: >>> >>> https://github.com/apache/spark/pull/77 >>> >>> 4. We've moved most of the documentation over to recommending users >>> build with Maven when creating official packages. This is just to >>> provide a single "reference build" of Spark since it's the one we test >>> and package for releases, we make sure all recursive dependencies are >>> correct, etc. I'd recommend that all downstream packagers use this >>> build. >>> >>> For day-to-day development I imagine sbt will remain more popular >>> (repl, incremental builds, etc). Prashant's work allows us to get the >>> "best of both worlds" which is great. >>> >>> - Patrick >>>
