Thanks for the feedback. It looks like there are more advantages to Maven than I was originally thinking of -- specifically, the better dependency resolution and assembly construction. (SBT assembly just takes all the JARs in lib_managed and packages them together unfortunately, which means you sometimes get multiple versions of the same artifact if you aren't very careful with exclusion rules). I think what we'll do is to wait until we see whether we can have a single Spark artifact that works with any Hadoop version, and go back to the build system issue then.
Matei On Jul 16, 2013, at 4:37 PM, Ryan LeCompte <[email protected]> wrote: > +1 for SBT. It's pretty much the de facto standard for mainstream Scala > projects. > > > On Tue, Jul 16, 2013 at 4:19 PM, Evan Chan <[email protected]> wrote: > If the APIs for those libraries such as Akka stay the same, you don't need > different branches. In SBT you can easily support two different sets of > deps depending on which Scala version you are building.... not sure if you > could do that with Maven. > > Here are the main differences that would require a different branch: > - if you are taking advantage of Akka Cluster, or features exclusive to newer > releases > - The Akka futures API changed packages between Scala 2.9.1/2 and Scala > 2.9.3/2.10. However, since the project upgraded to Scala 2.9.3, it should > migrate all use of futures to the scala.concurrent.* namespace to avoid more > code changes down the line. > > As far as SBT vs Maven vs Gradle etc..... > - I personally think SBT's "console" REPL with all dependencies in the > classpath is super valuable for development, ad hoc testing, trying new ideas > out. Not sure if Maven has that. > - SBT also has triggered recompilation/testing, which is again very valuable > for some of us > - Gradle has a easier to understand DSL, but it seems once you build a > complex enough system, it is no easier to maintain than SBT > - Maven is much better integrated into most IDEs, but SBT is far superior for > dev workflow if you don't use one of the major IDEs or are a command line > person. > > -Evan > > > On Tuesday, July 16, 2013 1:37:41 PM UTC-7, Matei Zaharia wrote: > Unfortunately, we'll probably have to have different branches of Spark for > different Scala versions, because there are also other libraries we depend on > (e.g. Akka) that have separate versions for Scala 2.10. You can actually find > a Scala 2.10 port of Spark in the scala-2.10 branch on GitHub. > > Matei > > > On Jul 16, 2013, at 1:33 PM, Cody Koeninger <[email protected]> wrote: > > > If you're looking at consolidating build systems, I'd ask to consider ease > > of cross-publishing for different Scala versions. My instinct is that sbt > > will be less troublesome in that regard (although as I understand it, the > > changes to the repl may present a problem). > > > > We're needing to use 2.10 for a project, so I'd be happy to put in some > > work on the issue. > > > -- > You received this message because you are subscribed to the Google Groups > "Spark Developers" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/groups/opt_out. > > > > > -- > You received this message because you are subscribed to the Google Groups > "Spark Developers" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/groups/opt_out. > >
