Cool. It sounds like focusing on sbt-pom-reader would be a good thing for you guys then.
There's a few... fun... issues around maven parent projects that are still running around with sbt-pom-reader that appear to be fundamental ivy-maven hate-based issues. IN any case, while I'm generally swamped with things, let me know anything I can do to improve the situation. The shading thing frustrates me slightly, as I thinka lot of projects that shade have been abandoned (that I looked into). Which plugin are you using? On Fri, Mar 14, 2014 at 10:21 PM, Matei Zaharia <matei.zaha...@gmail.com>wrote: > I like the pom-reader approach as well -- in particular, that it lets you > add extra stuff in your SBT build after loading the dependencies from the > POM. Profiles would be the one thing missing to be able to pass options > through. > > Matei > > On Mar 14, 2014, at 10:03 AM, Patrick Wendell <pwend...@gmail.com> wrote: > > > Hey Josh, > > > > I spent some time playing with the sbt-pom-reader and actually I think > > it might suite our needs pretty well. Basically our situation is that > > Maven has more mature support for packaging (shading, official > > assembly plug-in) and is used by pretty much every downstream packager > > of Spark. SBT is used by the developers heavily for incremental > > compilation, the console, etc. So far we've been maintaining entirely > > isolated builds in sbt and maven which is a huge development cost and > > something that's lead to divergence over time. > > > > The sbt-pom-reader I think could give us the best of both worlds. But > > it doesn't support profiles in Maven which we use pretty heavily. I > > played a bit this week and tried to get it to work without much luck: > > > https://github.com/pwendell/sbt-pom-reader/commit/ca1f1f2d6bf8891acb7212facf4807baaca8974d > > > > Any pointers or interest in helping us with that feature? I think with > > that in place we might have a good solution where the dependencies are > > declared in one place (poms) and we use Maven for packaging but we can > > use sbt for day to day development. > > > > > > > > On Fri, Mar 14, 2014 at 9:41 AM, Evan Chan <e...@ooyala.com> wrote: > >> Jsuereth - > >> > >> Thanks for jumping on this thread. There are a couple things which > >> would help SBT IMHO (w.r.t. issues mentioned here): > >> > >> - A way of generating a project-wide pom.xml (I believe make-pom only > >> generates it for each sub-project) > >> - and probably better or more feature-complete POMs, but the Maven > >> folks here can speak to that > >> - Make the sbt-pom-reader plugin officially part of SBT, and make it > >> work well (again the Maven folks need to jump in here, though plugins > >> can't be directly translated) > >> - Have the sbt-assembly plugin officially maintained by Typesafe, or > >> part of SBT.... most Maven folks expect not to have to include a > >> plugin to generate a fat jar, and it's a pretty essential plugin for > >> just about every SBT project > >> - Also there is no equivalent (AFAIK) to the maven shader plugin. > >> > >> I also wish that the dependency-graph plugin was included by default, > >> but that's just me :) > >> > >> -Evan > >> > >> > >> On Fri, Mar 14, 2014 at 6:47 AM, jsuereth <joshua.suer...@gmail.com> > wrote: > >>> Hey guys - > >>> > >>> If there's anything we can do to improve the sbt experience, let me > know. > >>> I'd be extremely interested to see how/where there are issues > integrating > >>> sbt with the existing Hadoop ecosystem. > >>> > >>> Particularly the difficulties in using Sbt + Maven together (something > which > >>> tends block more than just spark from adopting sbt). > >>> > >>> I'm more than happy to listen and see what we can do on the sbt side > to make > >>> this as seamless as possible for all parties. > >>> > >>> Thanks! > >>> > >>> > >>> > >>> -- > >>> View this message in context: > http://apache-spark-developers-list.1001551.n3.nabble.com/DISCUSS-Necessity-of-Maven-and-SBT-Build-in-Spark-tp2315p5682.html > >>> Sent from the Apache Spark Developers List mailing list archive at > Nabble.com. > >> > >> > >> > >> -- > >> -- > >> Evan Chan > >> Staff Engineer > >> e...@ooyala.com | > >
