Hey Everyone,
We are going to publish artifacts to maven central in the exact same
format no matter which build system we use.
For normal consumers of Spark {maven vs sbt} won't make a difference.
It will make a difference for people who are extended the Spark build
to do their own packaging. This is what I'm trying to gauge - does
anyone do this in a way where they feel only maven or only sbt
supports their particular issue.
- Patrick
On Fri, Feb 21, 2014 at 12:40 AM, Pascal Voitot Dev
<pascal.voitot....@gmail.com> wrote:
> Hi,
>
> My small contrib to the discussion.
> SBT is able to publish Maven artifacts generating the POM and all JAR &
> signed files.
> So even if not in the project, a Pom can be found somewhere.
>
> Pascal
>
>
>
> On Fri, Feb 21, 2014 at 9:28 AM, Paul Brown <p...@mult.ifario.us> wrote:
>
>> As a customer of the code, I don't care *how* the code gets built, but it
>> is important to me that the Maven artifacts (POM files, binaries, sources,
>> javadocs) are clean, accurate, up to date, and published on Maven Central.
>>
>> Some examples where structure/publishing failures have been bad for users:
>>
>> - For a long time (and perhaps still), Solr and Lucene were built by an Ant
>> build that produced incorrect POMs and required potential developers to
>> manually configure their IDEs.
>>
>> - For a long time (and perhaps still), Pig was built by Ant, published
>> incorrect POMs, and failed to publish useful auxiliary artifacts like
>> PigUnit and the PiggyBank as Maven-addressable artifacts. (That said,
>> thanks to Spark, we no longer use Pig...)
>>
>> - For a long time (and perhaps still), Cassandra depended on
>> non-generally-available libraries (high-scale, etc.) that made it
>> inconvenient to embed Cassandra in a larger system. Cassandra gets a
>> little slack because the build/structure was almost too terrible to look at
>> prior to incubation and it's gotten better...
>>
>> And those are just a few projects at Apache that come to mind; I could make
>> a longish list of offenders.
>>
>> btw, among other things that the Spark project probably *should* do would
>> be to publish artifacts with a classifier to distinguish the Hadoop version
>> linked against.
>>
>> I'll be a happy user of sbt-built artifacts, or if the project goes/sticks
>> with Maven I'm more than willing to help answer questions or provide PRs
>> for stickier items around assemblies, multiple artifacts, etc.
>>
>>
>> --
>> p...@mult.ifario.us | Multifarious, Inc. | http://mult.ifario.us/
>>
>>
>> On Thu, Feb 20, 2014 at 11:56 PM, Sean Owen <so...@cloudera.com> wrote:
>>
>> > Two builds is indeed a pain, since it's an ongoing chore to keep them
>> > in sync. For example, I am already seeing that the two do not quite
>> > declare the same dependencies (see recent patch).
>> >
>> > I think publishing artifacts to Maven central should be considered a
>> > hard requirement if it isn't already one from the ASF, and it may be?
>> > Certainly most people out there would be shocked if you told them
>> > Spark is not in the repo at all. And that requires at least
>> > maintaining a pom that declares the structure of the project.
>> >
>> > This does not necessarily mean using Maven to build, but is a reason
>> > that removing the pom is going to make this a lot harder for people to
>> > consume as a project.
>> >
>> > Maven has its pros and cons but there are plenty of people lurking
>> > around who know it quite well. Certainly it's easier for the Hadoop
>> > people to understand and work with. On the other hand, it supports
>> > Scala although only via a plugin, which is weaker support. sbt seems
>> > like a fairly new, basic, ad-hoc tool. Is there an advantage to it,
>> > other than being Scala (which is an advantage)?
>> >
>> > --
>> > Sean Owen | Director, Data Science | London
>> >
>> >
>> > On Fri, Feb 21, 2014 at 4:03 AM, Patrick Wendell <pwend...@gmail.com>
>> > wrote:
>> > > Hey All,
>> > >
>> > > It's very high overhead having two build systems in Spark. Before
>> > > getting into a long discussion about the merits of sbt vs maven, I
>> > > wanted to pose a simple question to the dev list:
>> > >
>> > > Is there anyone who feels that dropping either sbt or maven would have
>> > > a major consequence for them?
>> > >
>> > > And I say "major consequence" meaning something becomes completely
>> > > impossible now and can't be worked around. This is different from an
>> > > "inconvenience", i.e., something which can be worked around but will
>> > > require some investment.
>> > >
>> > > I'm posing the question in this way because, if there are features in
>> > > either build system that are absolutely-un-available in the other,
>> > > then we'll have to maintain both for the time being. I'm merely trying
>> > > to see whether this is the case...
>> > >
>> > > - Patrick
>> >
>>
