I'm in the process of trying to build Apache-fied Spark into our stack at
ClearStory. We've been using Maven to do Debian packaging as found in the
repl-bin module for quite a while, but that doesn't work now. To see the
failure, you need to build repl-bin with the `deb` profile -- e.g. "mvn
-Prepl-bin,deb install" from the project directory. The problem is with
the buildnumber-maven-plugin, which is used to attach the 8-digit short SHA
of the current git commit to the Debian package -- so, e.g.,
spark_0.8.0-SNAPSHOT-b993b2a3_all.deb. The buildnumber plugin relies upon
the <scm> tag to determine which source code management system should be
used to lookup the version number.
That all worked great until the following was added to spark/pom.xml:
<parent>
<groupId>org.apache</groupId>
<artifactId>apache</artifactId>
<version>11</version>
</parent>
The problem is that the parent pom that the above includes has this within
it:
<scm>
<connection>scm:svn:
http://svn.apache.org/repos/asf/maven/pom/tags/apache-11</connection
<developerConnection>scm:svn:
https://svn.apache.org/repos/asf/maven/pom/tags/apache-11
</developerConnection>
<url>http://svn.apache.org/viewvc/maven/pom/tags/apache-11</url>
</scm>
And that essentially overrides the <scm> section of spark/pom.xml, which
means that the buildnumber plugin thinks that we are using SVN instead of
git, and the SHA lookup fails.
So, that is one known problem in the Maven build. I'm not at all certain
that there aren't others or lurking problems because of different versions
of dependencies, plugins, etc. being included by that org.apache:apache
artifact and also specified within the Spark pom files.
I'm pretty sure that we can't not include the equivalent of
org.apache:apache:11, since there is a lot of important stuff in there to
do Apache release management, etc. So, is there another, equivalent
artifact that we can use that doesn't confuse the <scm> issue, or some
other way around this problem? It looks like there are some other
git-specific plugins out there that we may be able to use instead of
buildnumber-maven-plugin, so that may be a valid and necessary solution to
the Debian packaging problem.
Can someone who knows Maven better than I take a look at the overlap
between org.apache:apache and the Spark pom files to see if there are
problems lurking there? Even if there aren't problems other than with the
buildnumber plugin, we may be able to trim a lot out of the Spark poms that
is already present in the apache parent pom.
