Hi RJ, I think I remember noticing in the past that some Guava metadata ends up overwriting maven-generated metadata in the assembly's manifest. That's probably something we should fix if that still affects the build.
That being said, this is probably happening because you're using "install-file" instead of "install". If you want a workaround that doesn't require unshading things, you can change assembly.pom to (i) not skip the install plugin and (ii) have "jar" as the packaging, instead of pom. On Wed, Jan 14, 2015 at 1:08 PM, RJ Nowling <[email protected]> wrote: > Hi Sean, > > I confirmed that if I take the Spark 1.2.0 release (a428c446), undo the > guava PR [1], and use -Dmaven.install.skip=false with the workflow above, > the problem is fixed. > > RJ > > > [1] > https://github.com/apache/spark/commit/c9f743957fa963bc1dbed7a44a346ffce1a45cf2#diff-6382f8428b13fa6082fa688178f3dbcc > > On Wed, Jan 14, 2015 at 2:59 PM, RJ Nowling <[email protected]> wrote: > >> Thanks, Sean. >> >> Yes, Spark is incorrectly copying the spark assembly jar to >> com/google/guava in the maven repository. This is for the 1.2.0 release, >> just to clarify. >> >> I reverted the patches that shade Guava and removed the parts disabling >> the install plugin and it seemed to fix the issue. >> >> It seems that Spark poms are inheriting something from Guava. >> >> RJ >> >> On Wed, Jan 14, 2015 at 2:33 PM, Sean Owen <[email protected]> wrote: >> >>> Guava is shaded, although one class is left in its original package. >>> This shouldn't have anything to do with Spark's package or namespace >>> though. What are you saying is in com/google/guava? >>> >>> You can un-skip the install plugin with -Dmaven.install.skip=false >>> >>> On Wed, Jan 14, 2015 at 7:26 PM, RJ Nowling <[email protected]> wrote: >>> > Hi all, >>> > >>> > I'm trying to upgrade some Spark RPMs from 1.1.0 to 1.2.0. As part of >>> the >>> > RPM process, we build Spark with Maven. With Spark 1.2.0, though, the >>> > artifacts are placed in com/google/guava and there is no >>> org/apache/spark. >>> > >>> > I saw that the pom.xml files had been modified to prevent the install >>> > command and that the guava dependency was modified. Could someone who >>> is >>> > more familiar with the Spark maven files comment on what might be >>> causing >>> > this oddity? >>> > >>> > Thanks, >>> > RJ >>> > >>> > We build Spark like so: >>> > $ mvn -Phadoop-2.4 -Dmesos.version=0.20.0 -DskipTests clean package >>> > >>> > Then build a local Maven repo: >>> > >>> > mvn -Phadoop-2.4 \ >>> > -Dmesos.version=0.20.0 \ >>> > -DskipTests install:install-file \ >>> > >>> -Dfile=assembly/target/scala-2.10/spark-assembly-1.2.0-hadoop2.4.0.jar \ >>> > -DcreateChecksum=true \ >>> > -DgeneratePom=true \ >>> > -DartifactId=spark-assembly_2.1.0 \ >>> > -DlocalRepositoryPath=../maven2 >>> >> >> -- Marcelo --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
