Assembly jar file name does not match profile selection

2014-12-26 Thread Alessandro Baretta
I am building spark with sbt off of branch 1.2. I'm using the following command: sbt/sbt -Pyarn -Phadoop-2.3 assembly (http://spark.apache.org/docs/latest/building-spark.html#building-with-sbt) Although the jar file I obtain does contain the proper version of the hadoop libraries (v. 2.4), the

Re: Assembly jar file name does not match profile selection

2014-12-26 Thread Ted Yu
Can you try this command ? sbt/sbt -Pyarn -Phadoop-2.4 -Dhadoop.version=2.6.0 -Phive assembly On Fri, Dec 26, 2014 at 6:15 PM, Alessandro Baretta alexbare...@gmail.com wrote: I am building spark with sbt off of branch 1.2. I'm using the following command: sbt/sbt -Pyarn -Phadoop-2.3

Re: Assembly jar file name does not match profile selection

2014-12-26 Thread Alessandro Baretta
Here's what I get: ./assembly/target/scala-2.10/spark-assembly-1.3.0-SNAPSHOT-hadoop2.6.0.jar Alex On Fri, Dec 26, 2014 at 8:41 PM, Ted Yu yuzhih...@gmail.com wrote: Can you try this command ? sbt/sbt -Pyarn -Phadoop-2.4 -Dhadoop.version=2.6.0 -Phive assembly On Fri, Dec 26, 2014 at 6:15

Re: Assembly jar file name does not match profile selection

2014-12-26 Thread Kousuke Saruta
Hi Alessandro, It's fixed by SPARK-3787 and will be applied to 1.2.1 and 1.3.0. https://issues.apache.org/jira/browse/SPARK-3787 - Kousuke (2014/12/27 11:15), Alessandro Baretta wrote: I am building spark with sbt off of branch 1.2. I'm using the following command: sbt/sbt -Pyarn