RE: compiling against hadoop 2.2

2014-01-02 Thread Liu, Raymond
I think you also need to set yarn.version

Say something like 

mvn -Pyarn -Dhadoop.version=2.2.0 -Dyarn.version=2.2.0 -DskipTests clean package


hadoop.version is default to 2.2.0 while yarn.version not when you chose the 
new-yarn profile. We probably need to fix it later for easy usage.



Best Regards,
Raymond Liu

-Original Message-
From: Ted Yu [mailto:yuzhih...@gmail.com] 
Sent: Friday, January 03, 2014 1:07 PM
To: dev@spark.incubator.apache.org
Subject: compiling against hadoop 2.2

Hi,
I used the following command to compile against hadoop 2.2:
mvn clean package -DskipTests -Pnew-yarn

But I got a lot of compilation errors.

Did I use the wrong command ?

Cheers


RE: compiling against hadoop 2.2

2014-01-02 Thread Liu, Raymond
Sorry , mvn -Pnew-yarn -Dhadoop.version=2.2.0 -Dyarn.version=2.2.0 -DskipTests 
clean package

The one in previous mail not yet available.


Best Regards,
Raymond Liu


-Original Message-
From: Liu, Raymond 
Sent: Friday, January 03, 2014 2:09 PM
To: dev@spark.incubator.apache.org
Subject: RE: compiling against hadoop 2.2

I think you also need to set yarn.version

Say something like 

mvn -Pyarn -Dhadoop.version=2.2.0 -Dyarn.version=2.2.0 -DskipTests clean package


hadoop.version is default to 2.2.0 while yarn.version not when you chose the 
new-yarn profile. We probably need to fix it later for easy usage.



Best Regards,
Raymond Liu

-Original Message-
From: Ted Yu [mailto:yuzhih...@gmail.com] 
Sent: Friday, January 03, 2014 1:07 PM
To: dev@spark.incubator.apache.org
Subject: compiling against hadoop 2.2

Hi,
I used the following command to compile against hadoop 2.2:
mvn clean package -DskipTests -Pnew-yarn

But I got a lot of compilation errors.

Did I use the wrong command ?

Cheers


Re: compiling against hadoop 2.2

2014-01-02 Thread Ted Yu
Specification of yarn.version can be inserted following this line (#762 in
pom.xml), right ?
 hadoop.version2.2.0/hadoop.version


On Thu, Jan 2, 2014 at 10:10 PM, Liu, Raymond raymond@intel.com wrote:

 Sorry , mvn -Pnew-yarn -Dhadoop.version=2.2.0 -Dyarn.version=2.2.0
 -DskipTests clean package

 The one in previous mail not yet available.


 Best Regards,
 Raymond Liu


 -Original Message-
 From: Liu, Raymond
 Sent: Friday, January 03, 2014 2:09 PM
 To: dev@spark.incubator.apache.org
 Subject: RE: compiling against hadoop 2.2

 I think you also need to set yarn.version

 Say something like

 mvn -Pyarn -Dhadoop.version=2.2.0 -Dyarn.version=2.2.0 -DskipTests clean
 package


 hadoop.version is default to 2.2.0 while yarn.version not when you chose
 the new-yarn profile. We probably need to fix it later for easy usage.



 Best Regards,
 Raymond Liu

 -Original Message-
 From: Ted Yu [mailto:yuzhih...@gmail.com]
 Sent: Friday, January 03, 2014 1:07 PM
 To: dev@spark.incubator.apache.org
 Subject: compiling against hadoop 2.2

 Hi,
 I used the following command to compile against hadoop 2.2:
 mvn clean package -DskipTests -Pnew-yarn

 But I got a lot of compilation errors.

 Did I use the wrong command ?

 Cheers



RE: compiling against hadoop 2.2

2014-01-02 Thread Liu, Raymond
Yep, you are right. While we will merge in new code pretty soon ( maybe today? 
I hope so) on this part. Might shift a few lines

Best Regards,
Raymond Liu

-Original Message-
From: Ted Yu [mailto:yuzhih...@gmail.com] 
Sent: Friday, January 03, 2014 2:21 PM
To: dev@spark.incubator.apache.org
Subject: Re: compiling against hadoop 2.2

Specification of yarn.version can be inserted following this line (#762 in 
pom.xml), right ?
 hadoop.version2.2.0/hadoop.version


On Thu, Jan 2, 2014 at 10:10 PM, Liu, Raymond raymond@intel.com wrote:

 Sorry , mvn -Pnew-yarn -Dhadoop.version=2.2.0 -Dyarn.version=2.2.0 
 -DskipTests clean package

 The one in previous mail not yet available.


 Best Regards,
 Raymond Liu


 -Original Message-
 From: Liu, Raymond
 Sent: Friday, January 03, 2014 2:09 PM
 To: dev@spark.incubator.apache.org
 Subject: RE: compiling against hadoop 2.2

 I think you also need to set yarn.version

 Say something like

 mvn -Pyarn -Dhadoop.version=2.2.0 -Dyarn.version=2.2.0 -DskipTests 
 clean package


 hadoop.version is default to 2.2.0 while yarn.version not when you 
 chose the new-yarn profile. We probably need to fix it later for easy usage.



 Best Regards,
 Raymond Liu

 -Original Message-
 From: Ted Yu [mailto:yuzhih...@gmail.com]
 Sent: Friday, January 03, 2014 1:07 PM
 To: dev@spark.incubator.apache.org
 Subject: compiling against hadoop 2.2

 Hi,
 I used the following command to compile against hadoop 2.2:
 mvn clean package -DskipTests -Pnew-yarn

 But I got a lot of compilation errors.

 Did I use the wrong command ?

 Cheers



RE: compiling against hadoop 2.2

2014-01-02 Thread Liu, Raymond
And I am not sure where it is value able to providing different setting for 
hadoop/hdfs and yarn version. When build with SBT, they will always be the 
same. Maybe in mvn we should do so too. 

Best Regards,
Raymond Liu


-Original Message-
From: Liu, Raymond 
Sent: Friday, January 03, 2014 2:29 PM
To: dev@spark.incubator.apache.org
Subject: RE: compiling against hadoop 2.2

Yep, you are right. While we will merge in new code pretty soon ( maybe today? 
I hope so) on this part. Might shift a few lines

Best Regards,
Raymond Liu

-Original Message-
From: Ted Yu [mailto:yuzhih...@gmail.com]
Sent: Friday, January 03, 2014 2:21 PM
To: dev@spark.incubator.apache.org
Subject: Re: compiling against hadoop 2.2

Specification of yarn.version can be inserted following this line (#762 in 
pom.xml), right ?
 hadoop.version2.2.0/hadoop.version


On Thu, Jan 2, 2014 at 10:10 PM, Liu, Raymond raymond@intel.com wrote:

 Sorry , mvn -Pnew-yarn -Dhadoop.version=2.2.0 -Dyarn.version=2.2.0 
 -DskipTests clean package

 The one in previous mail not yet available.


 Best Regards,
 Raymond Liu


 -Original Message-
 From: Liu, Raymond
 Sent: Friday, January 03, 2014 2:09 PM
 To: dev@spark.incubator.apache.org
 Subject: RE: compiling against hadoop 2.2

 I think you also need to set yarn.version

 Say something like

 mvn -Pyarn -Dhadoop.version=2.2.0 -Dyarn.version=2.2.0 -DskipTests 
 clean package


 hadoop.version is default to 2.2.0 while yarn.version not when you 
 chose the new-yarn profile. We probably need to fix it later for easy usage.



 Best Regards,
 Raymond Liu

 -Original Message-
 From: Ted Yu [mailto:yuzhih...@gmail.com]
 Sent: Friday, January 03, 2014 1:07 PM
 To: dev@spark.incubator.apache.org
 Subject: compiling against hadoop 2.2

 Hi,
 I used the following command to compile against hadoop 2.2:
 mvn clean package -DskipTests -Pnew-yarn

 But I got a lot of compilation errors.

 Did I use the wrong command ?

 Cheers