Re: Support for spark 2.1.1 (HDP spark versions)

[satish.sidnakoppa]

I tried to build using
mvn -DskipTests -Pspark-2.1 -Dspark.version=2.1.1 clean package

Build failing for spark 2.1.1


[INFO] Apache CarbonData :: Parent  SUCCESS [  3.981
s]
.
.
.
[INFO] Apache CarbonData :: Bloom Index DataMap ... SUCCESS [ 13.166
s]
[INFO] Apache CarbonData :: Spark2  FAILURE [ 27.384
s]

ERROR] Failed to execute goal
org.scala-tools:maven-scala-plugin:2.15.2:compile (default) on project
carbondata-spark2: wrap: org.apache.commons.exec.ExecuteException: Process
exited with an error: 1(Exit value: 1) -> [Help 1]


But the build is failing




-
regards
Satish
--
Sent from: 
http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/

Re: Support for spark 2.1.1 (HDP spark versions)

[Liang Chen]

Hi

Did you successfully compile with "mvn -DskipTests -Pspark-2.1
-Dspark.version=2.1.1 clean package" ?
I used the community spark 2.1.1, it is working fine.

Regards
Liang


satish.sidnakoppa wrote
> http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/file/t389/versions.png;
>  
> 
> The carbon supported spark versions and Hortonworks - HDP supported
> versions
> are different.
> I am unable to integrate carbondata and HDP .
> 
> In brief , how can I build carbondata jar to on spark 2.1.1
> Because I am working on HDP 2.6.2 that has spark 2.1.1 as MAJOR version
> whereas carbondata default support is 2.1.0
> 
> Due to this when I run carbon session creation.I get below error
> 
> val cc =
> SparkSession.builder().config(sc.getConf).getOrCreateCarbonSession(hdfs
> path)
> and got error:
> java.lang.NoClassDefFoundError:
> org/apache/spark/sql/catalyst/CatalystConf 
> 
> 
> 
> --
> Sent from:
> http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/





--
Sent from: 
http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/

Support for spark 2.1.1 (HDP spark versions)

[satish.sidnakoppa]

 

The carbon supported spark versions and Hortonworks - HDP supported versions
are different.
I am unable to integrate carbondata and HDP .

In brief , how can I build carbondata jar to on spark 2.1.1
Because I am working on HDP 2.6.2 that has spark 2.1.1 as MAJOR version
whereas carbondata default support is 2.1.0

Due to this when I run carbon session creation.I get below error

val cc =
SparkSession.builder().config(sc.getConf).getOrCreateCarbonSession(hdfs
path)
and got error:
java.lang.NoClassDefFoundError:
org/apache/spark/sql/catalyst/CatalystConf 



--
Sent from: 
http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/