Re: Support for spark 2.1.1 (HDP spark versions)

[Liang Chen]

Hi

Did you successfully compile with "mvn -DskipTests -Pspark-2.1
-Dspark.version=2.1.1 clean package" ?
I used the community spark 2.1.1, it is working fine.

Regards
Liang


satish.sidnakoppa wrote
> ;
>  
> 
> The carbon supported spark versions and Hortonworks - HDP supported
> versions
> are different.
> I am unable to integrate carbondata and HDP .
> 
> In brief , how can I build carbondata jar to on spark 2.1.1
> Because I am working on HDP 2.6.2 that has spark 2.1.1 as MAJOR version
> whereas carbondata default support is 2.1.0
> 
> Due to this when I run carbon session creation.I get below error
> 
> val cc =
> SparkSession.builder().config(sc.getConf).getOrCreateCarbonSession(hdfs
> path)
> and got error:
> java.lang.NoClassDefFoundError:
> org/apache/spark/sql/catalyst/CatalystConf 
> 
> 
> 
> --
> Sent from:
> http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/





--
Sent from: 
http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/

Hive cannot query carbondata

[tiany]

  
Dear user:
   Version:
CDH version 5.10.0 ,hive version 1.2.1(Manual upgrade version,Original 
version 1.1.0),spark-2.3.2-bin-2.6.0-cdh5.15.0 
,carbondata:apache-carbondata-1.5.2-bin-spark2.3.2-hadoop2.6.0-cdh5.15.0.jar 
   See Annex:spark-defaults.conf carbon.properties 
  Wrong description:
 Through the hive client login,
  table:select * from  ty_test.origin2,
  error: org.apache.hadoop.hive.ql.parse.SemanticException: 1:15 Input format 
must implement InputFormat. Error encountered near token 'origin2'
   Operation steps:
1.Through the Spark thrif login.
2.Execute SQL "CREATE TABLE origin2(name String, PhoneNumber 
String) STORED AS carbondata;" Prompt success  
3.Through the hive client login,
 4..Execute SQL "select * from ty_test.origin2",This error 
occurred : error: org.apache.hadoop.hive.ql.parse.SemanticException: 1:15 Input 
format must implement InputFormat. Error encountered near token 'origin2'
  Error Details files error.txt

2019-02-28



田游
电话：0755-26037910 
传真：0755-26037912 邮编：518057 
手机：13896099076  Email：ti...@broadtech.com.cn
. If you receive this e-mail in error, please notify the sender by phone or 
email immediately and delete it !

 

Support for spark 2.1.1 (HDP spark versions)

[satish.sidnakoppa]

 

The carbon supported spark versions and Hortonworks - HDP supported versions
are different.
I am unable to integrate carbondata and HDP .

In brief , how can I build carbondata jar to on spark 2.1.1
Because I am working on HDP 2.6.2 that has spark 2.1.1 as MAJOR version
whereas carbondata default support is 2.1.0

Due to this when I run carbon session creation.I get below error

val cc =
SparkSession.builder().config(sc.getConf).getOrCreateCarbonSession(hdfs
path)
and got error:
java.lang.NoClassDefFoundError:
org/apache/spark/sql/catalyst/CatalystConf 



--
Sent from: 
http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/