hello,
        I tried to use branch-1.1 under hdp2.6.0-spark2.1.0 and met so many 
problems.
        1.build: use "mvn package -DskipTests -Pspark-2.1 -Dspark.version=2.1.0 
-Phadoop-2.7.2" and 
        while compiling core module,throw class not found error,like 
org.apache.thrift.TBase,
        modify pom.xml in core and add libthrift dependency,compile 
successfully.
        2.running in spark-shell like ;  
            spark-shell --jars 
carbondata_2.11-1.1.1-SNAPSHOT-shade-hadoop2.7.2.jar
            scala>import org.apache.spark.sql.SparkSession
            scala>import org.apache.spark.sql.CarbonSession._
            scala>val cc = 
SparkSession.builder().config(sc.getConf).getOrCreateCarbonSession(hdfs path)
            and got error:
            java.lang.NoClassDefFoundError: 
org/apache/spark/sql/catalyst/CatalystConf
at 
org.apache.spark.sql.hive.CarbonSessionState.analyzer$lzycompute(CarbonSessionState.scala:127)
at 
org.apache.spark.sql.hive.CarbonSessionState.analyzer(CarbonSessionState.scala:126)
at 
org.apache.spark.sql.execution.QueryExecution.analyzed$lzycompute(QueryExecution.scala:69)
at 
org.apache.spark.sql.execution.QueryExecution.analyzed(QueryExecution.scala:67)
at 
org.apache.spark.sql.execution.QueryExecution.assertAnalyzed(QueryExecution.scala:50)
at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:63)
at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:592)
... 52 elided
Caused by: java.lang.ClassNotFoundException: 
org.apache.spark.sql.catalyst.CatalystConf
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 59 more
check spark-catalyst.jar and didn't found class CatalystConf
3.use apache-spark catalyst jars and rerun with 
spark-shell --jars 
carbondata_2.11-1.1.1-SNAPSHOT-shade-hadoop2.7.2.jar,spark-catalyst_2.11-2.1.0.jar
and got error message like:
java.lang.NoClassDefFoundError: org/apache/spark/sql/catalyst/CatalystConf
at 
org.apache.spark.sql.hive.CarbonSessionState.analyzer$lzycompute(CarbonSessionState.scala:127)
at 
org.apache.spark.sql.hive.CarbonSessionState.analyzer(CarbonSessionState.scala:126)
at 
org.apache.spark.sql.execution.QueryExecution.analyzed$lzycompute(QueryExecution.scala:69)
at 
org.apache.spark.sql.execution.QueryExecution.analyzed(QueryExecution.scala:67)
at 
org.apache.spark.sql.execution.QueryExecution.assertAnalyzed(QueryExecution.scala:50)
at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:63)
at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:592)
... 52 elided
Caused by: java.lang.ClassNotFoundException: 
org.apache.spark.sql.catalyst.CatalystConf
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 59 more



sunerhan1...@sina.com

Reply via email to