Hello All,
I met a problem of file not exist. it looks like the store location
can’t be found. I have already set
carbon.store.location=hdfs://localhost:9000/carbon/store
<hdfs://localhost:9000/carbon/store> in $SPARK_HOME/conf/carbon.properties, but
when I start up spark-shell by following command and run some commands ,the
error is coming
spark-shell --master spark://localhost:7077 --jars
~/carbonlib/carbondata_2.11-1.0.0-incubating-shade-hadoop2.7.2.jar --conf
spark.carbon.storepath=hdfs://localhost:9000/carbon/store
<hdfs://localhost:9000/carbon/store>
scala> import org.apache.spark.sql.SparkSession
scala> import org.apache.spark.sql.CarbonSession._
scala> val carbon =
SparkSession.builder().config(sc.getConf).getOrCreateCarbonSession()
scala> carbon.sql("CREATE TABLE IF NOT EXISTS test_table(id string, name
string, city string, age Int) STORED BY 'carbondata’")
scala> carbon.sql("load data inpath
'hdfs://localhost:9000/resources/sample.csv' into table test_table”)
scala> carbon.sql("select * from test_table").show()
java.io.FileNotFoundException: File
/private/var/carbon.store/default/test_table/Fact/Part0/Segment_0 does not
exist.
at
org.apache.hadoop.hdfs.DistributedFileSystem$DirListingIterator.<init>(DistributedFileSystem.java:948)
at
org.apache.hadoop.hdfs.DistributedFileSystem$DirListingIterator.<init>(DistributedFileSystem.java:927)
My carbonate version is 1.0 and spark version is spark 2.1.
