What Spark and Hadoop versions are you on? I have it working in my Spark
app with the parquet-hive-bundle-1.5.0.jar bundled into my app fat-jar.
I¹m running Spark 1.0.2 and CDH5.
bin/spark-shell --master local[*] --driver-class-path
~/parquet-hive-bundle-1.5.0.jar
To see if that works?
On
Thanks a lot. Finally, I can create parquet table using your command
-driver-class-path.
I am using hadoop 2.3. Now, I will try to load data into the tables.
Thanks,
lyc
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Does-HiveContext-support-Parquet
.
Thanks,
lyc
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Does-HiveContext-support-Parquet-tp12209p12931.html
Sent from the Apache Spark User List mailing list archive at Nabble.com
(SparkSubmit.scala:292)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:55)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Does-HiveContext-support-Parquet
)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Many thanks for help!
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Does-HiveContext-support-Parquet-tp12209p12318.html
Sent from the Apache Spark User List mailing list archive at Nabble.com
First the JAR needs to be deployed using the ‹jars argument. Then in your
HQL code you need to use the DeprecatedParquetInputFormat and
DeprecatedParquetOutputFormat as described here
https://cwiki.apache.org/confluence/display/Hive/Parquet#Parquet-Hive0.10-0
.12
This is because SparkSQL is based
@huawei.com
Sent: 8/16/2014 12:56 AM
To: u...@spark.incubator.apache.orgmailto:u...@spark.incubator.apache.org
Subject: Re: Does HiveContext support Parquet?
Thank you for your reply.
Do you know where I can find some detailed information about how to use
Parquet in HiveContext?
Any information
Thanks for your help.
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Does-HiveContext-support-Parquet-tp12209p12231.html
Sent from the Apache Spark User List mailing list archive at Nabble.com
from all
applications?
Best,
FP
On Aug 16, 2014 5:29 PM, lyc yanchen@huawei.com wrote:
Thanks for your help.
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Does-HiveContext-support-Parquet-tp12209p12231.html
Sent from the Apache Spark User List
To: u...@spark.incubator.apache.orgmailto:u...@spark.incubator.apache.org
Subject: RE: Does HiveContext support Parquet?
Hi to all, sorry for not being fully on topic but I have 2 quick questions
about Parquet tables registered in Hive/sparq:
1) where are the created tables stored?
2) If I have
Hi to all, sorry for not being fully on topic but I have 2 quick questions
about Parquet tables registered in Hive/sparq:
Using HiveQL to CREATE TABLE will add a table to the metastore / warehouse
exactly as it would in hive. Registering is a purely temporary operation
that lives with the
://apache-spark-user-list.1001560.n3.nabble.com/Does-HiveContext-support-Parquet-tp12209.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
-
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
://apache-spark-user-list.1001560.n3.nabble.com/Does-HiveContext-support-Parquet-tp12209.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
-
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
Thank you for your reply.
Do you know where I can find some detailed information about how to use
Parquet in HiveContext?
Any information is appreciated.
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Does-HiveContext-support-Parquet-tp12209p12216.html
14 matches
Mail list logo
