I tried with simple spark-hive select and insert, and it works. But to directly
manipulate the ORCFile through RDD, spark has to be upgraded to support
hive-0.13 first. Because some ORC API is not exposed until Hive-0.12.
Thanks.
Zhan Zhang
On Aug 11, 2014, at 10:23 PM,
Yes. You are right, but I tried old hadoopFile for OrcInputFormat. In hive12,
OrcStruct is not exposing its api, so spark cannot access it. With Hive13, RDD
can read from OrcFile. Btw, I didn’t see ORCNewOutputFormat in hive-0.13.
Direct RDD manipulation (Hive13)
val inputRead =
I agree. We need the support similar to parquet file for end user. That’s the
purpose of Spark-2883.
Thanks.
Zhan Zhang
On Aug 14, 2014, at 11:42 AM, Yin Huai huaiyin@gmail.com wrote:
I feel that using hadoopFile and saveAsHadoopFile to read and write ORCFile
are more towards
I would expect this to work with Spark SQL (available in 1.0+) but there is
a JIRA open to confirm this works SPARK-2883
https://issues.apache.org/jira/browse/SPARK-2883.
On Mon, Aug 11, 2014 at 10:23 PM, vinay.kash...@socialinfra.net wrote:
Hi all,
Is it possible to use table with ORC
Thanks Micheal for the info.
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Support-for-ORC-Table-in-Shark-Spark-tp11952p12089.html
Sent from the Apache Spark User List mailing list archive at Nabble.com
Hi all,
Is it possible to use table with ORC format in Shark
version 0.9.1 with Spark 0.9.2 and Hive version 0.12.0..??
I have
tried creating the ORC table in Shark using the below
query
create table orc_table (x int, y string) stored as
orc
create table works, but when I try to insert values
