Any idea what causing this error
15/08/28 21:03:03 WARN scheduler.TaskSetManager: Lost task 34.0 in stage
9.0 (TID 20, dtord01hdw0228p.dc.dotomi.net): java.lang.RuntimeException:
cannot find field message_campaign_id from
[0:error_error_error_error_error_error_error, 1:cannot_determine_schema,
, 26 Aug 2015 17:48:44 -0700
Subject: Re: query avro hive table in spark sql
To: gpatc...@gmail.com
CC: user@spark.apache.org
I'd suggest looking at http://spark-packages.org/package/databricks/spark-avro
On Wed, Aug 26, 2015 at 11:32 AM, gpatcham gpatc...@gmail.com wrote:
Hi,
I'm trying to query
, can you post the whole stack for that
error?
Most of our datasets are also in AVRO format.
Yong
--
Date: Thu, 27 Aug 2015 09:45:45 -0700
Subject: Re: query avro hive table in spark sql
From: gpatc...@gmail.com
To: java8...@hotmail.com
CC: mich
library mismatch causing it, but not sure.
I never faced your 2nd problem, can you post the whole stack for that error?
Most of our datasets are also in AVRO format.
Yong
Date: Thu, 27 Aug 2015 09:45:45 -0700
Subject: Re: query avro hive table in spark sql
From: gpatc...@gmail.com
To: java8
for our experience, but still, some non-tech
people just want to use as a SQL shell in spark, like HIVE-CLI.
Yong
--
From: mich...@databricks.com
Date: Wed, 26 Aug 2015 17:48:44 -0700
Subject: Re: query avro hive table in spark sql
To: gpatc...@gmail.com
CC: user
compiled Spark?
BTY, spark-avro works great for our experience, but still, some non-tech
people just want to use as a SQL shell in spark, like HIVE-CLI.
Yong
--
From: mich...@databricks.com
Date: Wed, 26 Aug 2015 17:48:44 -0700
Subject: Re: query avro hive table
BTY, spark-avro works great for our experience, but still, some non-tech
people just want to use as a SQL shell in spark, like HIVE-CLI.
To clarify: you can still use the spark-avro library with pure SQL. Just
use the CREATE TABLE ... USING com.databricks.spark.avro OPTIONS (path
'...')
Can you select something from this table using Hive? And also could you post
your spark code which leads to this exception.
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/query-avro-hive-table-in-spark-sql-tp24462p24468.html
Sent from the Apache Spark User
I'd suggest looking at
http://spark-packages.org/package/databricks/spark-avro
On Wed, Aug 26, 2015 at 11:32 AM, gpatcham gpatc...@gmail.com wrote:
Hi,
I'm trying to query hive table which is based on avro in spark SQL and
seeing below errors.
15/08/26 17:51:12 WARN avro.AvroSerdeUtils: