Any idea what causing this error
15/08/28 21:03:03 WARN scheduler.TaskSetManager: Lost task 34.0 in stage
9.0 (TID 20, dtord01hdw0228p.dc.dotomi.net): java.lang.RuntimeException:
cannot find field message_campaign_id from
[0:error_error_error_error_error_error_error, 1:cannot_determine_schema,
, 26 Aug 2015 17:48:44 -0700
Subject: Re: query avro hive table in spark sql
To: gpatc...@gmail.com
CC: user@spark.apache.org
I'd suggest looking at http://spark-packages.org/package/databricks/spark-avro
On Wed, Aug 26, 2015 at 11:32 AM, gpatcham gpatc...@gmail.com wrote:
Hi,
I'm trying to query
, can you post the whole stack for that
error?
Most of our datasets are also in AVRO format.
Yong
--
Date: Thu, 27 Aug 2015 09:45:45 -0700
Subject: Re: query avro hive table in spark sql
From: gpatc...@gmail.com
To: java8...@hotmail.com
CC: mich
library mismatch causing it, but not sure.
I never faced your 2nd problem, can you post the whole stack for that error?
Most of our datasets are also in AVRO format.
Yong
Date: Thu, 27 Aug 2015 09:45:45 -0700
Subject: Re: query avro hive table in spark sql
From: gpatc...@gmail.com
To: java8
for our experience, but still, some non-tech
people just want to use as a SQL shell in spark, like HIVE-CLI.
Yong
--
From: mich...@databricks.com
Date: Wed, 26 Aug 2015 17:48:44 -0700
Subject: Re: query avro hive table in spark sql
To: gpatc...@gmail.com
CC: user
schema
using url. I'm stuck and couldn't find more info on this.
Any pointers ?
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/query-avro-hive-table-in-spark-sql-tp24462.html
Sent from the Apache Spark User List mailing list archive at Nabble.com
BTY, spark-avro works great for our experience, but still, some non-tech
people just want to use as a SQL shell in spark, like HIVE-CLI.
To clarify: you can still use the spark-avro library with pure SQL. Just
use the CREATE TABLE ... USING com.databricks.spark.avro OPTIONS (path
'...')
Can you select something from this table using Hive? And also could you post
your spark code which leads to this exception.
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/query-avro-hive-table-in-spark-sql-tp24462p24468.html
Sent from the Apache Spark User
)
Its not able to determine schema. Hive table is pointing to avro schema
using url. I'm stuck and couldn't find more info on this.
Any pointers ?
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/query-avro-hive-table-in-spark-sql-tp24462.html
Sent from
on this.
Any pointers ?
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/query-avro-hive-table-in-spark-sql-tp24462.html
Sent from the Apache Spark User List mailing list archive at Nabble.com
10 matches
Mail list logo