[
https://issues.apache.org/jira/browse/SPARK-19983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rajkumar updated SPARK-19983:
-----------------------------
Description:
Hi, I am creating a DataFrame and registering that DataFrame as temp table
using df.createOrReplaceTempView('mytable'). After that I try to write the
content from 'mytable' into Hive table(It has partition) using the following
query
insert overwrite table
myhivedb.myhivetable
partition(testdate) // ( 1) : Note here : I have a partition named 'testdate'
select
Field1,
Field2,
...
TestDate //(2) : Note here : I have a field named 'TestDate' ; Both (1) & (2)
have the same name
from
mytable
when I execute this query, I am getting the following error
Exception in thread "main"
org.apache.hadoop.hive.ql.metadata.Table$ValidationFailureSemanticException:
Partition spec {testdate=, TestDate=2013-01-01}
Looks like I am getting this error because of the same field names ; ie
testdate(the partition in Hive) & TestDate (The field in temp table 'mytable')
Whereas if my partition name testdate is different from the fieldname(ie
TestDate), the query executes successuflly. Example...
insert overwrite table
myhivedb.myhivetable
partition(my_partition) //Note here the partition name is not 'testdate'
select
Field1,
Field2,
...
TestDate
from
mytable
was:
Hi, I am creating a DataFrame and registering that DataFrame as temp table
using df.createOrReplaceTempView('mytable'). After that I try to write the
content from 'mytable' into Hive table(It has partition) using the following
query
insert overwrite table
myhivedb.myhivetable
partition(testdate) // ( 1) : Note here : I have a partition named 'testdate'
select
Field1,
Field2,
...
TestDate //(2) : Note here : I have a field named 'TestDate' ; Both (1) & (2)
have the same name
from
mytable
when I execute this query, I am getting the following error
Exception in thread "main"
org.apache.hadoop.hive.ql.metadata.Table$ValidationFailureSemanticException:
Partition spec {testdate=, TestDate=2013-01-01}
Looks like I am getting this error because of the same field names ; ie
testdate(the partition in Hive) & TestDate (The field in temp table 'mytable')
Whereas if my fieldname 'TestDate' is different, the query executes
successuflly. Example...
insert overwrite table
myhivedb.myhivetable
partition(testdate)
select
Field1,
Field2,
...
myDate //Note here : The field name is 'myDate' & not 'TestDate'
from
mytable
> Getting ValidationFailureSemanticException on 'INSERT OVEWRITE'
> ---------------------------------------------------------------
>
> Key: SPARK-19983
> URL: https://issues.apache.org/jira/browse/SPARK-19983
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 2.1.0
> Reporter: Rajkumar
> Priority: Blocker
> Labels: sparkSQL
>
> Hi, I am creating a DataFrame and registering that DataFrame as temp table
> using df.createOrReplaceTempView('mytable'). After that I try to write the
> content from 'mytable' into Hive table(It has partition) using the following
> query
> insert overwrite table
> myhivedb.myhivetable
> partition(testdate) // ( 1) : Note here : I have a partition named 'testdate'
> select
> Field1,
> Field2,
> ...
> TestDate //(2) : Note here : I have a field named 'TestDate' ; Both (1) &
> (2) have the same name
> from
> mytable
> when I execute this query, I am getting the following error
> Exception in thread "main"
> org.apache.hadoop.hive.ql.metadata.Table$ValidationFailureSemanticException:
> Partition spec {testdate=, TestDate=2013-01-01}
> Looks like I am getting this error because of the same field names ; ie
> testdate(the partition in Hive) & TestDate (The field in temp table 'mytable')
> Whereas if my partition name testdate is different from the fieldname(ie
> TestDate), the query executes successuflly. Example...
> insert overwrite table
> myhivedb.myhivetable
> partition(my_partition) //Note here the partition name is not 'testdate'
> select
> Field1,
> Field2,
> ...
> TestDate
> from
> mytable
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
