Yin Huai created SPARK-4077:
-------------------------------
Summary: A broken string timestamp value can Spark SQL return
wrong values for valid string timestamp values
Key: SPARK-4077
URL: https://issues.apache.org/jira/browse/SPARK-4077
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 1.1.0
Reporter: Yin Huai
The following case returns wrong results.
The text file is
{code}
2014-12-11 00:00:00,1
2014-12-11astring00:00:00,2
{code}
The DDL statement and the query are shown below...
{code}
sql("""
create external table date_test(my_date timestamp, id int)
row format delimited
fields terminated by ','
lines terminated by '\n'
LOCATION 'dateTest'
""")
sql("select * from date_test").collect.foreach(println)
{code}
The result is
{code}
[1969-12-31 19:00:00.0,1]
[null,2]
{code}
If I change the data to
{code}
2014-12-11 00:00:00,1
2014-12-11 00:00:00,2
{code}
The result is fine.
For the data with broken string timestamp value, I tried runSqlHive. The result
is fine.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
