bing huang created SPARK-20879:
----------------------------------
Summary: Spark SQL will read Date type column from avro file as Int
Key: SPARK-20879
URL: https://issues.apache.org/jira/browse/SPARK-20879
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 2.1.1, 2.1.0, 2.0.2, 2.0.1, 2.0.0, 1.6.3, 1.6.2, 1.6.1,
1.6.0
Reporter: bing huang
I was using the following code and attached avro file.
In the avro file we have 5 columns named (date1, date2, date3,
start_date,end_date) with type (string, string, date, string, string)
respectively.
You can use hive to verify the type of columns either by "select * from
000000_0.avro" or "describe 000000_0.avro".
The code of spark sql as below, as you can see the 3rd column's result is int.
val conf = new SparkConf().setAppName("test").setMaster("local[3]")
val sc = new SparkContext(conf)
val sqlContext = new SQLContext(sc)
val df = sqlContext.read.avro("000000_0.avro")
df.show(false)
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
