Veenit Shah created SPARK-26325: ----------------------------------- Summary: Interpret timestamp fields in Spark while reading json (timestampFormat) Key: SPARK-26325 URL: https://issues.apache.org/jira/browse/SPARK-26325 Project: Spark Issue Type: Bug Components: Spark Core Affects Versions: 2.4.0 Reporter: Veenit Shah
I am trying to read a pretty printed json which has time fields in it. I want to interpret the timestamps columns as timestamp fields while reading the json itself. However, it's still reading them as string when I {{printSchema}} E.g. Input json file - {code:java} [{ "time_field" : "2017-09-30 04:53:39.412496Z" }] {code} Code - {code:java} df = spark.read.option("multiLine", "true").option("timestampFormat","yyyy-MM-dd HH:mm:ss.SSSSSS'Z'").json('path_to_json_file') {code} Output of df.printSchema() - {code:java} root |-- time_field: string (nullable = true) {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org