Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/23201#discussion_r240156871
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JsonInferSchema.scala
---
@@ -121,7 +122,26 @@ private[sql] class JsonInferSchema(options:
JSONOptions) extends Serializable {
DecimalType(bigDecimal.precision, bigDecimal.scale)
}
decimalTry.getOrElse(StringType)
- case VALUE_STRING => StringType
+ case VALUE_STRING =>
+ val stringValue = parser.getText
--- End diff --
> If we switch the order here, we don't need the length check here, right?
@cloud-fan, that works only if we use default date/timestamp patterns. Both
should do the exact match with pattern, which unfortunately the current parsing
library (SimpleDateFormat) does not allow.
The order here is just to make it look better and both shouldn't be
dependent on its order. I think we should support those inferences after
completely switching the library to `java.time.format.*` without a legacy. That
should make this change easier without a hole.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
