[
https://issues.apache.org/jira/browse/SPARK-23194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16674468#comment-16674468
]
Maxim Gekk commented on SPARK-23194:
------------------------------------
This changes
https://github.com/apache/spark/commit/4d6704db4d490bd1830ed3c757525f41058523e0
in from_json supported PERMISSIVE and FAILFAST modes as well as the
columnNameOfCorruptRecord option. The ticket can be closed probably.
> from_json in FAILFAST mode doesn't fail fast, instead it just returns nulls
> ---------------------------------------------------------------------------
>
> Key: SPARK-23194
> URL: https://issues.apache.org/jira/browse/SPARK-23194
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 2.3.0
> Reporter: Burak Yavuz
> Priority: Major
>
> from_json accepts Json parsing options such as being PERMISSIVE to parsing
> errors or failing fast. It seems from the code that even though the default
> option is to fail fast, we catch that exception and return nulls.
>
> In order to not change behavior, we should remove that try-catch block and
> change the default to permissive, but allow failfast mode to indeed fail.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
