[
https://issues.apache.org/jira/browse/SPARK-23194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16631652#comment-16631652
]
Daniel Mateus Pires commented on SPARK-23194:
---------------------------------------------
Any news on this ? not being able to set the from_json mode and use the
columnNameOfCorruptRecord option is pretty limiting, and the documentation of
"from_json" suggests that all the spark.read.json options are available
{code:java}
* @param options options to control how the json is parsed. accepts the same
options and the json data source.
{code}
> from_json in FAILFAST mode doesn't fail fast, instead it just returns nulls
> ---------------------------------------------------------------------------
>
> Key: SPARK-23194
> URL: https://issues.apache.org/jira/browse/SPARK-23194
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 2.3.0
> Reporter: Burak Yavuz
> Priority: Major
>
> from_json accepts Json parsing options such as being PERMISSIVE to parsing
> errors or failing fast. It seems from the code that even though the default
> option is to fail fast, we catch that exception and return nulls.
>
> In order to not change behavior, we should remove that try-catch block and
> change the default to permissive, but allow failfast mode to indeed fail.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
