Hi all, I am writing to initiate a discussion on enhancing Spark JSON parsing to support standard compliance.
## Motivation In the current version of Spark, the JSON parser is designed to be compatible with Hive, which means that some behaviors may not adhere strictly to standard JSON compliance. For instance, it permits the use of single quotes in JSON parsing for functions such as get_json_object and json_tuple. Additionally, there are other instances where the behavior diverges from standard JSON practices. ## Proposal To address these inconsistencies, we propose introducing a configuration option that allows users to choose between legacy behavior and standard compliance. This approach ensures that existing workflows remain unaffected while providing the flexibility to adopt standard JSON practices if desired. See JIRA: https://issues.apache.org/jira/browse/SPARK-53281 Thanks, Philo
