[jira] [Assigned] (SPARK-41248) Add config flag to control before of JSON partial results parsing in SPARK-40646
[ https://issues.apache.org/jira/browse/SPARK-41248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-41248: Assignee: Ivan Sadikov > Add config flag to control before of JSON partial results parsing in > SPARK-40646 > > > Key: SPARK-41248 > URL: https://issues.apache.org/jira/browse/SPARK-41248 > Project: Spark > Issue Type: Improvement > Components: SQL >Affects Versions: 3.4.0 >Reporter: Ivan Sadikov >Assignee: Ivan Sadikov >Priority: Major > Attachments: json-benchmark-with-SPARK-40646.log, > json-benchmark-without-SPARK-40646.log > > > This is a follow-up for https://issues.apache.org/jira/browse/SPARK-40646. > > It was observed in internal benchmarks that the JSON partial results parsing > can be 30% slower compared to parsing without the patch. I could not find a > regression and the Apache Spark JSON benchmark results are very similar with > and without SPARK-40646. > However, I would still like to add a config flag to enable/disable the > feature in the case the regression is observed in users' queries. > Benchmark results are attached below. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-41248) Add config flag to control before of JSON partial results parsing in SPARK-40646
[ https://issues.apache.org/jira/browse/SPARK-41248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41248: Assignee: Apache Spark > Add config flag to control before of JSON partial results parsing in > SPARK-40646 > > > Key: SPARK-41248 > URL: https://issues.apache.org/jira/browse/SPARK-41248 > Project: Spark > Issue Type: Improvement > Components: SQL >Affects Versions: 3.4.0 >Reporter: Ivan Sadikov >Assignee: Apache Spark >Priority: Major > > This is a follow-up for https://issues.apache.org/jira/browse/SPARK-40646. > > It was observed in internal benchmarks that the JSON partial results parsing > can be 30% slower compared to parsing without the patch. I could not find a > regression and the OSS benchmark results are very similar with and without > SPARK-40646. > However, I would still like to add a config flag to enable/disable the > feature in the case the regression is observed in users' queries. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-41248) Add config flag to control before of JSON partial results parsing in SPARK-40646
[ https://issues.apache.org/jira/browse/SPARK-41248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41248: Assignee: (was: Apache Spark) > Add config flag to control before of JSON partial results parsing in > SPARK-40646 > > > Key: SPARK-41248 > URL: https://issues.apache.org/jira/browse/SPARK-41248 > Project: Spark > Issue Type: Improvement > Components: SQL >Affects Versions: 3.4.0 >Reporter: Ivan Sadikov >Priority: Major > > This is a follow-up for https://issues.apache.org/jira/browse/SPARK-40646. > > It was observed in internal benchmarks that the JSON partial results parsing > can be 30% slower compared to parsing without the patch. I could not find a > regression and the OSS benchmark results are very similar with and without > SPARK-40646. > However, I would still like to add a config flag to enable/disable the > feature in the case the regression is observed in users' queries. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
