[
https://issues.apache.org/jira/browse/SPARK-61584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun resolved SPARK-61584.
-----------------------------------
Fix Version/s: 5.0.0
Resolution: Fixed
Issue resolved by pull request 60409
[https://github.com/apache/spark/pull/60409]
> Run PushablePredicateSuite without a Spark session
> --------------------------------------------------
>
> Key: SPARK-61584
> URL: https://issues.apache.org/jira/browse/SPARK-61584
> Project: Spark
> Issue Type: Improvement
> Components: SQL, Tests
> Affects Versions: 5.0.0
> Reporter: Szehon Ho
> Assignee: Szehon Ho
> Priority: Major
> Labels: pull-request-available
> Fix For: 5.0.0
>
>
> h3. Pull request
> [apache/spark#60409|https://github.com/apache/spark/pull/60409]
> h3. Motivation
> {{PushablePredicateSuite}} tests Catalyst-expression-to-V2-predicate
> conversion and never
> accesses a {{SparkSession}}. Using {{SparkFunSuite}} with {{SQLConfHelper}}
> avoids unrelated
> session startup and teardown.
> h3. Coverage traceability
> * All three tests remain.
> * All four combinations of the two predicate-related SQL configurations in
> every test remain,
> for 12 configuration executions in total.
> * Every predicate-conversion assertion remains.
> * The intentional coverage difference is that this suite no longer constructs
> a
> {{SparkSession}} or tests propagation from {{SparkConf}}. It applies the
> same ANSI and
> predicate settings directly to the {{SQLConf}} read by the code under test.
> h3. Benchmark
> Two local runs of the full suite on each commit produced:
> || Commit || Run 1 || Run 2 || Average ||
> | Parent ({{0d52f01d106}}) | 3.101 s | 2.942 s | 3.022 s |
> | PR ({{6cefcbb3d01}}) | 1.691 s | 1.619 s | 1.655 s |
> This saves 1.367 seconds, or 45.2%, on average.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]