[
https://issues.apache.org/jira/browse/SPARK-58507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Rosen updated SPARK-58507:
-------------------------------
Summary: Avoid redundant closure-class parsing in ClosureCleaner's
indylambda path (was: ClosureCleaner's indylambda path repeats expensive work
on every clean() call)
> Avoid redundant closure-class parsing in ClosureCleaner's indylambda path
> -------------------------------------------------------------------------
>
> Key: SPARK-58507
> URL: https://issues.apache.org/jira/browse/SPARK-58507
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
> Affects Versions: 4.0.0
> Reporter: Josh Rosen
> Assignee: Josh Rosen
> Priority: Major
> Labels: pull-request-available
>
> ClosureCleaner's indylambda path repeats expensive work on every clean() call:
> 1. Non-capturing closures are fully analyzed for nothing. SparkContext.runJob
> cleans every closure unconditionally; for a closure that captures nothing
> (e.g. the "iter => iter.toArray" that RDD.collect passes to runJob) there is
> nothing to clean, yet the cleaner still loads the capturing class and runs a
> full ASM parse of its bytecode before discarding the result. ~15% of
> indylambda cleans in certain CI test workloads are non-capturing.
> 2. The return-statement fail-fast re-parses the same bytecode over and over.
> The verdict of ReturnStatementFinder is a pure function of the capturing
> class's immutable bytecode, and capturing classes repeat heavily across jobs
> because they are mostly Spark's own classes (SparkContext, RDD, Dataset,
> WholeStageCodegenExec): a DataFrameSuite run performed 3,251 full ASM parses
> over just 21 distinct classes (155x repeat ratio); an RDDSuite run, 1,306
> over 11 (119x).
> Measured cost (JFR profiling):
> - Cleaning-related frames appear in 9.8-13.2% of test-JVM CPU samples across
> six suites, including core-only RDDSuite.
> - Per-call parse cost scales with the capturing class's file size: ~46 us
> for a 9 KB class, ~340 us for a 113 KB one; RDD.class and SparkContext.class
> are ~190-205 KB.
> - In a loop of minimal collect()/count() jobs, roughly half of per-job
> driver time goes to re-parsing SparkContext.class and RDD.class (~5 parses
> per job). This overhead is paid on every job submission, so short interactive
> queries and streaming microbatches see it most.
> Proposed improvement:
> - Check SerializedLambda.getCapturedArgCount == 0 (an O(1) read of a field
> the cleaner already holds) before loading and parsing the capturing class,
> skipping the work entirely for non-capturing closures. This is safe with
> respect to the return-statement fail-fast because a non-local return captures
> its NonLocalReturnControl key, so such a closure always has at least one
> captured argument.
> - Memoize the return-statement verdict per class in a java.lang.ClassValue,
> using a collecting visitor so a single parse answers for every method of the
> class. ClassValue entries are reclaimed with the class, so runtime-generated
> closure classes (REPL lines, Ammonite commands) are not pinned.
> With the change, cleaning-related frames fall to ~0% of CPU samples
> (xbean.asm9: 11.02% -> 0.17% on the profiled suite), getClassReader is
> invoked once per distinct class instead of once per job, and RDDSuite wall
> clock drops ~8.5%.
> Behavior is unchanged, including the ReturnStatementInClosureException
> fail-fast for every capturing closure.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
