[
https://issues.apache.org/jira/browse/SPARK-46710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mars updated SPARK-46710:
-------------------------
Description:
Broadcast data cleaning can only rely on cleaning when GC is triggered, which
may lead to a lot of waste of memory usage , and may also cause query
instability if a single GC takes too long.
After the execution of sql is completed, the broadcast data generated during
the execution of the sql can be cleaned to reduce memory on driver or executor.
was:Faster cleaning of broadcast data generated by sql is beneficial to
saving driver/executor memory and avoiding long-term GC. This can make a long
running spark service more stable
> Clean up the broadcast data generated when sql execution ends
> -------------------------------------------------------------
>
> Key: SPARK-46710
> URL: https://issues.apache.org/jira/browse/SPARK-46710
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 4.0.0
> Reporter: Mars
> Priority: Major
> Labels: pull-request-available
>
> Broadcast data cleaning can only rely on cleaning when GC is triggered, which
> may lead to a lot of waste of memory usage , and may also cause query
> instability if a single GC takes too long.
> After the execution of sql is completed, the broadcast data generated during
> the execution of the sql can be cleaned to reduce memory on driver or
> executor.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
