wuyi created SPARK-35330:
----------------------------
Summary: Tune shuffle requests frequency dynamically in the case
of Netty OOM
Key: SPARK-35330
URL: https://issues.apache.org/jira/browse/SPARK-35330
Project: Spark
Issue Type: Improvement
Components: Spark Core
Affects Versions: 3.2.0
Reporter: wuyi
In [https://github.com/apache/spark/pull/32287,] the PR proposes to use a flag
to indicate the Netty OOM status and defer the fetch requests when OOM happens.
However, it doesn't change the request frequency after OOM happens. Thus, it's
possible to hit the OOM again later if the deferred requests are still too much
to fetch concurrently. Therefore, tuning the fetch frequency might be a good
idea to solve the issue. Please see the detailed discussion at
https://github.com/apache/spark/pull/32287#discussion_r625287419
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
