[
https://issues.apache.org/jira/browse/SPARK-27726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16845109#comment-16845109
]
David C Navas commented on SPARK-27726:
---------------------------------------
sub-tasks closed as dupes against this umbrella, and linked as duplicates of
this ticket.
> Performance of InMemoryStore suffers under load
> -----------------------------------------------
>
> Key: SPARK-27726
> URL: https://issues.apache.org/jira/browse/SPARK-27726
> Project: Spark
> Issue Type: Umbrella
> Components: Spark Core
> Affects Versions: 2.3.3
> Reporter: David C Navas
> Assignee: David C Navas
> Priority: Major
> Fix For: 2.4.4, 3.0.0
>
> Attachments: GCRateIssues.pdf, PerformanceBeforeAndAfter.pdf
>
>
> When our Spark system has been under load for an extended period of time, GC
> remains highly active and the jobs page becomes unresponsive even when load
> eases. Please see the attached GCRateIssues for more details regarding the
> problem definition.
> We found a number of separate issues which are detailed in the subtasks. I
> anticipate committing a single PR for all subtasks whose commits roughly
> align with the descriptions in the subtasks.
> The performance of the code is measured before and after the change and is
> attached in the document PerformanceBeforeAndAfter. tl;dr in our use case,
> we saw about five (computed) orders of magnitude improvement.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
