[
https://issues.apache.org/jira/browse/FLINK-6295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15972583#comment-15972583
]
ASF GitHub Bot commented on FLINK-6295:
---------------------------------------
Github user zentol commented on the issue:
https://github.com/apache/flink/pull/3709
That this happens when HA is enabled is a really important detail; finally
figured out what happens.
Here's roughly what going on:
* EG A is cached by EGHolder for ID_A
* ZK goes down
* JM revokes leadership, throws out all jobs, specifically EG A stored
under ID_A
* ZK starts up again
* (in your case the same) JM gets leadership back
* JM recovers jobs, which means creating a new EG B, the ID of which is
ID_A again
When a new request hits the EGHolder cache for ID_A there's still the old
EG cached. That old EG will remain there until the GC kicks in, from which
point on the new EG is used.
> use LoadingCache instead of WeakHashMap to lower latency
> --------------------------------------------------------
>
> Key: FLINK-6295
> URL: https://issues.apache.org/jira/browse/FLINK-6295
> Project: Flink
> Issue Type: Bug
> Components: Webfrontend
> Reporter: Tao Wang
> Assignee: Tao Wang
>
> Now in ExecutionGraphHolder, which is used in many handlers, we use a
> WeakHashMap to cache ExecutionGraph(s), which is only sensitive to garbage
> collection.
> The latency is too high when JVM do GC rarely, which will make status of jobs
> or its tasks unmatched with the real ones.
> LoadingCache is a common used cache implementation from guava lib, we can use
> its time based eviction to lower latency of status update.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
