[
https://issues.apache.org/jira/browse/FLINK-31709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18097057#comment-18097057
]
Prashant Bhardwaj commented on FLINK-31709:
-------------------------------------------
Hey [~mapohl],
Can you assign this to me? I recently hit this exact gap while investigating
FLINK-39989
I traced the root cause through both the operator and Flink core:
- After a job reaches a globally terminal state, HA correctly drops it from the
ExecutionPlanStore
- The JobResultStore records the terminal result, but with delete-on-commit:
true (default) it's deleted after cleanup
- Even if retained (delete-on-commit: false), no REST endpoint exposes
JobResultStore data - /jobs/overview and /jobs/:jobid only consult in-memory
stores
- After a JM restart, the job is invisible to any external consumer (including
the operator), which has no way to distinguish "job completed" from "job never
existed"
This is exactly the gap you described - merging JobResultStore and
ExecutionGraphInfoStore (or at least exposing terminal job results via REST)
would let the operator observe what happened to a job that completed while it
was down.
I've reproduced the issue on Minikube (operator 1.14.0, Flink 2.2) and have a
detailed code trace of both sides. Happy to write the FLIP for this.
> JobResultStore and ExecutionGraphInfoStore could be merged
> ----------------------------------------------------------
>
> Key: FLINK-31709
> URL: https://issues.apache.org/jira/browse/FLINK-31709
> Project: Flink
> Issue Type: New Feature
> Components: Runtime / Coordination
> Reporter: Matthias Pohl
> Priority: Major
>
> This is a initial proposal for an improvement in coordination layer:
> The {{JobResultStore}} (JRS) was introduced as part of
> [FLIP-194|https://cwiki.apache.org/confluence/display/FLINK/FLIP-194%3A+introduce+the+jobresultstore].
> For now, it only stores the JobResult. Through the JRS, jobs can be marked
> as finished even when the JobManager fails and the information from the
> {{ExecutionGraphInfoStore}} is lost (see FLINK-11813).
> While implementing {{FLIP-194}}, it became apparent, that we have some
> redundancy between the JRS and the {{ExecutionGraphInfoStore}}. Both
> components store some meta information of a finished job. The
> {{ExecutionGraphInfoStore}} is used to make information about the finished
> job available in user-facing APIs (REST, web-UI). The JRS is used to expose
> the job's state to the cleanup logic and stores limited data.
> This proposal is about merging the two and making the
> {{ArchivedExecutionGraph}} information available even after a JobManager is
> restarted. That way, completed jobs can be still listed in the job overview
> after a Flink cluster restart. Additionally, we could provide the last
> checkpoint information. The JRS would be a way to access this information
> even after the Flink cluster is shut down. The latter feature would be also a
> way to improve the Flink Kubernetes Operator's latest-state handling.
> I want to point out that this issue is just a braindump. It might require
> coming up with a proper FLIP since we need to discuss what information should
> be made available through the JRS and whether that's considered public API in
> some way.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
