[
https://issues.apache.org/jira/browse/FLINK-26175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matthias Pohl updated FLINK-26175:
----------------------------------
Description:
We introduce new interfaces for local cleanup and global cleanup in
FLINK-25432. The initial plan was to call them independently of each other. The
motivation behind it was to allow independent cleanup logic (e.g. in the
{{DefaultJobGraphStore}} for releasing and deleting the data from the
{{StateHandleStore}}).
This can be improved by adding an order to cleanup and calling the local
cleanup on each resource before calling the corresponding global cleanup in the
Dispatcher.
This works under the assumption that it's ok to keep the resource in an
inconsistent state during cleanup (i.e. it's ok for the resource to be in a
state were the local artifacts are already cleaned but the global artifacts are
not). That holds true because of how the {{Dispatcher}} interacts with the
{{JobResultStore}}. It functions as a transaction manager for the cleanup here.
was:
We introduce new interfaces for local cleanup and global cleanup in
FLINK-25432. The initial plan was to call them independently of each other. The
motivation behind it was to allow independent cleanup logic (e.g. in the
`DefaultJobGraphStore` for releasing and deleting the data from the
`StateHandleStore`).
This can be improved by adding an order to cleanup and calling the local
cleanup on each resource before calling the corresponding global cleanup in the
Dispatcher.
This works under the assumption that it's ok to keep the resource in an
inconsistent state during cleanup (i.e. it's ok for the resource to be in a
state were the local artifacts are already cleaned but the global artifacts are
not). That holds true because of how the {{Dispatcher}} interacts with the
{{JobResultStore}}. It functions as a transaction manager for the cleanup here.
> Make local cleanup being called in global cleanup situations
> ------------------------------------------------------------
>
> Key: FLINK-26175
> URL: https://issues.apache.org/jira/browse/FLINK-26175
> Project: Flink
> Issue Type: Technical Debt
> Components: Runtime / Coordination
> Affects Versions: 1.16.0
> Reporter: Matthias Pohl
> Priority: Major
>
> We introduce new interfaces for local cleanup and global cleanup in
> FLINK-25432. The initial plan was to call them independently of each other.
> The motivation behind it was to allow independent cleanup logic (e.g. in the
> {{DefaultJobGraphStore}} for releasing and deleting the data from the
> {{StateHandleStore}}).
> This can be improved by adding an order to cleanup and calling the local
> cleanup on each resource before calling the corresponding global cleanup in
> the Dispatcher.
> This works under the assumption that it's ok to keep the resource in an
> inconsistent state during cleanup (i.e. it's ok for the resource to be in a
> state were the local artifacts are already cleaned but the global artifacts
> are not). That holds true because of how the {{Dispatcher}} interacts with
> the {{JobResultStore}}. It functions as a transaction manager for the cleanup
> here.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
