[
https://issues.apache.org/jira/browse/FLINK-11874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16790221#comment-16790221
]
Yun Tang commented on FLINK-11874:
----------------------------------
[~yanghua] Thanks for your suggestion.
I thought I had given the why-what-how in the JIRA's description:
*why*: Current interface {{CheckpointStorage}} mixed JM and TM side, which
would be confusing for users and developers to distinguish them. Especially
when developers implement some function which should not be called in TM side
such as init directories.
*what*: {{CheckpointStorage}} would not exist in TM side but also in JM side,
in which they acted as different roles.
*how*: Split JM's side methods out of current {{CheckpointStorage}} interface
to a new interface such as {{CheckpointCoordinatorStorage}}. By using this, we
could cast {{CheckpointStorage}} to {{CheckpointCoordinatorStorage}} within
checkpoint coordinator so that those methods could be called only in JM side.
This JIRA was first inspired when we review
[FLINK-11696|https://github.com/apache/flink/pull/7942#discussion_r264335302]'s
PR, it would be really very helpful to point out anything that should do.
If you found anything deserved to be discussed more clearly to send a design
proposal, you could leave your suggestions and concerns in details :)
> Split CheckpointStorage interface to distinguish JM and TM side
> ---------------------------------------------------------------
>
> Key: FLINK-11874
> URL: https://issues.apache.org/jira/browse/FLINK-11874
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Checkpointing
> Reporter: Yun Tang
> Assignee: Yun Tang
> Priority: Major
> Fix For: 1.9.0
>
>
> Currently, interface {{CheckpointStorage}} mixed JM and TM side, which would
> be confusing for users and developers to distinguish them.
> Take [FLINK-11696|https://issues.apache.org/jira/browse/FLINK-11696] as an
> example, the directories should only be created once from JM side. However,
> since we mixed the JM and TM side, TMs would also create directories again.
> We could let interface {{CheckpointStorage}} only has two methods:
> {code:java}
> CheckpointStreamFactory resolveCheckpointStorageLocation(
> long checkpointId,
> CheckpointStorageLocationReference reference)
> CheckpointStateOutputStream createTaskOwnedStateStream()
> {code}
> And a new interface {{ChekcpointCoordinatorStorgae}} could be introduced
> extending {{CheckpointStorage}} and have methods below:
> {code:java}
> boolean supportsHighlyAvailableStorage()
> boolean hasDefaultSavepointLocation()
> CompletedCheckpointStorageLocation resolveCheckpoint(String externalPointer)
> CheckpointStorageLocation initializeLocationForCheckpoint(long checkpointId)
> CheckpointStorageLocation initializeLocationForSavepoint(
> long checkpointId,
> @Nullable String externalLocationPointer)
> {code}
> With this refactor, JM would only use cast {{ChekcpointCoordinatorStorgae}}
> so that we could shield TMs from accidentally calling unexpected methods.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
