[
https://issues.apache.org/jira/browse/FLINK-4814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16166576#comment-16166576
]
Elias Levy commented on FLINK-4814:
-----------------------------------
Yes please. That would be helpful. I would suggest multiple levels of
overriding. E.g. the cluster config can provide defaults, as it does now for
some things. You should be able to package a config file in your job's
resources that override the cluster's defaults. You should also be able to
pass the job a config file at run time to override values in both of those.
And command line arguments and programmatically set values should override all
everything else.
It would be good for the job to print its configuration on start up to aid
debugging of settings, like the Kafka producer/consumer do.
Also, as I mentioned in the mailing list, the mini cluster used in the local
execution environment doesn't load a config file, so there are things, like
external checkpoints, that can't be tested at the moment from within an IDE.
API-wise it could also be more consistent. At the moment you configure some
settings via the stream environment (e.g.
{{env.setStreamTimeCharacteristic()}}), others via {{env.getConfig.foo}}, more
via {{env.getCheckpointConfig.bar}}, and some only via the config file.
> Remove extra storage location for externalized checkpoint metadata
> ------------------------------------------------------------------
>
> Key: FLINK-4814
> URL: https://issues.apache.org/jira/browse/FLINK-4814
> Project: Flink
> Issue Type: Sub-task
> Components: State Backends, Checkpointing
> Reporter: Ufuk Celebi
>
> Follow up for FLINK-4512.
> Store checkpoint meta data in checkpoint directory. That makes it simpler
> for users to track and clean up checkpoints manually, if they want to retain
> externalized checkpoints across cancellations and terminal failures.
> Every state backend needs to be able to provide a storage location for the
> checkpoint metadata. The memory state backend would hence not work with
> externalized checkpoints, unless one sets explicitly a parameter
> `setExternalizedCheckpointsLocation(uri)`.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
