Hi Congxian, I am making rest call to get the checkpoint config: curl -X GET \
http://localhost:8081/jobs/d2c91a44f23efa2b6a0a89b9f1ca5a3d/checkpoints/config and here is the response: { "mode": "at_least_once", "interval": 3000, "timeout": 10000, "min_pause": 1000, "max_concurrent": 1, "externalization": { "enabled": false, "delete_on_cancellation": true }, "state_backend": "FsStateBackend" } I uploaded a screenshot of how azure blob storage looks like after the cancel call : https://ibb.co/vY64pMZ Thanks a lot! Eleanore On Sat, Sep 26, 2020 at 11:23 PM Congxian Qiu <qcx978132...@gmail.com> wrote: > Hi Eleanore > > What the `CheckpointRetentionPolicy`[1] did you set for your job? if > `ExternalizedCheckpointCleanup.RETAIN_ON_CANCELLATION` is set, then the > checkpoint will be kept when canceling a job. > > PS the image did not show > > [1] > https://ci.apache.org/projects/flink/flink-docs-stable/ops/state/checkpoints.html#retained-checkpoints > Best, > Congxian > > > Eleanore Jin <eleanore....@gmail.com> 于2020年9月27日周日 下午1:50写道： > >> Hi experts, >> >> I am running flink 1.10.2 on kubernetes as per job cluster. Checkpoint is >> enabled, with interval 3s, minimumPause 1s, timeout 10s. I'm >> using FsStateBackend, snapshots are persisted to azure blob storage >> (Microsoft cloud storage service). >> >> Checkpointed state is just source kafka topic offsets, the flink job is >> stateless as it does filter/json transformation. >> >> The way I am trying to stop the flink job is via monitoring rest api >> mentioned in doc >> <https://ci.apache.org/projects/flink/flink-docs-stable/monitoring/rest_api.html#jobs-jobid-1> >> >> e.g. >> curl -X PATCH \ >> ' >> http://localhost:8081/jobs/3c00535c182a3a00258e2f57bc11fb1a?mode=cancel' >> \ >> -H 'Content-Type: application/json' \ >> -d '{}' >> >> This call returned successfully with statusCode 202, then I stopped the >> task manager pods and job manager pod. >> >> According to the doc, the checkpoint should be cleaned up after the job >> is stopped/cancelled. >> What I have observed is, the checkpoint dir is not cleaned up, can you >> please shield some lights on what I did wrong? >> >> Below shows the checkpoint dir for a cancelled flink job. >> [image: image.png] >> >> Thanks! >> Eleanore >> >>
