Hi Flink Developers.
We enabled entropy injection for s3, here is our setting on Yarn Cluster.
s3.entropy.key: _entropy_
s3.entropy.length: 1
state.checkpoints.dir: 's3a://{bucket name}/dev/checkpoints/_entropy_'
I have two questions:
1. After enabling entropy, job's checkpoint path changed to:
*s3://{bucket name}/dev/checkpoints/_entropy_/{job_id}chk-607*
SInce we don't know which key is mapped to _entropy_
It cannot be used to relaunch flink jobs by running
*flink run -s **s3://{bucket
name}/dev/checkpoints/_entropy_/{job_id}chk-607*
If you also enabled entropy injection for s3, any suggestion how to recover
failed jobs using entropy checkpoints?
2.We added entropy settings on the Yarn cluster.
But we can only see flink jobs in version 1.11 shows the entropy checkpoint
path.
For flink jobs version 1.9, they are still using checkpoint paths without
entropy like:
*s3://{bucket name}/dev/checkpoints/{job_id}/chk-607*
Is this path equal to s3://*{bucket name}*
*/dev/checkpoints/_entropy_/{job_id}**chk-607?*
Does entropy work for v1.9? If so, why does v1.9 job show checkpoint paths
*without* entropy?
Appreciated any suggestions.
Thanks
Best regards
Rainie
