Kenny Ma created FLINK-24878:
--------------------------------
Summary: Flink entropy injection not working
Key: FLINK-24878
URL: https://issues.apache.org/jira/browse/FLINK-24878
Project: Flink
Issue Type: Bug
Components: Connectors / FileSystem
Affects Versions: 1.12.0, 1.11.4
Reporter: Kenny Ma
I've followed the instructions described by this doc
[https://nightlies.apache.org/flink/flink-docs-release-1.11/ops/filesystems/s3.html]
to enable entropy injection for Flink checkpointing. Unfortunately the feature
is not working and the entropy key is not being replaced in the AWS S3
checkpoint location.
To reproduce the problem with the Flink StateMachineExample example:
{code:java}
$FLINK_HOME/bin/flink run -c
org.apache.flink.streaming.examples.statemachine.StateMachineExample
examples/streaming/StateMachineExample.jar --sleep 100 --backend file
--checkpoint-dir s3://s3-bucket-xyz/_entropy_/checkpoint/
--incremental-checkpoints true{code}
The behavior is different depending on the S3 filesystem library used.
If using the flink-s3-fs-hadoop library, the S3 path for the checkpoint is
s3://s3-bucket-xyz/_entropy_/checkpoint/...
If using the flink-s3-fs-presto library, the S3 path for the checkpoint is
s3://s3-bucket-xyz/checkpoint/...
For both S3 filesystem libraries the entropy injection failed to happen.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
