Apparently Flink 1.14.0 doesn't correctly translate S3 options when they are set programmatically. I'm creating a local environment like this to connect to local MinIO instance:
val flinkConf = new Configuration() flinkConf.setString("s3.endpoint", "http://127.0.0.1:9000") flinkConf.setString("s3.aws.credentials.provider", "org.apache.hadoop.fs.s3a.AnonymousAWSCredentialsProvider") val env = StreamExecutionEnvironment.createLocalEnvironmentWithWebUI(flinkConf) Then StreamingFileSink fails with a huge stack trace with most relevant messages being Caused by: org.apache.hadoop.fs.s3a.auth.NoAuthWithAWSException: No AWS Credentials provided by SimpleAWSCredentialsProvider EnvironmentVariableCredentialsProvider InstanceProfileCredentialsProvider : com.amazonaws.SdkClientException: Failed to connect to service endpoint: which means that Hadoop tried to enumerate all of the credential providers instead of using the one set in configuration. What am I doing wrong?