Hi, Maybe a bit crazy idea, but you could also try extending the S3 filesystem and add the metadata there. You could write a thin wrapper for the existing filesystem. If you'd like to go that route you might want to check this page[1]. You could use that filesystem with your custom scheme.
Best, Dawid [1] https://ci.apache.org/projects/flink/flink-docs-release-1.10/ops/filesystems/#file-systems On 19/06/2020 21:19, dhurandar S wrote: > We are creating files in S3 and we want to update the S3 object > metadata with some security-related information for governance purposes. > > Right now Apache Flink totally abstracts how and when S3 object gets > created in the system. > > Is there a way that we can pass the S3 object metadata and update it > for the object created. > > If not, > > How can we know when Apache Flink has created an S3 file. > Deterministically. > Since once its created in S3 we can write Java code after that to add > those metadata information? > > -- > Thank you and regards, > Dhurandar >
signature.asc
Description: OpenPGP digital signature
