Hi Kirti, I think the default file `Source` does not download files locally in Flink, but reads them directly from S3. However, Flink also supports configuring temporary directories through `io.tmp.dirs`. If it is a user-defined source, it can be obtained from FlinkS3FileSystem. After the Flink job is completed, the directory will be cleaned up.
Best, Shammon FY On Fri, Sep 22, 2023 at 3:11 PM Kirti Dhar Upadhyay K via user < user@flink.apache.org> wrote: > Hi Community, > > > > I am using Flink File Source with Amazon S3. > > Please help me on below questions- > > > > 1. When Split Enumerator assigns split to Source Reader, does it > downloads the file temporarily and then starts reading/decoding the records > from file or it creates direct stream with S3? > > > > 1. If it is downloaded locally then on which path? Is it configurable? > > > > 1. Does this temporary file automatically gets deleted or any explicit > cleanup is required? > > > > > > Regards, > > Kirti Dhar >
