recommendation for this limitation like size of files or number of
files or checkpointing state backend etc?
Regards,
Kirti Dhar
From: Martijn Visser
Sent: 20 April 2023 18:14
To: Kirti Dhar Upadhyay K
Cc: user@flink.apache.org
Subject: Re: SplitEnumerator and SourceReader
Hi Kirti Dhar,
1
Hi Kirti Dhar,
1. The SourceReader downloads the file, which is assigned to him by the
SplitEnumerator
2. This depends on the format; a BulkFormat like Parquet or ORC can be read
in batches of records at a time.
3. The SplitEnumerator runs on the JobManager, not on a TaskManager. Have
you read som
