I have a Streaming process where new directories are added daily in S3. s3://foo/bar/2021-01-18/data.gz s3://foo/bar/2021-01-19/data.gz s3://foo/bar/2021-01-20/data.gz
If I start the process, it won't pick up anything other than the directories visible when the process was started. The textInput has this applied: textInputFormat.setNestedFileEnumeration(true); DataStreamSource<String> lines = env.readFile(textInputFormat, inputPath, FileProcessingMode.PROCESS_CONTINUOUSLY, 1000); -- Wayne D. Young aka Billy Bob Bain billybobb...@gmail.com
