Hi Billy, Could you maybe share some minimal code reproducing the problem? For example I would suggest to start with reading from local files with some trivial application.
Best Piotrek pt., 22 sty 2021 o 00:21 Billy Bain <billybobb...@gmail.com> napisał(a): > I have a Streaming process where new directories are added daily in S3. > > s3://foo/bar/2021-01-18/data.gz > s3://foo/bar/2021-01-19/data.gz > s3://foo/bar/2021-01-20/data.gz > > If I start the process, it won't pick up anything other than the > directories visible when the process was started. > > The textInput has this applied: > textInputFormat.setNestedFileEnumeration(true); > > DataStreamSource<String> lines = env.readFile(textInputFormat, inputPath, > FileProcessingMode.PROCESS_CONTINUOUSLY, 1000); > -- > Wayne D. Young > aka Billy Bob Bain > billybobb...@gmail.com >
