Hi Niels! There is the Continuous File Monitoring Source, used via
StreamExecutionEnvironment.readFile(FileInputFormat<OUT> inputFormat, String filePath, FileProcessingMode watchType, long interval); This can be used to both continuously ingest from files, or to read files once. Kostas can probably comment more about whether and how you can make the file order deterministic. Stephan On Fri, Jan 20, 2017 at 11:20 AM, Niels Basjes <ni...@basjes.nl> wrote: > Hi, > > For testing and optimizing a streaming application I want to have a "100% > accurate repeatable" substitute for a Kafka source. > I was thinking of creating a streaming source class that simply reads the > records from a (static unchanging) set of files. > Each file would then produce the data which (in the live situation) come > from a single Kafka partition. > > I hate reinventing the wheel so I'm wondering is something like this > already been built by someone? > If so, where can I find it? > > -- > Best regards / Met vriendelijke groeten, > > Niels Basjes >
