Any news about this Robert? Thanks in advance, Flavio
On Thu, Dec 4, 2014 at 10:03 PM, Robert Metzger <[email protected]> wrote: > Hi, > > I think there is no support for appending to HDFS files in Flink yet. > HDFS supports it, but there are some adjustments in the system required > (not deleting / creating directories before writing; exposing the append() > methods in the FS abstractions). > > I'm planning to work on the FS abstractions in the next week, if I have > enough time, I can also look into adding support for append(). > > Another approach could be adding support for recursively reading > directories with the input formats. Vasia asked for this feature a few days > ago on the mailing list. If we would have that feature, you could just > write to a directory and read the parent directory (with all the dirs for > the appends). > > Best, > Robert > > On Thu, Dec 4, 2014 at 5:59 PM, Flavio Pompermaier <[email protected]> > wrote: > >> Hi guys, >> how can I efficiently appends data (as plain strings or also avro >> records) to HDFS using Flink? >> Do I need to use Flume or can I avoid it? >> >> Thanks in advance, >> Flavio >> >> >
