Friendly remainder. Any thoughts on this approach ? On Tue, Apr 9, 2019 at 11:36 AM Kailash Dayanand <kdayan...@lyft.com> wrote:
> cc'ing few folks who are interested in this discussion. > > On Tue, Apr 9, 2019 at 11:35 AM Kailash Dayanand <kdayan...@lyft.com> > wrote: > >> Hello, >> >> I am looking to contribute a ProtoParquetWriter support which can be used >> in Bulk format for the StreamingFileSink api. There has been earlier >> discussions on this in the user mailing list: https://goo.gl/ya2StL and >> thought it would be a good addition to have. >> >> For implementation, looking at the current APIs present at >> ProtoParquetWriter with the parguet project (http://tinyurl.com/y378be42), >> it looks like there is some different in the interface between Avro and >> Proto writes (ProtoParquetWriter does not have a builder class as well as >> not interface with Outputfile). Due to this, I was looking at directly >> extending the ParquetWriter within Flink to define the Builder static class >> and have newer interfaces. This is needed as the bulk writer takes a >> builder to crate the ParquetWriter in the bulkWriter.Factory. ( >> http://tinyurl.com/yyg9cn9b) >> >> Any thoughts if this is a reasonable approach? >> >> Thanks >> Kailash >> >
