Re: Customize StreamingFileSink: Enable extending StreamingFileSink class

> On Wed, Aug 7, 2019 at 7:54 AM Fabian Hueske wrote: > >> Hi Kailash, >> >> Yes, I think creating another Jira and PR would be the right thing to do. >> >> Thank you, >> Fabian >> >> Am Fr., 2. Aug. 2019 um 10:29 Uhr schrieb Kailash Dayanan

Re: Support for ProtoBuf data format in StreamingFileSink

cc'ing few folks who are interested in this discussion. On Tue, Apr 9, 2019 at 11:35 AM Kailash Dayanand wrote: > Hello, > > I am looking to contribute a ProtoParquetWriter support which can be used > in Bulk format for the StreamingFileSink api. There has been earlier > di

Support for ProtoBuf data format in StreamingFileSink

Hello, I am looking to contribute a ProtoParquetWriter support which can be used in Bulk format for the StreamingFileSink api. There has been earlier discussions on this in the user mailing list: https://goo.gl/ya2StL and thought it would be a good addition to have. For implementation, looking at

Re: Support for ProtoBuf data format in StreamingFileSink

Friendly remainder. Any thoughts on this approach ? On Tue, Apr 9, 2019 at 11:36 AM Kailash Dayanand wrote: > cc'ing few folks who are interested in this discussion. > > On Tue, Apr 9, 2019 at 11:35 AM Kailash Dayanand > wrote: > >> Hello, >> >> I am look

Adding Metrics to StreamingFileSink

Hello, I was looking to add metrics to the streaming file sink. Currently the only details available is the generic information about for any operator like the number of records in, number of records out etc. I was looking at adding some metrics and contributing back as well as enabling the metric

[Discuss]: Adding Metrics to StreamingFileSink

Hello, I was looking to add metrics to the streaming file sink. Currently the only details available is the generic information about for any operator like the number of records in, number of records out etc. I was looking at adding some metrics and contributing back as well as enabling the metric

Re: [Discuss]: Adding Metrics to StreamingFileSink

t should be possible to extend the connector (we run into the same > issues > > with KinesisConsumer). > > > > Metrics are essential to understand the performance, especially for > things > > like S3 writes, error, retries, memory buffers and so on. > > > > Tho

Customize StreamingFileSink: Enable extending StreamingFileSink class

Hello, There were a few things which we missing in https://issues.apache.org/jira/browse/FLINK-12539, due to which it was not possible to extend the StreamingFileSink. To fix this I have created a follow-up PR but I am not sure if this is the best way to make these changes: https://github.com/apac

Proposal - Change shard discovery in Flink Kinesis Connector to use ListShards

Based on the discussion at here , I want to propose using the latest ListShards API instead of the DescribeStreams on AWS to overcome the rate limits currently imposed on De

Re: Proposal - Change shard discovery in Flink Kinesis Connector to use ListShards

Hello Bowen, Thank you very much for your feedback. I had a question with regard to your concern on compatibilities issues I notice that aws-java-sdk imported in the flink connector is shaded. So it is not likely to cause conflicts in classpath due to AWS EMR depending on a older version of the S