Re: Flink on Azure HDInsight

2016-04-30 Thread Stephan Ewen
Hi! As far as I know, some people have been using Flink together with Azure, and we try and do some release validation on Azure as well. There is even a section in the docs that describes how to use Hadoop's Azure Table formats with Flink https://ci.apache.org/projects/flink/flink-docs-master/api

Count of Grouped DataSet

2016-04-30 Thread nsengupta
Hello Flinksters, What is the most idiomatic way in Flink to get the count of records grouped by a Key (the Key can have multiple fields)? I have referred to this ticket but because it is still open, I can't make out what has been the final d

Re: AvroWriter for Rolling sink

2016-04-30 Thread Igor Berman
[FLINK-3854] Support Avro key-value rolling sink writer #1953 On 27 April 2016 at 19:29, Igor Berman wrote: > Hi Aljoscha, > > avro-mapred jar contains different M/R output formats for avro, and their > writers > it's primary used in M/R jobs that produce avro output > see some details here : ht

Re: Regarding Broadcast of datasets in streaming context

2016-04-30 Thread Biplob Biswas
Hi Gyula, I read your workaround and started reading about flink iterations, coflatmap operators and other things. Now, I do understand a few things but the solution you provided is not completely clear to me. I understand the following things from your post. 1. You initially have a datastream of