Re: Benchmark for Stateful Functions

Hi Omid, There currently aren't any benchmarks that I know of for Stateful Functions. However, Stateful Functions applications run on top of Apache Flink and therefore share the same network stack / runtime. So, if throughput and latency is your only concern, you should be able carry over any res

Re: Publishing Sink Task watermarks outside flink

Following up on this, I tried tweaking the Jdbc Sink as Timo suggested and was successful in it. Basically I added a member *long maxWatermarkSeen *in JDBCOutputFormat, so whenever a new record is added to the batch it updates the *maxWatermarkSeen* for this subtask with *org.apache.flink.streamin

Re: multiple joins in one job

Thanks for your replay. But as I known, if the time attribute will be retained and the time attribute field of both streams is selected in the result after joining, who is the final time attribute variable? Benchao Li 于2020年4月30日周四 下午8:25写道： > Hi lec, > > AFAIK, time attribute will be prese

Re: History Server Not Showing Any Jobs - File Not Found?

yes, exactly; I want to rule out that (somehow) HDFS is the problem. I couldn't reproduce the issue locally myself so far. On 01/05/2020 22:31, Hailu, Andreas wrote: Hi Chesnay, yes – they were created using Flink 1.9.1 as we’ve only just started to archive them in the past couple weeks. Coul

Re: Flink: For terabytes of keyed state.

Hi Congxian, Thank you so much for your response, that really helps! >From your experience, how long does it take for Flink to redistribute terabytes of state data on node addition / node failure. Thanks! On Sun, May 3, 2020 at 6:56 PM Congxian Qiu wrote: > Hi > > 1. From my experience, Flink

Re: Savepoint memory overhead

Hi >From the given fig, seems that the end-to-end duration of the two failed checkpoint is small(it is not timeout due to some reason), could you please check why did they fail? Maybe you can find something in jm log such as "Decline checkpoint {} by task {} of job {} at {}." then you can go to th

Re: Flink: For terabytes of keyed state.

Hi 1. From my experience, Flink can support such big state, you can set appropriate parallelism for the stateful operator. for RocksDB you may need to care about the disk performance. 2. Inside Flink, the state is separated by key-group, each task/parallelism contains multiple key-groups. Flink d

[ANNOUNCE] Weekly Community Update 2020/18

Dear community, happy to share - a brief - community update this week with an update on Flink 1.10.1, our application to Google Season of Docs 2020, a discussion to support Hadoop 3, a recap of Flink Forward Virtual 2020 and a bit more. Flink Development == * [releases] Yu has publis