flink checkpoint超时问题

2019-10-10 Thread lg
各位好: 我在使用flink的过程中遇到了下面的问题,刚开始接触flink,对排查问题的思路不是很清晰,麻烦大家提供下思路哈,谢谢。 应用场景:我这里使用Standalone Cluster方式搭建了一flink集群,其中Task Managers=10,Task Slots=54。 flink的stream过程大致为: kafka topic1的数据 --> 异步调用外部资源对数据填充 --> 存入kafka topic2。

Flink design questions - parallel processing and state management

2018-09-27 Thread lg...@yahoo.com
Hi, We are new to Flink and would like to get some ideas of achieving parallelism and for maintaining state in Flink for few interesting problems. We receive files that contain multiple types of events. We need to process them in the following way: 1. Multiple event sources send event files to