@小多 Thank you! I'll taka a look.------------------------------------------------------------------发件人：小多 <[email protected]>发送时间：2017年1月27日(星期五) 23:22收件人：dev <[email protected]>; JingsongLee <[email protected]>抄 送：Kenneth Knowles <[email protected]>; Aljoscha Krettek <[email protected]>主 题：Re: How to implement Timer in runner Hi Jingsong, Take a look at Blink (a fork of Flink at Alibaba Search), they had implemented a RocksDBPriorityState and used for timer. On Fri, Jan 27, 2017 at 9:33 PM, JingsongLee <[email protected]> wrote: Thanks for the reply.Maybe we need a external priority queue.Happy Chinese New Year!
------------------------------------------------------------------发件人：Aljoscha Krettek <[email protected]>发送时间：2017年1月25日(星期三) 18:38收件人：dev <[email protected]>; lzljs3620320 <[email protected]>; Kenneth Knowles <[email protected]>主 题：Re: How to implement Timer in runner Hi Jingsong,you're right, it is indeed somewhat tricky to find a good data structure for out-of-core timers. That's why we have them in memory in Flink for now and that's also why I'm afraid I don't have any good advice for you right now. We're aware of the problem in Flink but we're not yet working on a concrete solution. Cheers,Aljoscha On Tue, 24 Jan 2017 at 21:42 Dan Halperin <[email protected]> wrote: Hi Jingsong, Sorry for the delayed response; this email ended up being misclassified by my mail server and I missed it. Maybe Kenn or Aljoscha has suggestions on how runners can best implement timers? Dan On Thu, Jan 19, 2017 at 9:55 PM, lzljs3620320 <[email protected]> wrote: > Hi there, > I'm working on the beam integration for an internal system at Alibaba. Now > most of the runners put timers in memory, such as Flink, Apex, etc. (I do not > know > the implementation of Google Dataflow).But in our scene, unbounded data > has a large number of keys，which will lead to OOM(timers in memory). So > we want to store timers in state(RocksDb in disk).The problem is how to > extract fired event time timers when refresh the input > watermark. Do we have to scan all keys and timers(Now timer is composed of > Key, id, namespace, timestamp, domain)？Is there a better > implement? I'm wondering if you could give me some advice on how to implement > timers in state efficiently. Thank you! > Best,Jingsong Lee -- Engineer, Geek...
