Managing Flink+TSDB backups - best practice?

We plan to use Flink for streaming analytics, sending its results to a time-series database (TSDB). When we make an error in code or deployment, this will cause incorrect data to flow into the TSDB (quite probably with scattered timestamps). We want to be able to fix that kind of problem by restor

Re: Any plans to make Flink configurable with pure data?

Hi Arvid, thanks for the response. I don't think I was being very clear. This is a streaming application. Each customer will have a Flink job running in its own private cluster. The rough *shape *of the analysis that the Flink job is doing for each customer is always the same, because we do just o

Any plans to make Flink configurable with pure data?

To a naive Flink newcomer (me) it's a little surprising that there is no pure "data" mechanism for specifying a Flink pipeline, only "code" interfaces. With the DataStream interface I can use Java, Scala or Python to set up a pipeline and then execute it - but that doesn't really seem to *need *a p

Re: Timers not firing until stream end

gt; +44 7961 125282 See our latest features <https://t.sidekickopen70.com/s2t/c/5/f18dQhb0S7kv8cpgQZVc6VPt59hl3kW7_k2842PjkFxW2R1KhZ7v4vclW2Rxbb82bzNKzf7GYHvr01?te=W3R5hFj4cm2zwW3Q_1QY1JxXxvW3SYLMH3T0vWRW1JxwY51LDhHjW3K1M0S1GFyF-3b732&si=5987503666495488&pi=c7e08161-9dc0-4a59-c805-6b696d22e

Re: Timers not firing until stream end

ot;0","value":0.22,"is_online":true,"log":"prevMsg.ts 8000 msg_in.ts 9000 Cancelled previous timer. "} {"ts":9000,"id":"1","value":0.23,"is_online":true,"log":"prevMsg.ts 8000 msg_in.ts 9000 Cancelled previous timer. "

Re: Timers not firing until stream end

never be emitted (except for those at the job)? > Is there any way to emit periodic watermarks based on event time? > Is there any way to enable punctuated watermarks for the existing > watermark strategies without having to implement a custom one? > > On 1/27/2021 5:57 PM, Pil

Re: Timers not firing until stream end

I am calling env.setStreamTimeCharacteristic(TimeCharacteristic.EventTime); I am using Flink 11.1 (because I need to run on AWS Kinesis Data Analytics). -Pilgrim -- Learn more at https://devicepilot.com @devicepilot

Re: Timers not firing until stream end

hat we want to see. But they are emitted only once the stream has ended. {"ts":5001,"id":"1","is_online":false} {"ts":11001,"id":"1","is_online":false} {"ts":11001,"id":"0","is_onl

Timers not firing until stream end

A newbie question: I've created a basic Flink DataStream job for an IoT use-case, with file source and sink for testing. I key by device ID, then in a ProcessFunction set an EventTime Timer to fire if a device falls silent, i.e. a timeout, which I cancel if another message arrives from that device