Hi Kanchi,
Could you provide with more information on it? Like at what stage this log
prints (job recovering, running, etc), any more detailed job or stacktrace.
Best,
Zhanghao Chen
From: Kanchi Masalia via user
Sent: Friday, February 16, 2024 4:07
To: Neha
Hi Brent,
Sounds like a good plan to start with. Application mode gives the best
isolation level, but can be costly for large number of small jobs as at least 2
containers are required (1 JobManager and 1 TaskManager) for each job. If your
jobs are mostly small in size (1 or 2 TM), you might
With streaming execution, the entire pipeline is always running, which is
necessary so that results can be continuously produced. But with batch
execution, the job graph can be segmented into separate pipelined stages
that can be executed sequentially, each running to completion before the
next
Hey everyone,
I've been looking at Flink to handle a fairly complex use case and was
hoping for some feedback on if the approach I'm thinking about with Flink
seems reasonable. When researching what people build on Flink, it seems
like a lot of the focus tends to be on running fewer
Hi Thias
I considered CheckpointedFunction.
In snapshotState() I would have to update the state of each key, extracting
the in-memory "state" of each key and putting it in the state with
state.update(...) .
This must happen per key,
But snapshotState() has no visibility of the keys. And I have no
Hi,
I have a netiquette question - is it ok to share blog posts here that
are specific to Apache Flink and not vendor-focussed?
thanks,
Robin.
Hi everyone,
I am currently exploring the fault tolerance and recovery mechanism in
batch mode within Apache Flink.
If I terminate the task manager process while the job is running, the job
restarts from the point of failure. However, at some point, the job
restarts from the very beginning.
The
