Hi,
I have one flink job which has two tasks
Task1- Source some static data over https and keep it in memory, this keeps
refreshing it every 1 hour
Task2- Process some real-time events from Kafka and uses static data to
validate something and transform, then forward to other Kafka topic.
so far, e
thanks for helping with some inputs, yes I am using rich function and
handling objects created in open, and also and network calls are getting
called in a run.
but currently, I got stuck running this same task on *all task managers*
(nodes), when I submit the job, this task1(static data task) runs
I don't really understand how task2 reads static data from task1,
but I think you can integrate the logic of getting static data from http in
task1 into task2 and keep only one kind of task.
Best,
Weihua
On Wed, Jun 15, 2022 at 10:07 AM Great Info wrote:
> thanks for helping with some inputs,
-> If so, I think you can set Task1 and Task2 to the same parallelism and
set them in the same slot sharing group. In this way, Task1 and Task2 will
be deployed into the same slot(That is, the same task manager).
*Updating task details *
*Task1- Source some static data over HTTPS and keep it in me
Kindly help with this, I got stuck
-> If so, I think you can set Task1 and Task2 to the same parallelism and
set them in the same slot sharing group. In this way, Task1 and Task2 will
be deployed into the same slot(That is, the same task manager).
*Updating task details *
*Task1- Source some stati
Hi Great,
-> Will these methods work?
I think it will not work. It can control that the slots are evenly
distributed on the TM, but cannot control the correspondence between tasks
and slots.
For example, the Flink cluster has 2 TMs, each TM has 4 slots(so total 8
slots in this cluster), and your j
