This kind of sounds like a Outputstream flushing issue. Try calling
"System.out.flush()" now and then in your sink and report back.
On 04/04/2019 18:04, Felipe Gutierrez wrote:
Hello,
I am studying the parallelism of tasks on DataStream. So, I have
configured Flink to execute on my machine (master node) and one
virtual machine (worker node). The master has 4 cores
(taskmanager.numberOfTaskSlots: 4) and the worker only 2 cores
(taskmanager.numberOfTaskSlots: 2). I don't need to set this on the
'conf/flink-conf.yaml', this was just to ensure that I am relating the
properties with the right concepts.
When I create a application with parallelism of 1, 2, or 4, sometimes
I can see the output of the "print()" method, other times no. I checke
the output files of the task managers
("flink-flink-taskexecutor-0-master.out" or
"flink-flink-taskexecutor-0-worker.out") and I cancel the job and
start it again. All of sudden I can see the output on the .out file.
I was thinking that it was because I am creating a job with more
parallelism that the cluster supports, but this behavior also happens
when I set the parallelism of my job to less than the slots available.
I guess if I see on the Flink dashboar X Task slots available and when
I deploy my Job, the Job is running and the slots available decreased
according to the number of parallelims of my Job, everything should be
correct, doesn't it? I also created a Dummy Sink just to print the
output, but the behavior is the same.
Here is my code:
https://github.com/felipegutierrez/explore-flink/blob/master/src/main/java/org/sense/flink/examples/stream/MqttSensorRandomPartitionByKeyDAG.java#L48
Thanks,
Felipe
*--*
*-- Felipe Gutierrez*
*-- skype: felipe.o.gutierrez
*
*--****_https://felipeogutierrez.blogspot.com_*
