For example, if a flink job reads from kafka do something and writes to
kafka. Do we need to take any actions when the job kafka consumer lag is
low or 0 but some tasks have constant backpressure? Do we need to increase
the parallelism or do some network tuning so that backpressure is constant
0? I
Hello everyone,
I am trying to understand the mechanism by which Flink distributed the data and
the tasks among the nodes/task managers in the cluster, assuming all TMs have
equal resources. I am using the DataSet API on my own machine.
I will try to address the issue with the following question
Hi all!
I'm trying to submit a Flink Job in Application Mode in the Kubernetes
cluster. I see some problems when an application has a big number of
operators (more than 20 same operators) - it freezes for ~6 minutes after
*2023-06-21 15:46:45,082 WARN
org.apache.flink.connector.kafka.sink.KafkaSi
Hi Dongwoo,
Thank you very much for your response. It has been very helpful to me.
Your email mentioned the configuration of keytab and krb.file, as well as how
to configure and write them into HDFS security.
However, if the pod doesn't know the location of the HDFS namenode, it needs to
load
