回复: How/where to check the operator id

Hi Megh, Unfortunately, Flink currently does not expose operator IDs via REST API, nor log them unless some operators' states are missing during state recovery as operator IDs are mostly used internally. Could you further collaborate a bit on why you need the operator ID info? Maybe you can

回复: backpressure for just one subtask

Hi Kenan, You may check the stack trace and task-level flame graph [1] to investigate it. [1] https://nightlies.apache.org/flink/flink-docs-release-1.17/docs/ops/debugging/flame_graphs/ Best, Zhanghao Chen 发件人: Kenan Kılıçtepe 发送时间: 2023年9月26日 23:06 收件人: user

回复: 退订

Hi, 请发送任意内容的邮件到 user-zh-unsubscr...@flink.apache.org 来取消订阅邮件。 Best, Zhanghao Chen 发件人: guangyu05 发送时间: 2023年9月26日 18:45 收件人: user-zh@flink.apache.org 主题: 退订 退订

回复: ProcessWindowFunction Parallelism

Hi Patricia, You are using the .windowall API which generates a global window. This operation is inherently non-parallel since all elements have to pass through the same operator instance so it cannot be set to any parallelism larger than 1. Best, Zhanghao Chen

回复: HA in k8s operator

Hi Krzysztof, TM HA is taken charge by the Flink cluster itself is beyond K8s operator's responsibility. Flink will try to recover a failed Task as long as the restart limit is not reached otherwise the job will transition into terminal FAILED status. You may check the job restart strategy [1]

Re: Flink 1.15 KubernetesHaServicesFactory

Hi Alexey, This is expected as Flink 1.15 introduced a new multiple component leader election service that only runs a single leader election per Flink process. You may set `high-availability.use-old-ha-services: true` to use the old high availability services in case of any issues as well.

回复: 退订

Hi, 请发送任意内容的邮件到 user-zh-unsubscr...@flink.apache.org 地址来取消订阅来自 user-zh@flink.apache.org 邮件组的邮件 Best, Zhanghao Chen 发件人: Lynn Chen 发送时间: 2023年9月15日 16:56 收件人: user-zh@flink.apache.org 主题: 退订 退订

回复: flink-metrics如何获取applicationid

Hi, 请发送任意内容的邮件到 user-zh-unsubscr...@flink.apache.org 地址来取消订阅来自 user-zh@flink.apache.org 邮件组的邮件 Best, Zhanghao Chen 发件人: im huzi 发送时间: 2023年9月15日 18:14 收件人: user-zh@flink.apache.org 主题: Re: flink-metrics如何获取applicationid 退订 On Wed, Aug 30, 2023 at 19:14

回复: Install Flink Document as a PDF Book

Hi Yunhui, AFAIK, it is not possible yet. Best, Zhanghao Chen 发件人: Yunhui Han 发送时间: 2023年9月14日 18:18 收件人: user@flink.apache.org 主题: Install Flink Document as a PDF Book Hi, all How can I install the latest Flink Document as a PDF book? Best

回复: [Question] How to scale application based on 'reactive' mode

Hi Dennis, 1. In Flink 1.18 + non-reactive mode, autoscaler adjusts the job's parallelism and the job will request for extra TMs if the current ones cannot satisfy its need and redundant TMs will be released automatically later for being idle. In other words, parallelism changes cause TM

Re: Uneven TM Distribution of Flink on YARN

CCing @Weihua Hu<mailto:huweihua@gmail.com> , who is an expert on this. Do you have any ideas on the phenomenon here? Best, Zhanghao Chen From: Lu Niu Sent: Tuesday, August 29, 2023 12:11:35 PM To: Chen Zhanghao Cc: Kenan Kılıçtepe ; user Subje

回复: Uneven TM Distribution of Flink on YARN

Hi Lu Niu, TM distribution on YARN nodes is managed by YARN RM and is out of the scope of Flink. On the other hand, cluster.evenly-spread-out-slots forces even distribution of tasks among Flink TMs, and has nothing to do with your concerns. Also, the config currently only supports Standalone

回复: Stoping a Job Without killing Task Manager

Hi Kenan, Flink introduces the watchdog mechanism that makes TM suicide as the final safety net for properly cancel a task that got stuck in the cancellation phase for too long. The mechanism is necessary as there's no other alternatives to force killing a stuck task other than killing the

回复: How to use pipeline.jobvertex-parallelism-overrides property.

Hi Krzysztof, I agree with you that sometimes one may want to break chains so there's value in supporting overriding parallelisms for individual operators. However, the config pipeline.jobvertex-parallelism-overrides is added in the context of support Flink autoscaling. For autoscaling, we

回复: Flink operator job restart

Hi Ethan, You can refer to the K8s operator doc on how to do a stateful job upgrade: Job Management | Apache Flink Kubernetes Operator.

回复: Questions related to Autoscaler

Q1: if you use operator to submit a standalone mode job with reactive mode enabled, KEDA should still work. Q2: For Flink versions, 1.17 is recommended, but 1.15 is also okay if you backport the necessary changes listed in Autoscaler | Apache Flink Kubernetes

回复: Questions regarding adaptive scheduler with YARN and application mode

Hi Leon, Adaptive scheduler alone cannot autoscale a Flink job. It simply adjusts the parallelism of a job based on available slots [1]. To autoscale a job, we further need a policy to suggest the recommended resources for the job and a mechanism to adjust the allocated resources of the job