Re: Exception: Coordinator of operator xxxx does not exist or the job vertex this operator belongs to is not initialized.
Hi Biao, I agree with you that this exception is not very meaningful and can be noisy in the JM logs, especially when running large-scale batch jobs in a session cluster. IIRC, there isn't a current config to filter out or silence such exceptions in batch mode. So I've created a JIRA ticket ( https://issues.apache.org/jira/browse/FLINK-35622) to track this issue for possible future optimizations. Best, Junrui Geng Biao 于2024年6月16日周日 13:41写道： > Hi Junrui, > Thanks for your answer! Since this exception is not very meaningful, is > there a solution or a flink config to filter out or silent such exception > in batch mode? When I run some large scale batch jobs in a session cluster, > it turns out that the JM log will be fulfilled with this exception which > makes it difficult to find detailed execution information about the job. > > Best, > Biao Geng > > 发送自 Outlook for iOS <https://aka.ms/o0ukef> > -- > *发件人:* Junrui Lee > *发送时间:* Sunday, June 16, 2024 12:49:10 PM > *收件人:* Corin > *抄送:* user@flink.apache.org > *主题:* Re: Exception: Coordinator of operator xxxx does not exist or the > job vertex this operator belongs to is not initialized. > > Hi, > > This exception is common in batch jobs and is caused by the collect sink > attempting to fetch data from the corresponding operator coordinator on the > JM based on the operator ID. However, due to the sequential scheduling of > batch jobs, if a job vertex has not been initialized yet, the corresponding > operator coordinator cannot be found, leading to the printing of this > message. This log does not impact the normal execution of the job because > the collect sink will keep retrying to send the request. > > Best, > Junrui > > Corin 于2024年6月16日周日 12:45写道： > > When I run a batch job using Flink 1.19, I used collect() in the job, and > many times the following error appears in the JobManager log: Caused by: > org.apache.flink.util.FlinkException: Coordinator of operator does not > exist or the job vertex this operator belongs to is not initialized. What > is the cause of this exception? > >
Re: Exception: Coordinator of operator xxxx does not exist or the job vertex this operator belongs to is not initialized.
Hi Junrui, Thanks for your answer! Since this exception is not very meaningful, is there a solution or a flink config to filter out or silent such exception in batch mode? When I run some large scale batch jobs in a session cluster, it turns out that the JM log will be fulfilled with this exception which makes it difficult to find detailed execution information about the job. Best, Biao Geng 发送自 Outlook for iOS<https://aka.ms/o0ukef> 发件人: Junrui Lee 发送时间: Sunday, June 16, 2024 12:49:10 PM 收件人: Corin 抄送: user@flink.apache.org 主题: Re: Exception: Coordinator of operator xxxx does not exist or the job vertex this operator belongs to is not initialized. Hi, This exception is common in batch jobs and is caused by the collect sink attempting to fetch data from the corresponding operator coordinator on the JM based on the operator ID. However, due to the sequential scheduling of batch jobs, if a job vertex has not been initialized yet, the corresponding operator coordinator cannot be found, leading to the printing of this message. This log does not impact the normal execution of the job because the collect sink will keep retrying to send the request. Best, Junrui Corin mailto:jens...@126.com>> 于2024年6月16日周日 12:45写道： When I run a batch job using Flink 1.19, I used collect() in the job, and many times the following error appears in the JobManager log: Caused by: org.apache.flink.util.FlinkException: Coordinator of operator does not exist or the job vertex this operator belongs to is not initialized. What is the cause of this exception?
Re: Exception: Coordinator of operator xxxx does not exist or the job vertex this operator belongs to is not initialized.
Hi, This exception is common in batch jobs and is caused by the collect sink attempting to fetch data from the corresponding operator coordinator on the JM based on the operator ID. However, due to the sequential scheduling of batch jobs, if a job vertex has not been initialized yet, the corresponding operator coordinator cannot be found, leading to the printing of this message. This log does not impact the normal execution of the job because the collect sink will keep retrying to send the request. Best, Junrui Corin 于2024年6月16日周日 12:45写道： > When I run a batch job using Flink 1.19, I used collect() in the job, and > many times the following error appears in the JobManager log: Caused by: > org.apache.flink.util.FlinkException: Coordinator of operator does not > exist or the job vertex this operator belongs to is not initialized. What > is the cause of this exception? >
Exception: Coordinator of operator xxxx does not exist or the job vertex this operator belongs to is not initialized.
When I run a batch job using Flink 1.19, I used collect() in the job, and many times the following error appears in the JobManager log: Caused by: org.apache.flink.util.FlinkException: Coordinator of operator does not exist or the job vertex this operator belongs to is not initialized. What is the cause of this exception?
