I’ve been running Flink in production on EMR (YARN) for some time and have
found the metrics system to be quite useful, but there is one specific case
where I’m missing a signal for this scenario:
* When a job has been submitted, but YARN does not have enough resources to
provide
Observed
Hello Kelly,
I thought that Flink scheduler only starts a job if all requested
containers/TMs are available and allotted to that job.
How can I reproduce your issue on Flink with YARN?
Thank you,
Piper
On Thu, Nov 21, 2019, 1:48 PM Kelly Smith wrote:
> I’ve been running Flink in production
in each state for better observability).
Does that make sense?
Thanks,
Kelly
From: Piper Piper
Date: Thursday, November 21, 2019 at 12:59 PM
To: Kelly Smith
Cc: "user@flink.apache.org"
Subject: Re: Metrics for Task States
Hello Kelly,
I thought that Flink scheduler only starts a
*Date: *Thursday, November 21, 2019 at 12:59 PM
> *To: *Kelly Smith
> *Cc: *"user@flink.apache.org"
> *Subject: *Re: Metrics for Task States
>
>
>
> Hello Kelly,
>
>
>
> I thought that Flink scheduler only starts a job if all requested
> containers/TM
> observability).
>>
>>
>>
>> Does that make sense?
>>
>>
>>
>> Thanks,
>>
>> Kelly
>>
>>
>>
>> *From: *Piper Piper
>> *Date: *Thursday, November 21, 2019 at 12:59 PM
>> *To: *Kelly Smith
>> *Cc: *
or is fixed.
Get Outlook for Android<https://aka.ms/ghei36>
From: Piper Piper
Sent: Friday, November 22, 2019 9:10:41 PM
To: Kelly Smith
Cc: user@flink.apache.org
Subject: Re: Metrics for Task States
I am trying to reason why this problem should occur (i.e
---
> *From:* Piper Piper
> *Sent:* Friday, November 22, 2019 9:10:41 PM
> *To:* Kelly Smith
> *Cc:* user@flink.apache.org
> *Subject:* Re: Metrics for Task States
>
> I am trying to reason why this problem should occur (i.e. why Flink could
> not reject the j
a side-car container once we move
to Kubernetes.
Kelly
From: Caizhi Weng
Date: Monday, November 25, 2019 at 1:41 AM
To: Kelly Smith
Cc: Piper Piper , "user@flink.apache.org"
Subject: Re: Metrics for Task States
Hi Kelly,
As far as I know Flink currently does not have such metrics