Hi Luke!
Thanks !! We use the Flink Runner and run SDK workers as processes [1]
within a k8s pod. Can you please share broad steps on how one can do in the
runner ?
[1]
https://github.com/apache/beam/blob/master/runners/java-fn-execution/src/main/java/org/apache/beam/runners/fnexecution/environment/ProcessEnvironmentFactory.java
On Mon, Aug 8, 2022 at 8:51 AM Luke Cwik via user
wrote:
> That code only executes within a runner and is only used by certain
> runners and wouldn't work in general from user code that is monitoring the
> job or user code executing within one of the workers.
>
> You would need to author code that is likely runner specific to look up
> the number of workers associated with a job as I don't believe there is a
> general way to do this for an arbitrary Apache Beam runner.
>
> Which runner would you most likely want to use?
>
> On Sun, Aug 7, 2022 at 1:02 PM aryan m wrote:
>
>> Hi Users!
>> Is there a recommended approach to publish metrics on the number of
>> sdk workers available/running as a gauge ?
>>
>>
>> [1]
>> https://github.com/apache/beam/blob/master/runners/java-fn-execution/src/main/java/org/apache/beam/runners/fnexecution/control/DefaultJobBundleFactory.java#L267
>> [2]
>> https://github.com/apache/beam/blob/master/runners/java-fn-execution/src/main/java/org/apache/beam/runners/fnexecution/control/DefaultJobBundleFactory.java#L148
>>
>>
>> -- Aryan
>>
>
