Hi everyone, I also think the reporter won't work for my use case, it could work but then I am trying to modify the placement of operators on taskmanagers manually based on my some decisions made from metric collection, reporter makes it harder to do so. Right now, I have REST APIs, but I was thinking if in some way I could create a new MetricFetcher object and use that for collecting metrics instead of REST APIs.
Is it possible to use MetricFetcher? Thanks a lot for the help, Atul On Thu, Mar 9, 2023 at 7:39 PM Mason Chen <mas.chen6...@gmail.com> wrote: > Hi all, > > Metric reporter may be useful if you only need per component level metrics > like jobmanager and each taskmanager since the metric reporter runs in each > component. However, for an aggregated job level view of metrics, there is > no better out-of-the-box/user-facing way to get metrics than the REST API. > A good example in using the REST API is the k8s operator autoscaler which > needs to scrape metrics: > > https://github.com/apache/flink-kubernetes-operator/tree/main/flink-kubernetes-operator-autoscaler/src/main/java/org/apache/flink/kubernetes/operator/autoscaler > . > > Best, > Mason > > On Thu, Mar 9, 2023 at 4:34 AM Hang Ruan <ruanhang1...@gmail.com> wrote: > > > Hi, Atul, > > > > I think the metric reporter[1] will be helpful for you. > > > > Best, > > Hang > > > > [1] > > > > > https://nightlies.apache.org/flink/flink-docs-release-1.16/docs/deployment/metric_reporters/ > > > > Atul Lal <atul.lal...@gmail.com> 于2023年3月9日周四 17:49写道： > > > > > Hi everyone, > > > > > > I am trying to do some experiments with Flink. I am trying to modify > the > > > source code of Flink for this experiment, and I am starting a process > > > thread from JobMaster.java constructor whenever a new job is started in > > > Flink. In this thread, I want to monitor a few metrics related to the > job > > > and make some decisions based on it. > > > > > > Is there any way to collect metrics related to a job without using REST > > > endpoints? Because I think using REST endpoints here is pointless as > this > > > is internal code running on JobMaster. If there is no other way than > > using > > > REST endpoints, is there any easy way to serialize or parse JSON > > responses > > > from those endpoints as the response structures are already defined in > > the > > > classes. > > > > > > I would really appreciate it if someone could help me with this. > > > > > > Thank you, > > > Atul > > > > > >
