A proper solution will required a custom Flink build, were you want to
modify
org.apache.flink.runtime.metrics.util.MetricUtils#instantiateProcessMetricGroup
and
org.apache.flink.runtime.metrics.util.MetricUtils#instantiateTaskManagerMetricGroup
to add your custom metrics.
This is where the jm/tm really start initializing metrics, and as such
is your best bet.
In there you will want to define some (thread-safe!) counters that you
expose, which are modified by a custom appender that you define
programmatically and initialize at runtime.
On 28/02/2020 17:33, Theo Diefenthal wrote:
Hi,
From my backend service, I appreciate to collect metrics about the log
messages, i.e. how many error and warn messages were printed over
time, see e.g. for Micrometer:
https://github.com/micrometer-metrics/micrometer/blob/master/micrometer-core/src/main/java/io/micrometer/core/instrument/binder/logging/LogbackMetrics.java
I'd like to collect those metrics for Flink as well, for all task
managers and the job manager(s).
For the Task Manager, I probably could just define a static variable
which initialisizes the logging metrics for logback/log4j, but how to
integrate best with flink and register the counters into flink metric
groups? And how to do that for the job manager as well?
What do you think is the best way to integrate the logging metrics to
flink?
Best regards
Theo
--
SCOOP Software GmbH - Gut Maarhausen - Eiler Straße 3 P - D-51107 Köln
Theo Diefenthal
T +49 221 801916-196 - F +49 221 801916-17 - M +49 160 90506575
theo.diefent...@scoop-software.de - www.scoop-software.de
Sitz der Gesellschaft: Köln, Handelsregister: Köln,
Handelsregisternummer: HRB 36625
Geschäftsführung: Dr. Oleg Balovnev, Frank Heinen,
Martin Müller-Rohde, Dr. Wolfgang Reddig, Roland Scheel
