Hi Gabriele,
I'm using Kafka 5.3.1, which is apache kafka 2.3 and I'm using JMX to
retrieve metrics from brokers.
The only metric I saw from "kafka.log" is LogFlushStats, but nothing
about Log.partition
This is the pattern I have deployed for kafka.log metrics, maybe I
need a different for partitions sizes?
- pattern : kafka.log<type=Log, name=(.+), topic=(.+),
partition=(.+)><>Value
name: kafka_log_$1
labels:
topic: "$2"
partition: "$3"
what do you think?
On Mon, Feb 24, 2020 at 4:05 PM Gabriele Paggi <[email protected]> wrote:
>
> Hi Richard,
>
> If you are running Kafka > 1.0.0 the information you are looking for
> is exposed by the Admin API, describeLogDirs method, or via CLI with
> kafka-log-dirs.sh.
> The metrics are also exposed by the dropwizard metrics reporter, e.g.
> for Graphite:
> kafkalog001.kafka.log.Log.partition.49.topic.__consumer_offsets.Size.value
> 1315862 1582578155
>
> Gabriele
>
> On Mon, 24 Feb 2020 at 19:54, Richard Rossel <[email protected]> wrote:
> >
> > Hi List,
> > I'm trying to find the way to keep track of topics' size (or
> > partitions) across brokers, but
> > I didn't found any Kafka metrics for that. I start wonder if that is
> > possible.
> >
> > I have ideas for a work around (querying log files) but I wonder what
> > will be the right way or how do you keep monitoring this type of
> > metric?
> >
> > Thanks.-
> >
> > --
> > Richard Rossel
> > Atlanta - GA
>
>
>
> --
> Gabriele
--
Richard Rossel
Atlanta - GA
