Re: [VOTE] KIP-92 - Add per partition lag metrics to KafkaConsumer

[Becket Qin]

Just updated the wiki to clarify that.

Thanks everyone for the votes. KIP-92 has passed with +3 (binding) and +2
(non-binding).

On Wed, Dec 21, 2016 at 1:53 PM, Ismael Juma  wrote:

> That seems reasonable. It would be good to update the KIP to make this
> clear.
>
> Ismael
>
> On Wed, Dec 21, 2016 at 1:49 PM, Becket Qin  wrote:
>
> > Hi Ismael,
> >
> > Thanks for the comments. Good observation. I guess for max lag of all the
> > partitions the average value is less meaningful because the lag can be
> from
> > different partitions, so an average of lags from different partitions at
> > different times seems hard to reason about. On the other hand, for per
> > partition lag, the average value of the same partition at different times
> > seems having some real meaning?
> >
> > Thanks,
> >
> > Jiangjie (Becket) Qin
> >
> > On Wed, Dec 21, 2016 at 1:02 PM, Ismael Juma  wrote:
> >
> > > Thanks for the KIP, it's a useful improvement. Just one question, the
> KIP
> > > states that it's about adding per partition metrics for metrics that
> > > already exist. However, when I look at the code, it seems that
> > > `records-lag-avg` doesn't exist?
> > >
> > > this.recordsFetchLag = metrics.sensor("records-lag");
> > > this.recordsFetchLag.add(metrics.metricName("records-lag-max",
> > > this.metricGrpName, "The maximum lag in terms of number of records for
> > any
> > > partition in this window"), new Max());
> > >
> > > Ismael
> > >
> > > On Thu, Dec 15, 2016 at 5:32 PM, Becket Qin 
> > wrote:
> > >
> > > > Hi,
> > > >
> > > > I want to start a voting thread on KIP-92 which proposes to add per
> > > > partition lag metrics to KafkaConsumer. The KIP wiki page is below:
> > > >
> > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > > 92+-+Add+per+partition+lag+metrics+to+KafkaConsumer
> > > >
> > > > Thanks,
> > > >
> > > > Jiangjie (Becket) Qin
> > > >
> > >
> >
>

Re: [VOTE] KIP-92 - Add per partition lag metrics to KafkaConsumer

[Ismael Juma]

That seems reasonable. It would be good to update the KIP to make this
clear.

Ismael

On Wed, Dec 21, 2016 at 1:49 PM, Becket Qin  wrote:

> Hi Ismael,
>
> Thanks for the comments. Good observation. I guess for max lag of all the
> partitions the average value is less meaningful because the lag can be from
> different partitions, so an average of lags from different partitions at
> different times seems hard to reason about. On the other hand, for per
> partition lag, the average value of the same partition at different times
> seems having some real meaning?
>
> Thanks,
>
> Jiangjie (Becket) Qin
>
> On Wed, Dec 21, 2016 at 1:02 PM, Ismael Juma  wrote:
>
> > Thanks for the KIP, it's a useful improvement. Just one question, the KIP
> > states that it's about adding per partition metrics for metrics that
> > already exist. However, when I look at the code, it seems that
> > `records-lag-avg` doesn't exist?
> >
> > this.recordsFetchLag = metrics.sensor("records-lag");
> > this.recordsFetchLag.add(metrics.metricName("records-lag-max",
> > this.metricGrpName, "The maximum lag in terms of number of records for
> any
> > partition in this window"), new Max());
> >
> > Ismael
> >
> > On Thu, Dec 15, 2016 at 5:32 PM, Becket Qin 
> wrote:
> >
> > > Hi,
> > >
> > > I want to start a voting thread on KIP-92 which proposes to add per
> > > partition lag metrics to KafkaConsumer. The KIP wiki page is below:
> > >
> > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > 92+-+Add+per+partition+lag+metrics+to+KafkaConsumer
> > >
> > > Thanks,
> > >
> > > Jiangjie (Becket) Qin
> > >
> >
>

Re: [VOTE] KIP-92 - Add per partition lag metrics to KafkaConsumer

[Becket Qin]

Hi Ismael,

Thanks for the comments. Good observation. I guess for max lag of all the
partitions the average value is less meaningful because the lag can be from
different partitions, so an average of lags from different partitions at
different times seems hard to reason about. On the other hand, for per
partition lag, the average value of the same partition at different times
seems having some real meaning?

Thanks,

Jiangjie (Becket) Qin

On Wed, Dec 21, 2016 at 1:02 PM, Ismael Juma  wrote:

> Thanks for the KIP, it's a useful improvement. Just one question, the KIP
> states that it's about adding per partition metrics for metrics that
> already exist. However, when I look at the code, it seems that
> `records-lag-avg` doesn't exist?
>
> this.recordsFetchLag = metrics.sensor("records-lag");
> this.recordsFetchLag.add(metrics.metricName("records-lag-max",
> this.metricGrpName, "The maximum lag in terms of number of records for any
> partition in this window"), new Max());
>
> Ismael
>
> On Thu, Dec 15, 2016 at 5:32 PM, Becket Qin  wrote:
>
> > Hi,
> >
> > I want to start a voting thread on KIP-92 which proposes to add per
> > partition lag metrics to KafkaConsumer. The KIP wiki page is below:
> >
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > 92+-+Add+per+partition+lag+metrics+to+KafkaConsumer
> >
> > Thanks,
> >
> > Jiangjie (Becket) Qin
> >
>

Re: [VOTE] KIP-92 - Add per partition lag metrics to KafkaConsumer

[Ismael Juma]

Thanks for the KIP, it's a useful improvement. Just one question, the KIP
states that it's about adding per partition metrics for metrics that
already exist. However, when I look at the code, it seems that
`records-lag-avg` doesn't exist?

this.recordsFetchLag = metrics.sensor("records-lag");
this.recordsFetchLag.add(metrics.metricName("records-lag-max",
this.metricGrpName, "The maximum lag in terms of number of records for any
partition in this window"), new Max());

Ismael

On Thu, Dec 15, 2016 at 5:32 PM, Becket Qin  wrote:

> Hi,
>
> I want to start a voting thread on KIP-92 which proposes to add per
> partition lag metrics to KafkaConsumer. The KIP wiki page is below:
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> 92+-+Add+per+partition+lag+metrics+to+KafkaConsumer
>
> Thanks,
>
> Jiangjie (Becket) Qin
>

Re: [VOTE] KIP-92 - Add per partition lag metrics to KafkaConsumer

[Guozhang Wang]

+1, thanks.

On Wed, Dec 21, 2016 at 10:26 AM, radai  wrote:

> +1
>
> On Wed, Dec 21, 2016 at 9:51 AM, Dong Lin  wrote:
>
> > +1 (non-binding)
> >
> > On Thu, Dec 15, 2016 at 5:32 PM, Becket Qin 
> wrote:
> >
> > > Hi,
> > >
> > > I want to start a voting thread on KIP-92 which proposes to add per
> > > partition lag metrics to KafkaConsumer. The KIP wiki page is below:
> > >
> > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > 92+-+Add+per+partition+lag+metrics+to+KafkaConsumer
> > >
> > > Thanks,
> > >
> > > Jiangjie (Becket) Qin
> > >
> >
>



-- 
-- Guozhang

Re: [VOTE] KIP-92 - Add per partition lag metrics to KafkaConsumer

[Joel Koshy]

+1

On Wed, Dec 21, 2016 at 10:26 AM, radai  wrote:

> +1
>
> On Wed, Dec 21, 2016 at 9:51 AM, Dong Lin  wrote:
>
> > +1 (non-binding)
> >
> > On Thu, Dec 15, 2016 at 5:32 PM, Becket Qin 
> wrote:
> >
> > > Hi,
> > >
> > > I want to start a voting thread on KIP-92 which proposes to add per
> > > partition lag metrics to KafkaConsumer. The KIP wiki page is below:
> > >
> > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > 92+-+Add+per+partition+lag+metrics+to+KafkaConsumer
> > >
> > > Thanks,
> > >
> > > Jiangjie (Becket) Qin
> > >
> >
>

Re: [VOTE] KIP-92 - Add per partition lag metrics to KafkaConsumer

[radai]

+1

On Wed, Dec 21, 2016 at 9:51 AM, Dong Lin  wrote:

> +1 (non-binding)
>
> On Thu, Dec 15, 2016 at 5:32 PM, Becket Qin  wrote:
>
> > Hi,
> >
> > I want to start a voting thread on KIP-92 which proposes to add per
> > partition lag metrics to KafkaConsumer. The KIP wiki page is below:
> >
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > 92+-+Add+per+partition+lag+metrics+to+KafkaConsumer
> >
> > Thanks,
> >
> > Jiangjie (Becket) Qin
> >
>

Re: [VOTE] KIP-92 - Add per partition lag metrics to KafkaConsumer

[Dong Lin]

+1 (non-binding)

On Thu, Dec 15, 2016 at 5:32 PM, Becket Qin  wrote:

> Hi,
>
> I want to start a voting thread on KIP-92 which proposes to add per
> partition lag metrics to KafkaConsumer. The KIP wiki page is below:
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> 92+-+Add+per+partition+lag+metrics+to+KafkaConsumer
>
> Thanks,
>
> Jiangjie (Becket) Qin
>

[VOTE] KIP-92 - Add per partition lag metrics to KafkaConsumer

[Becket Qin]

Hi,

I want to start a voting thread on KIP-92 which proposes to add per
partition lag metrics to KafkaConsumer. The KIP wiki page is below:

https://cwiki.apache.org/confluence/display/KAFKA/KIP-92+-+Add+per+partition+lag+metrics+to+KafkaConsumer

Thanks,

Jiangjie (Becket) Qin