Re: [DISCUSS] KIP-840: Config file option for MessageReader/MessageFormatter in ConsoleProducer/ConsoleConsumer

[Alexandre Garnier]

Hi Luke,

Thanks for the feedback.

1. Usually, in "Proposed Changes" section, we won't put PR link there.
> Maybe you can take KIP-824 as a reference.
>
I don't know what to add more here than "Add the new option", it's a really
simple straightforward modification.

2. Also, it'd be better you can provide some example reader-config and
> formatter-config file.
> And how they work within the script.
>
I did add examples.

3. If user provide both `--reader-config or --formatter-config` and
> `--property` at the same time, how will we handle this case?
> Could you add that into the KIP?
>
This was covered by the sentence "As for
--producer-property/--consumer-property with
--consumer.config/--producer.config, any value from option --property would
override value from config file."
I did add an example for this situation.


Le jeu. 9 juin 2022 à 05:21, Luke Chen  a écrit :

> Hi Alexandre,
>
> Thanks for the KIP.
>
> Some comments:
>
> 1. Usually, in "Proposed Changes" section, we won't put PR link there.
> Maybe you can take KIP-824
> 
> as a reference.
>
> 2. Also, it'd be better you can provide some example reader-config and
> formatter-config file.
> And how they work within the script.
>
> 3. If user provide both `--reader-config or --formatter-config` and
> `--property` at the same time, how will we handle this case?
> Could you add that into the KIP?
>
> Thank you.
> Luke
>
>
> On Sun, May 29, 2022 at 2:32 AM Alexandre Garnier 
> wrote:
>
>> Hi!
>>
>> Thanks for the feedback.
>> It's a good point, I updated KIP accordingly and did put the
>> dot-separated option in rejected alternatives.
>>
>> Le ven. 27 mai 2022 à 10:22, deng ziming  a
>> écrit :
>> >
>> > Thanks for the KIP, this is a good improvement. I only have one minor
>> suggestion.
>> >
>> > Currently many command line tools supports config file argument, but
>> their name style is not unified, for example, most newly added tools are
>> using --command-config, but ConsoleConsumer use —consumer.config。 I think
>> we should unify the naming style from now on, I recommend us to use
>> --reader-config and --formatter-config for the newly added arguments.
>> >
>> > --
>> > Best,
>> > Ziming
>> >
>> >
>> > > On May 26, 2022, at 4:36 PM, Alexandre Garnier 
>> wrote:
>> > >
>> > > Hello everyone,
>> > >
>> > > Any feedback on this KIP https://cwiki.apache.org/confluence/x/bBqhD?
>> > > It is a straightforward improvement without any impact on existing
>> users,
>> > > so not much to discuss besides maybe the option name.
>> > >
>> > > --
>> > > Alex
>> > >
>> > >
>> > > Le mer. 18 mai 2022 à 10:44, Alexandre Garnier  a
>> écrit :
>> > >
>> > >> Hi everyone,
>> > >>
>> > >> I created a KIP to add a config file option of reader/formatter for
>> > >> kafka-console-(consumer|producer).sh tools.
>> > >> https://cwiki.apache.org/confluence/x/bBqhD
>> > >>
>> > >> Thanks for your feedback,
>> > >> --
>> > >> Alex
>> > >>
>> >
>>
>

Re: [DISCUSS] KIP-840: Config file option for MessageReader/MessageFormatter in ConsoleProducer/ConsoleConsumer

[Luke Chen]

Hi Alexandre,

Thanks for the update.
It looks better and clear now.

I'll vote for it later.

Thank you for this improvement!
Luke

On Thu, Jun 9, 2022 at 3:33 PM Alexandre Garnier  wrote:

> Hi Luke,
>
> Thanks for the feedback.
>
> 1. Usually, in "Proposed Changes" section, we won't put PR link there.
>> Maybe you can take KIP-824 as a reference.
>>
> I don't know what to add more here than "Add the new option", it's a
> really simple straightforward modification.
>
> 2. Also, it'd be better you can provide some example reader-config and
>> formatter-config file.
>> And how they work within the script.
>>
> I did add examples.
>
> 3. If user provide both `--reader-config or --formatter-config` and
>> `--property` at the same time, how will we handle this case?
>> Could you add that into the KIP?
>>
> This was covered by the sentence "As for
> --producer-property/--consumer-property with
> --consumer.config/--producer.config, any value from option --property would
> override value from config file."
> I did add an example for this situation.
>
>
> Le jeu. 9 juin 2022 à 05:21, Luke Chen  a écrit :
>
>> Hi Alexandre,
>>
>> Thanks for the KIP.
>>
>> Some comments:
>>
>> 1. Usually, in "Proposed Changes" section, we won't put PR link there.
>> Maybe you can take KIP-824
>> 
>> as a reference.
>>
>> 2. Also, it'd be better you can provide some example reader-config and
>> formatter-config file.
>> And how they work within the script.
>>
>> 3. If user provide both `--reader-config or --formatter-config` and
>> `--property` at the same time, how will we handle this case?
>> Could you add that into the KIP?
>>
>> Thank you.
>> Luke
>>
>>
>> On Sun, May 29, 2022 at 2:32 AM Alexandre Garnier 
>> wrote:
>>
>>> Hi!
>>>
>>> Thanks for the feedback.
>>> It's a good point, I updated KIP accordingly and did put the
>>> dot-separated option in rejected alternatives.
>>>
>>> Le ven. 27 mai 2022 à 10:22, deng ziming  a
>>> écrit :
>>> >
>>> > Thanks for the KIP, this is a good improvement. I only have one minor
>>> suggestion.
>>> >
>>> > Currently many command line tools supports config file argument, but
>>> their name style is not unified, for example, most newly added tools are
>>> using --command-config, but ConsoleConsumer use —consumer.config。 I think
>>> we should unify the naming style from now on, I recommend us to use
>>> --reader-config and --formatter-config for the newly added arguments.
>>> >
>>> > --
>>> > Best,
>>> > Ziming
>>> >
>>> >
>>> > > On May 26, 2022, at 4:36 PM, Alexandre Garnier 
>>> wrote:
>>> > >
>>> > > Hello everyone,
>>> > >
>>> > > Any feedback on this KIP https://cwiki.apache.org/confluence/x/bBqhD
>>> ?
>>> > > It is a straightforward improvement without any impact on existing
>>> users,
>>> > > so not much to discuss besides maybe the option name.
>>> > >
>>> > > --
>>> > > Alex
>>> > >
>>> > >
>>> > > Le mer. 18 mai 2022 à 10:44, Alexandre Garnier  a
>>> écrit :
>>> > >
>>> > >> Hi everyone,
>>> > >>
>>> > >> I created a KIP to add a config file option of reader/formatter for
>>> > >> kafka-console-(consumer|producer).sh tools.
>>> > >> https://cwiki.apache.org/confluence/x/bBqhD
>>> > >>
>>> > >> Thanks for your feedback,
>>> > >> --
>>> > >> Alex
>>> > >>
>>> >
>>>
>>

Re: [VOTE] KIP-840: Config file option for MessageReader/MessageFormatter in ConsoleProducer/ConsoleConsumer

[Luke Chen]

Hi Alexandre,

+1 (binding) from me.

Thank you.
Luke

On Thu, Jun 9, 2022 at 10:24 AM John Roesler  wrote:

> Thanks for the KIP, Alexandre!
>
> I’m +1 (binding)
>
> -John
>
> On Wed, Jun 8, 2022, at 20:54, deng ziming wrote:
> > Thank you for this KIP,
> >
> > +1 (non-binding)
> >
> > --
> > Best,
> > Ziming
> >
> >> On Jun 7, 2022, at 8:53 PM, Alexandre Garnier  wrote:
> >>
> >> Hi!
> >>
> >> A little reminder to vote for this KIP.
> >>
> >> Thanks.
> >>
> >>
> >> Le mer. 1 juin 2022 à 10:58, Alexandre Garnier  a
> écrit :
> >>>
> >>> Hi everyone!
> >>>
> >>> I propose to start voting for KIP-840:
> >>> https://cwiki.apache.org/confluence/x/bBqhD
> >>>
> >>> Thanks,
> >>> --
> >>> Alex
>

[jira] [Created] (KAFKA-13974) Rename `INVALID_UPDATE_VERSION` to `INVALID_PARTITION_EPOCH`

[David Jacot (Jira)]

David Jacot created KAFKA-13974:
---

 Summary: Rename `INVALID_UPDATE_VERSION` to 
`INVALID_PARTITION_EPOCH`
 Key: KAFKA-13974
 URL: https://issues.apache.org/jira/browse/KAFKA-13974
 Project: Kafka
  Issue Type: Improvement
Reporter: David Jacot
Assignee: David Jacot


Rename `INVALID_UPDATE_VERSION` to `INVALID_PARTITION_EPOCH`. 
`INVALID_UPDATE_VERSION` is tight to ZK whereas `INVALID_PARTITION_EPOCH` is 
more generic.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)

[GitHub] [kafka-site] showuon merged pull request #416: MINOR: add java 8 deprecation note and java 17 support

[GitBox]

showuon merged PR #416:
URL: https://github.com/apache/kafka-site/pull/416


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@kafka.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [VOTE] KIP-831: Add metric for log recovery progress

[Luke Chen]

Bump this thread.

So far, I've got:
4 +1 (non-bingin) from Ziming, Divij, James, Raman
1 +1 (binding) from Tom Bentley

Thanks for people who vote for this KIP.
Any other votes/comments are welcomed?

Thank you.
Luke

On Wed, May 25, 2022 at 10:55 PM Tom Bentley  wrote:

> Hi Luke,
>
> Thanks for the KIP. +1 (binding).
>
> Kind regards,
>
> Tom
>
> On Fri, 20 May 2022 at 02:58, Raman Verma 
> wrote:
>
> > ^^  (non binding)
> >
> > On Thu, May 19, 2022 at 5:40 PM Raman Verma  wrote:
> > >
> > > Hello Luke,
> > >
> > > The proposal looks good to me. Thanks. +1
> > >
> > >
> > > On Tue, May 17, 2022 at 9:26 PM James Cheng 
> > wrote:
> > > >
> > > > +1 (non-binding)
> > > >
> > > > -James
> > > >
> > > > Sent from my iPhone
> > > >
> > > > > On May 16, 2022, at 12:12 AM, Luke Chen  wrote:
> > > > >
> > > > > Hi all,
> > > > >
> > > > > I'd like to start a vote on KIP to expose metrics for log recovery
> > > > > progress. These metrics would let the admins have a way to monitor
> > the log
> > > > > recovery progress.
> > > > >
> > > > > Details can be found here:
> > > > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-831%3A+Add+metric+for+log+recovery+progress
> > > > >
> > > > > Any feedback is appreciated.
> > > > >
> > > > > Thank you.
> > > > > Luke
> > >
> > >
> > >
> > > --
> > > Best Regards,
> > > Raman Verma
> >
> >
> >
> > --
> > Best Regards,
> > Raman Verma
> >
> >
>

Re: [VOTE] KIP-831: Add metric for log recovery progress

[Yu Kvicii]

Hi there. Thanks for the proposal
+1 binding

> On Jun 9, 2022, at 21:04, Luke Chen  wrote:
> 
> Bump this thread.
> 
> So far, I've got:
> 4 +1 (non-bingin) from Ziming, Divij, James, Raman
> 1 +1 (binding) from Tom Bentley
> 
> Thanks for people who vote for this KIP.
> Any other votes/comments are welcomed?
> 
> Thank you.
> Luke
> 
> On Wed, May 25, 2022 at 10:55 PM Tom Bentley  wrote:
> 
>> Hi Luke,
>> 
>> Thanks for the KIP. +1 (binding).
>> 
>> Kind regards,
>> 
>> Tom
>> 
>> On Fri, 20 May 2022 at 02:58, Raman Verma 
>> wrote:
>> 
>>> ^^  (non binding)
>>> 
>>> On Thu, May 19, 2022 at 5:40 PM Raman Verma  wrote:
 
 Hello Luke,
 
 The proposal looks good to me. Thanks. +1
 
 
 On Tue, May 17, 2022 at 9:26 PM James Cheng 
>>> wrote:
> 
> +1 (non-binding)
> 
> -James
> 
> Sent from my iPhone
> 
>> On May 16, 2022, at 12:12 AM, Luke Chen  wrote:
>> 
>> Hi all,
>> 
>> I'd like to start a vote on KIP to expose metrics for log recovery
>> progress. These metrics would let the admins have a way to monitor
>>> the log
>> recovery progress.
>> 
>> Details can be found here:
>> 
>>> 
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-831%3A+Add+metric+for+log+recovery+progress
>> 
>> Any feedback is appreciated.
>> 
>> Thank you.
>> Luke
 
 
 
 --
 Best Regards,
 Raman Verma
>>> 
>>> 
>>> 
>>> --
>>> Best Regards,
>>> Raman Verma
>>> 
>>> 
>> 

[jira] [Created] (KAFKA-13975) Mechanism to gate advertised APIs/versions based on MetadataVersion

[David Jacot (Jira)]

David Jacot created KAFKA-13975:
---

 Summary: Mechanism to gate advertised APIs/versions based on 
MetadataVersion
 Key: KAFKA-13975
 URL: https://issues.apache.org/jira/browse/KAFKA-13975
 Project: Kafka
  Issue Type: Improvement
Reporter: David Jacot
Assignee: David Jacot


At the moment, we don't have any mechanism to gate advertised APIs and/or API 
versions based on the MetadataVersion. We have cases where this would be 
helpful.

For instance, while working on KIP-841, we have had a case where we must not 
advertise the AlterPartition version 2 if the controller does not use topics 
ids.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)

[jira] [Resolved] (KAFKA-13917) Avoid calling lookupCoordinator() in tight loop

[Luke Chen (Jira)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-13917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Luke Chen resolved KAFKA-13917.
---
Fix Version/s: 3.3.0
   Resolution: Fixed

> Avoid calling lookupCoordinator() in tight loop
> ---
>
> Key: KAFKA-13917
> URL: https://issues.apache.org/jira/browse/KAFKA-13917
> Project: Kafka
>  Issue Type: Improvement
>  Components: consumer
>Affects Versions: 3.1.0, 3.1.1, 3.1.2
>Reporter: Viktor Somogyi-Vass
>Assignee: Viktor Somogyi-Vass
>Priority: Major
> Fix For: 3.3.0
>
>
> Currently the heartbeat thread's lookupCoordinator() is called in a tight 
> loop if brokers crash and the consumer is left running. Besides that it 
> floods the logs on debug level, it increases CPU usage as well.
> The fix is easy, just need to put a backoff call after coordinator lookup.
> Reproduction:
> # Start a few brokers
> # Create a topic and produce to it
> # Start consuming
> # Stop all brokers
> At this point lookupCoordinator() will be called in a tight loop.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)