[jira] [Resolved] (KAFKA-12980) Allow consumers to return from poll when position advances due to aborted transactions

2021-12-08 Thread Jason Gustafson (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-12980. - Fix Version/s: 3.2.0 Resolution: Fixed > Allow consumers to return from poll

Re: [VOTE] KIP-805: Add range and scan query support in IQ v2

2021-12-08 Thread Matthias J. Sax
Thanks for the KIP. +1 (binding) On 12/5/21 7:03 PM, Luke Chen wrote: Hi Vasiliki, Thanks for the KIP! It makes sense to have the range and scan query in IQv2, as in IQv1. +1 (non-binding) Thank you. Luke On Thu, Dec 2, 2021 at 5:41 AM John Roesler wrote: Thanks for the KIP, Vicky! I’m

Re: [DISCUSS] KIP-805: Add range and scan query support in IQ v2

2021-12-08 Thread Matthias J. Sax
Thanks for the details! I also chatted with John about it, and he filed https://issues.apache.org/jira/browse/KAFKA-13526 to incorporate some feedback as follow up work. IMHO, the hard coded query translation is not ideal and should be plugable. But for a v1 of IQv2 (pun intended) the

Re: [DISCUSS] KIP-782: Expandable batch size in producer

2021-12-08 Thread Artem Livshits
Hi Jun, 11. That was my initial thinking as well, but in a discussion some people pointed out the change of behavior in some scenarios. E.g. if someone for some reason really wants batches to be at least 16KB and sets large linger.ms, and most of the time the batches are filled quickly enough

[jira] [Created] (KAFKA-13526) IQv2: Consider more generic logic for mapping between binary and typed queries

2021-12-08 Thread John Roesler (Jira)
John Roesler created KAFKA-13526: Summary: IQv2: Consider more generic logic for mapping between binary and typed queries Key: KAFKA-13526 URL: https://issues.apache.org/jira/browse/KAFKA-13526

[jira] [Created] (KAFKA-13525) IQv2: Implement KeyQuery from the KIP

2021-12-08 Thread John Roesler (Jira)
John Roesler created KAFKA-13525: Summary: IQv2: Implement KeyQuery from the KIP Key: KAFKA-13525 URL: https://issues.apache.org/jira/browse/KAFKA-13525 Project: Kafka Issue Type: Sub-task

[jira] [Created] (KAFKA-13524) IQv2: Add option to query from caches

2021-12-08 Thread John Roesler (Jira)
John Roesler created KAFKA-13524: Summary: IQv2: Add option to query from caches Key: KAFKA-13524 URL: https://issues.apache.org/jira/browse/KAFKA-13524 Project: Kafka Issue Type: Sub-task

Re: [DISCUSS] KIP-782: Expandable batch size in producer

2021-12-08 Thread Jun Rao
Hi, Artem, Thanks for the reply. 11. Got it. To me, batch.size is really used for throughput and not for latency guarantees. There is no guarantee when 16KB will be accumulated. So, if users want any latency guarantee, they will need to specify linger.ms accordingly. Then, batch.size can just be

[jira] [Created] (KAFKA-13523) Implement IQv2 support in global stores

2021-12-08 Thread John Roesler (Jira)
John Roesler created KAFKA-13523: Summary: Implement IQv2 support in global stores Key: KAFKA-13523 URL: https://issues.apache.org/jira/browse/KAFKA-13523 Project: Kafka Issue Type: Sub-task

[jira] [Resolved] (KAFKA-13506) IQv2: Transmit position to standbys

2021-12-08 Thread John Roesler (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Roesler resolved KAFKA-13506. -- Resolution: Fixed > IQv2: Transmit position to standbys > ---

[jira] [Resolved] (KAFKA-13491) Implement IQv2 Framework

2021-12-08 Thread John Roesler (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Roesler resolved KAFKA-13491. -- Resolution: Fixed > Implement IQv2 Framework > > >

[jira] [Created] (KAFKA-13522) IQv2: Implement position tracking and bounding in API

2021-12-08 Thread John Roesler (Jira)
John Roesler created KAFKA-13522: Summary: IQv2: Implement position tracking and bounding in API Key: KAFKA-13522 URL: https://issues.apache.org/jira/browse/KAFKA-13522 Project: Kafka Issue

Re: [DISCUSS] KIP-801: Implement an Authorizer that stores metadata in __cluster_metadata

2021-12-08 Thread José Armando García Sancio
Hi Colin, Thanks for the KIP. 1. Can you talk about how the set of ACLs needed to authorize controllers and brokers will get bootstrapped? I am particularly interested in how we are going to configure a new cluster so that the controllers nodes can authorize each other's requests to establish

Re: [DISCUSS] KIP-796: Interactive Query v2

2021-12-08 Thread Guozhang Wang
Thanks for the clarification, it looks good to me now. On Wed, Nov 17, 2021 at 9:21 PM John Roesler wrote: > Ah, sorry, Guozhang, > > It seem I was a bit too eager with starting the vote thread. > > 13: I think that makes perfect sense. I've updated the KIP. > > 14: Oof, I can't believe I

Re: [VOTE] KIP-792: Add "generation" field into consumer protocol

2021-12-08 Thread Guozhang Wang
Hi Luke, Thanks for the KIP. One thing I'd like to double check is that, since the ConsumerProtocolSubscription is not auto generated from the json file, we need to make sure the old-versioned leader would be able to ignore the new field during an upgrade e.g. without crashing. Other than that,

Re: [DISCUSS] KIP-782: Expandable batch size in producer

2021-12-08 Thread Artem Livshits
Hi Jun, 10. My understanding is that MemoryRecords would under the covers be allocated in chunks, so logically it still would be one MemoryRecords object, it's just instead of allocating one large chunk upfront, smaller chunks are allocated as needed to grow the batch and linked into a list. 11.

Re: [DISCUSS] KIP-805: Add range and scan query support in IQ v2

2021-12-08 Thread Vasiliki Papavasileiou
Hey Matthias, Thank you for looking into the KIP! We are adding raw versions of typed queries, like `RawRangeQuery` because it simplifies internal query handling since the bytes stores only support raw queries. A typed RangeQuery is handled by the `MeteredStore` which creates a new

[jira] [Created] (KAFKA-13521) Supress changelog schema version breaks migration

2021-12-08 Thread Hector Geraldino (Jira)
Hector Geraldino created KAFKA-13521: Summary: Supress changelog schema version breaks migration Key: KAFKA-13521 URL: https://issues.apache.org/jira/browse/KAFKA-13521 Project: Kafka

[GitHub] [kafka-site] bbejeck commented on pull request #387: Fixes KAFKA-13520

2021-12-08 Thread GitBox
bbejeck commented on pull request #387: URL: https://github.com/apache/kafka-site/pull/387#issuecomment-988878505 thanks for the fix @rmoff! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [kafka-site] bbejeck merged pull request #387: Fixes KAFKA-13520

2021-12-08 Thread GitBox
bbejeck merged pull request #387: URL: https://github.com/apache/kafka-site/pull/387 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Created] (KAFKA-13520) Quickstart does not work at topic creation step

2021-12-08 Thread Robin Moffatt (Jira)
Robin Moffatt created KAFKA-13520: - Summary: Quickstart does not work at topic creation step Key: KAFKA-13520 URL: https://issues.apache.org/jira/browse/KAFKA-13520 Project: Kafka Issue

[jira] [Created] (KAFKA-13519) Same JAAS configuration used for all producers

2021-12-08 Thread Sergey Lemekhov (Jira)
Sergey Lemekhov created KAFKA-13519: --- Summary: Same JAAS configuration used for all producers Key: KAFKA-13519 URL: https://issues.apache.org/jira/browse/KAFKA-13519 Project: Kafka Issue

[jira] [Created] (KAFKA-13518) Update gson and netty-codec in 3.0.0

2021-12-08 Thread Pavel Kuznetsov (Jira)
Pavel Kuznetsov created KAFKA-13518: --- Summary: Update gson and netty-codec in 3.0.0 Key: KAFKA-13518 URL: https://issues.apache.org/jira/browse/KAFKA-13518 Project: Kafka Issue Type: Bug