Re: [VOTE] KIP-1023: Follower fetch from tiered offset
Hi All, This KIP is accepted with 3 +1 binding votes(Luke, Christo, Jun, Satish) and 2 +1 non-binding votes(Kamal, Omnia). Thank you all for voting. Abhijeet. On Sat, Apr 27, 2024 at 5:50 AM Satish Duggana wrote: > Thanks Abhijeet for the KIP. > +1 from me. > > ~Satish > > On Fri, 26 Apr 2024 at 8:35 PM, Jun Rao wrote: > > > Hi, Abhijeet, > > > > Thanks for the KIP. +1 > > > > Jun > > > > On Thu, Apr 25, 2024 at 10:30 PM Abhijeet Kumar < > > abhijeet.cse@gmail.com> > > wrote: > > > > > Hi All, > > > > > > I would like to start the vote for KIP-1023 - Follower fetch from > tiered > > > offset > > > > > > The KIP is here: > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1023%3A+Follower+fetch+from+tiered+offset > > > > > > Regards. > > > Abhijeet. > > > > > >
[jira] [Created] (KAFKA-16640) Replace TestUtils#resource by scala.util.Using
Chia-Ping Tsai created KAFKA-16640: -- Summary: Replace TestUtils#resource by scala.util.Using Key: KAFKA-16640 URL: https://issues.apache.org/jira/browse/KAFKA-16640 Project: Kafka Issue Type: Improvement Reporter: Chia-Ping Tsai Assignee: Chia-Ping Tsai `scala.util.Using` is in both scala 2.13 and scala-collection-compat so we don't need to have custom try-resource function. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-16639) AsyncKafkaConsumer#close does not send heartbeat to leave group
Chia-Ping Tsai created KAFKA-16639: -- Summary: AsyncKafkaConsumer#close does not send heartbeat to leave group Key: KAFKA-16639 URL: https://issues.apache.org/jira/browse/KAFKA-16639 Project: Kafka Issue Type: Bug Reporter: Chia-Ping Tsai Assignee: Chia-Ping Tsai This bug can be reproduced by immediately closing a consumer which is just created. The root cause is that we skip the new heartbeat used to leave group when there is a in-flight heartbeat request ([https://github.com/apache/kafka/blob/5de5d967adffd864bad3ec729760a430253abf38/clients/src/main/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManager.java#L212).] It seems to me the simple solution is that we create a heartbeat request when meeting above situation and then send it by pollOnClose ([https://github.com/apache/kafka/blob/5de5d967adffd864bad3ec729760a430253abf38/clients/src/main/java/org/apache/kafka/clients/consumer/internals/RequestManager.java#L62).] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-16638) Align the naming convention for config and default variables in *Config classes
Omnia Ibrahim created KAFKA-16638: - Summary: Align the naming convention for config and default variables in *Config classes Key: KAFKA-16638 URL: https://issues.apache.org/jira/browse/KAFKA-16638 Project: Kafka Issue Type: Task Reporter: Omnia Ibrahim Some classes in the code is violating the naming naming convention for config, doc, and default variables which is: * `_CONFIG` suffix for defining the configuration * `_DEFAULT` suffix for default value * `_DOC` suffix for doc The following classes need to be updated * `CleanerConfig` and `RemoteLogManagerConfig` to use `_CONFIG` suffix instead of `_PROP`. * Others like `LogConfig` and `QuorumConfig` to use `_DEFAULT` suffix instead of `DEFAULT_` prefix . * same goes with `CommonClientConfigs`, `StreamsConfig` however these are public interfaces and will need a KIP to rename the default value variables and mark the old one as deprecated. This might need to be broken to different Jira. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-16637) KIP-848 does not work well
sanghyeok An created KAFKA-16637: Summary: KIP-848 does not work well Key: KAFKA-16637 URL: https://issues.apache.org/jira/browse/KAFKA-16637 Project: Kafka Issue Type: Bug Reporter: sanghyeok An -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [DISCUSS] KIP-1018: Introduce max remote fetch timeout config
Christo, Updated the KIP with the remote fetch latency metric. Please take another look! -- Kamal On Sun, Apr 28, 2024 at 12:23 PM Kamal Chandraprakash < kamal.chandraprak...@gmail.com> wrote: > Hi Federico, > > Thanks for the suggestion! Updated the config name to " > remote.fetch.max.wait.ms". > > Christo, > > Good point. We don't have the remote-read latency metrics to measure the > performance of the remote read requests. I'll update the KIP to emit this > metric. > > -- > Kamal > > > On Sat, Apr 27, 2024 at 4:03 PM Federico Valeri > wrote: > >> Hi Kamal, it looks like all TS configurations starts with "remote." >> prefix, so I was wondering if we should name it >> "remote.fetch.max.wait.ms". >> >> On Fri, Apr 26, 2024 at 7:07 PM Kamal Chandraprakash >> wrote: >> > >> > Hi all, >> > >> > If there are no more comments, I'll start a vote thread by tomorrow. >> > Please review the KIP. >> > >> > Thanks, >> > Kamal >> > >> > On Sat, Mar 30, 2024 at 11:08 PM Kamal Chandraprakash < >> > kamal.chandraprak...@gmail.com> wrote: >> > >> > > Hi all, >> > > >> > > Bumping the thread. Please review this KIP. Thanks! >> > > >> > > On Thu, Feb 1, 2024 at 9:11 PM Kamal Chandraprakash < >> > > kamal.chandraprak...@gmail.com> wrote: >> > > >> > >> Hi Jorge, >> > >> >> > >> Thanks for the review! Added your suggestions to the KIP. PTAL. >> > >> >> > >> The `fetch.max.wait.ms` config will be also applicable for topics >> > >> enabled with remote storage. >> > >> Updated the description to: >> > >> >> > >> ``` >> > >> The maximum amount of time the server will block before answering the >> > >> fetch request >> > >> when it is reading near to the tail of the partition >> (high-watermark) and >> > >> there isn't >> > >> sufficient data to immediately satisfy the requirement given by >> > >> fetch.min.bytes. >> > >> ``` >> > >> >> > >> -- >> > >> Kamal >> > >> >> > >> On Thu, Feb 1, 2024 at 12:12 AM Jorge Esteban Quilcate Otoya < >> > >> quilcate.jo...@gmail.com> wrote: >> > >> >> > >>> Hi Kamal, >> > >>> >> > >>> Thanks for this KIP! It should help to solve one of the main issues >> with >> > >>> tiered storage at the moment that is dealing with individual >> consumer >> > >>> configurations to avoid flooding logs with interrupted exceptions. >> > >>> >> > >>> One of the topics discussed in [1][2] was on the semantics of ` >> > >>> fetch.max.wait.ms` and how it's affected by remote storage. Should >> we >> > >>> consider within this KIP the update of `fetch.max.wail.ms` docs to >> > >>> clarify >> > >>> it only applies to local storage? >> > >>> >> > >>> Otherwise, LGTM -- looking forward to see this KIP adopted. >> > >>> >> > >>> [1] https://issues.apache.org/jira/browse/KAFKA-15776 >> > >>> [2] >> https://github.com/apache/kafka/pull/14778#issuecomment-1820588080 >> > >>> >> > >>> On Tue, 30 Jan 2024 at 01:01, Kamal Chandraprakash < >> > >>> kamal.chandraprak...@gmail.com> wrote: >> > >>> >> > >>> > Hi all, >> > >>> > >> > >>> > I have opened a KIP-1018 >> > >>> > < >> > >>> > >> > >>> >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1018%3A+Introduce+max+remote+fetch+timeout+config+for+DelayedRemoteFetch+requests >> > >>> > > >> > >>> > to introduce dynamic max-remote-fetch-timeout broker config to >> give >> > >>> more >> > >>> > control to the operator. >> > >>> > >> > >>> > >> > >>> > >> > >>> >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1018%3A+Introduce+max+remote+fetch+timeout+config+for+DelayedRemoteFetch+requests >> > >>> > >> > >>> > Let me know if you have any feedback or suggestions. >> > >>> > >> > >>> > -- >> > >>> > Kamal >> > >>> > >> > >>> >> > >> >> >
