[jira] [Resolved] (KAFKA-16160) AsyncKafkaConsumer is trying to connect to a disconnected node in a tight loop

2024-05-22 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Nee resolved KAFKA-16160. Resolution: Cannot Reproduce > AsyncKafkaConsumer is trying to connect to a disconnected n

[jira] [Created] (KAFKA-16819) CoordinatorRequestManager seems to return 0ms during the coordinator discovery

2024-05-22 Thread Philip Nee (Jira)
Philip Nee created KAFKA-16819: -- Summary: CoordinatorRequestManager seems to return 0ms during the coordinator discovery Key: KAFKA-16819 URL: https://issues.apache.org/jira/browse/KAFKA-16819 Project

[jira] [Created] (KAFKA-16799) NetworkClientDelegate is not backing off if the node is not found

2024-05-20 Thread Philip Nee (Jira)
Philip Nee created KAFKA-16799: -- Summary: NetworkClientDelegate is not backing off if the node is not found Key: KAFKA-16799 URL: https://issues.apache.org/jira/browse/KAFKA-16799 Project: Kafka

[jira] [Created] (KAFKA-16778) AsyncKafkaConsumer fetcher might occasionally try to fetch to a revoked partition

2024-05-15 Thread Philip Nee (Jira)
Philip Nee created KAFKA-16778: -- Summary: AsyncKafkaConsumer fetcher might occasionally try to fetch to a revoked partition Key: KAFKA-16778 URL: https://issues.apache.org/jira/browse/KAFKA-16778

[jira] [Resolved] (KAFKA-16474) AsyncKafkaConsumer might send out heartbeat request without waiting for its response

2024-05-03 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Nee resolved KAFKA-16474. Resolution: Fixed Ran the test several times, the client log also looks fine

Re: [VOTE] KIP-909: DNS Resolution Fallures Should Not Fail the Client

2024-04-29 Thread Philip Nee
he KIP! > > > > Regards, > > > > Rajini > > > > > > On Tue, Apr 25, 2023 at 8:52 PM Philip Nee wrote: > > > > > Thanks for the vote. We've decided to make a minor change to the > default > > > timeout from 5min to 2min. > > > &

Re: [DISCUSS] KIP-909: Allow clients to rebootstrap DNS lookup failure

2024-04-29 Thread Philip Nee
also update the "examples" modules with > this new non-retriable exception. Wdyt? > > On Fri, Apr 21, 2023 at 11:13 PM Philip Nee wrote: > > > > Hey Jason, > > > > Thanks again. I've updated the KIP. > > > > P > > > > On Fri, Apr 21, 2023

[jira] [Resolved] (KAFKA-16579) Revert changes to consumer_rolling_upgrade_test.py for the new async Consumer

2024-04-19 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Nee resolved KAFKA-16579. Resolution: Fixed > Revert changes to consumer_rolling_upgrade_test.py for the new async Consu

[jira] [Resolved] (KAFKA-16389) consumer_test.py’s test_valid_assignment fails with new consumer

2024-04-18 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Nee resolved KAFKA-16389. Resolution: Fixed > consumer_test.py’s test_valid_assignment fails with new consu

[jira] [Created] (KAFKA-16474) AsyncKafkaConsumer might rapidly send out successive heartbeat causing partitions getting revoked

2024-04-04 Thread Philip Nee (Jira)
Philip Nee created KAFKA-16474: -- Summary: AsyncKafkaConsumer might rapidly send out successive heartbeat causing partitions getting revoked Key: KAFKA-16474 URL: https://issues.apache.org/jira/browse/KAFKA-16474

[jira] [Created] (KAFKA-16433) beginningOffsets and offsetsForTimes don't behave consistently when providing a zero timeout

2024-03-27 Thread Philip Nee (Jira)
Philip Nee created KAFKA-16433: -- Summary: beginningOffsets and offsetsForTimes don't behave consistently when providing a zero timeout Key: KAFKA-16433 URL: https://issues.apache.org/jira/browse/KAFKA-16433

[jira] [Created] (KAFKA-16405) Mismatch assignment error when running consumer rolling upgrade system tests

2024-03-22 Thread Philip Nee (Jira)
Philip Nee created KAFKA-16405: -- Summary: Mismatch assignment error when running consumer rolling upgrade system tests Key: KAFKA-16405 URL: https://issues.apache.org/jira/browse/KAFKA-16405 Project

[jira] [Created] (KAFKA-16390) consumer_bench_test.py failed using AsyncKafkaConsumer

2024-03-19 Thread Philip Nee (Jira)
Philip Nee created KAFKA-16390: -- Summary: consumer_bench_test.py failed using AsyncKafkaConsumer Key: KAFKA-16390 URL: https://issues.apache.org/jira/browse/KAFKA-16390 Project: Kafka Issue

[jira] [Resolved] (KAFKA-16116) AsyncKafkaConsumer: Add missing rebalance metrics

2024-02-28 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Nee resolved KAFKA-16116. Resolution: Fixed > AsyncKafkaConsumer: Add missing rebalance metr

Re: Kafka consumer group crashing and not able to consume once service is up

2024-02-08 Thread Philip Nee
, 2024 at 11:49 PM Jain, Ankit wrote: > Update the below response. > > > > *From:* Marigowda, Santhosh Aditya > *Sent:* Thursday, February 8, 2024 1:14 PM > *To:* Philip Nee ; dev@kafka.apache.org > *Cc:* Jain, Ankit ; Namboodiri, Vishnu < > vishnu.nambood...@in

Re: Kafka consumer group crashing and not able to consume once service is up

2024-02-07 Thread Philip Nee
Hi Santhosh, Your problem statement confuses me a bit (apologize). You mentioned "if one of the kafka consumer(Service)" - Do you have a single member consumer group? Could you elaborate on the setup a bit? Did you also mean after restarting the "service", the service was not able to resume from

[jira] [Resolved] (KAFKA-16115) AsyncKafkaConsumer: Add missing heartbeat metrics

2024-02-02 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Nee resolved KAFKA-16115. Resolution: Fixed > AsyncKafkaConsumer: Add missing heartbeat metr

[jira] [Resolved] (KAFKA-16113) AsyncKafkaConsumer: Add missing offset commit metrics

2024-01-19 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Nee resolved KAFKA-16113. Resolution: Fixed > AsyncKafkaConsumer: Add missing offset commit metr

[jira] [Created] (KAFKA-16160) AsyncKafkaConsumer is trying to connect to a disconnected note in a tight loop

2024-01-17 Thread Philip Nee (Jira)
Philip Nee created KAFKA-16160: -- Summary: AsyncKafkaConsumer is trying to connect to a disconnected note in a tight loop Key: KAFKA-16160 URL: https://issues.apache.org/jira/browse/KAFKA-16160 Project

[jira] [Created] (KAFKA-16159) Prune excessive logging from Telemetry Reporter

2024-01-17 Thread Philip Nee (Jira)
Philip Nee created KAFKA-16159: -- Summary: Prune excessive logging from Telemetry Reporter Key: KAFKA-16159 URL: https://issues.apache.org/jira/browse/KAFKA-16159 Project: Kafka Issue Type: Task

[jira] [Resolved] (KAFKA-16112) Review JMX metrics in Async Consumer and determine the missing ones

2024-01-12 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Nee resolved KAFKA-16112. Resolution: Fixed These are the results of this ticket   |KAFKA-16113| |KAFKA-16116| |KAFKA

[jira] [Created] (KAFKA-16116) AsyncKafkaConsumer: Add missing rebalance metrics

2024-01-11 Thread Philip Nee (Jira)
Philip Nee created KAFKA-16116: -- Summary: AsyncKafkaConsumer: Add missing rebalance metrics Key: KAFKA-16116 URL: https://issues.apache.org/jira/browse/KAFKA-16116 Project: Kafka Issue Type

[jira] [Created] (KAFKA-16115) AsyncKafkaConsumer: Add missing heartbeat metrics

2024-01-11 Thread Philip Nee (Jira)
Philip Nee created KAFKA-16115: -- Summary: AsyncKafkaConsumer: Add missing heartbeat metrics Key: KAFKA-16115 URL: https://issues.apache.org/jira/browse/KAFKA-16115 Project: Kafka Issue Type

[jira] [Created] (KAFKA-16113) AsyncKafkaConsumer: Add missing offset commit metrics

2024-01-10 Thread Philip Nee (Jira)
Philip Nee created KAFKA-16113: -- Summary: AsyncKafkaConsumer: Add missing offset commit metrics Key: KAFKA-16113 URL: https://issues.apache.org/jira/browse/KAFKA-16113 Project: Kafka Issue Type

[jira] [Resolved] (KAFKA-15250) DefaultBackgroundThread is running tight loop

2024-01-08 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Nee resolved KAFKA-15250. Fix Version/s: (was: 3.8.0) Resolution: Fixed > DefaultBackgroundThread is runn

Re: [Question] About Kafka producer design decision making

2024-01-02 Thread Philip Nee
hey sean - a lot of uses of the Futures are in the public API and therefore take voting/effort to be changed. i don't know any reason for intentionally avoiding the use of CompletableFuture, however, others might know more than I do. thanks, P On Tue, Nov 14, 2023 at 1:27 AM 신수웅(Sean Sin)

[jira] [Resolved] (KAFKA-15948) Refactor AsyncKafkaConsumer shutdown

2023-12-27 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Nee resolved KAFKA-15948. Assignee: Philip Nee (was: Phuc Hong Tran) Resolution: Fixed > Refactor AsyncKafkaConsu

Re: [ANNOUNCE] New Kafka PMC Member: Divij Vaidya

2023-12-27 Thread Philip Nee
congrats divij! On Wed, Dec 27, 2023 at 8:55 AM Justine Olshan wrote: > Congratulations Divij! > > On Wed, Dec 27, 2023 at 4:20 AM Gaurav Narula wrote: > > > Congratulations Divij! > > > > Regards, > > Gaurav > > > > > On 27-Dec-2023, at 17:44, Mickael Maison > > wrote: > > > > > >

Re: Kafka trunk test & build stability

2023-12-21 Thread Philip Nee
Hey Sophie - I've gotten 2 inflight PRs each with more than 15 retries... Namely: https://github.com/apache/kafka/pull/15023 and https://github.com/apache/kafka/pull/15035 justin filed a flaky test report here though: https://issues.apache.org/jira/browse/KAFKA-16045 P On Thu, Dec 21, 2023 at

[jira] [Created] (KAFKA-16034) AsyncKafkaConsumer will get Invalid Request error when trying to rejoin on fenced/unknown member Id

2023-12-19 Thread Philip Nee (Jira)
Philip Nee created KAFKA-16034: -- Summary: AsyncKafkaConsumer will get Invalid Request error when trying to rejoin on fenced/unknown member Id Key: KAFKA-16034 URL: https://issues.apache.org/jira/browse/KAFKA-16034

[jira] [Created] (KAFKA-16027) Refactor MetadataTest#testUpdatePartitionLeadership

2023-12-18 Thread Philip Nee (Jira)
Philip Nee created KAFKA-16027: -- Summary: Refactor MetadataTest#testUpdatePartitionLeadership Key: KAFKA-16027 URL: https://issues.apache.org/jira/browse/KAFKA-16027 Project: Kafka Issue Type

[jira] [Created] (KAFKA-16026) AsyncConsumer does not send a poll event to the background thread

2023-12-18 Thread Philip Nee (Jira)
Philip Nee created KAFKA-16026: -- Summary: AsyncConsumer does not send a poll event to the background thread Key: KAFKA-16026 URL: https://issues.apache.org/jira/browse/KAFKA-16026 Project: Kafka

[jira] [Created] (KAFKA-16024) SaslPlaintextConsumerTest#testCoordinatorFailover is flaky

2023-12-17 Thread Philip Nee (Jira)
Philip Nee created KAFKA-16024: -- Summary: SaslPlaintextConsumerTest#testCoordinatorFailover is flaky Key: KAFKA-16024 URL: https://issues.apache.org/jira/browse/KAFKA-16024 Project: Kafka Issue

[jira] [Created] (KAFKA-16023) PlaintextConsumerTest needs to wait for reconciliation to complete before proceeding

2023-12-15 Thread Philip Nee (Jira)
Philip Nee created KAFKA-16023: -- Summary: PlaintextConsumerTest needs to wait for reconciliation to complete before proceeding Key: KAFKA-16023 URL: https://issues.apache.org/jira/browse/KAFKA-16023

[jira] [Created] (KAFKA-16022) AsyncKafkaConsumer sometimes complains "No current assignment for partition {}"

2023-12-15 Thread Philip Nee (Jira)
Philip Nee created KAFKA-16022: -- Summary: AsyncKafkaConsumer sometimes complains "No current assignment for partition {}" Key: KAFKA-16022 URL: https://issues.apache.org/jira/browse/KAFKA-16022

[jira] [Created] (KAFKA-16019) Some of the tests in PlaintextConsumer can't seem to deterministically invokes and verify the consumer callback

2023-12-15 Thread Philip Nee (Jira)
Philip Nee created KAFKA-16019: -- Summary: Some of the tests in PlaintextConsumer can't seem to deterministically invokes and verify the consumer callback Key: KAFKA-16019 URL: https://issues.apache.org/jira/browse

[jira] [Created] (KAFKA-15993) Enable integration tests that relies on rebalance listener

2023-12-11 Thread Philip Nee (Jira)
Philip Nee created KAFKA-15993: -- Summary: Enable integration tests that relies on rebalance listener Key: KAFKA-15993 URL: https://issues.apache.org/jira/browse/KAFKA-15993 Project: Kafka Issue

[jira] [Created] (KAFKA-15948) Refactor AsyncKafkaConsumer shutdown

2023-11-29 Thread Philip Nee (Jira)
Philip Nee created KAFKA-15948: -- Summary: Refactor AsyncKafkaConsumer shutdown Key: KAFKA-15948 URL: https://issues.apache.org/jira/browse/KAFKA-15948 Project: Kafka Issue Type: Bug

[jira] [Created] (KAFKA-15946) AsyncKafkaConsumer should retry commits on the application thread instead of autoretry

2023-11-29 Thread Philip Nee (Jira)
Philip Nee created KAFKA-15946: -- Summary: AsyncKafkaConsumer should retry commits on the application thread instead of autoretry Key: KAFKA-15946 URL: https://issues.apache.org/jira/browse/KAFKA-15946

[jira] [Created] (KAFKA-15942) Implement ConsumerInterceptor

2023-11-29 Thread Philip Nee (Jira)
Philip Nee created KAFKA-15942: -- Summary: Implement ConsumerInterceptor Key: KAFKA-15942 URL: https://issues.apache.org/jira/browse/KAFKA-15942 Project: Kafka Issue Type: Improvement

[jira] [Created] (KAFKA-15913) Remove excessive use of spy in CosumerTestBuilder

2023-11-28 Thread Philip Nee (Jira)
Philip Nee created KAFKA-15913: -- Summary: Remove excessive use of spy in CosumerTestBuilder Key: KAFKA-15913 URL: https://issues.apache.org/jira/browse/KAFKA-15913 Project: Kafka Issue Type

[jira] [Created] (KAFKA-15872) Investigate autocommit retry logic

2023-11-21 Thread Philip Nee (Jira)
Philip Nee created KAFKA-15872: -- Summary: Investigate autocommit retry logic Key: KAFKA-15872 URL: https://issues.apache.org/jira/browse/KAFKA-15872 Project: Kafka Issue Type: Sub-task

[jira] [Resolved] (KAFKA-15836) KafkaConsumer subscribes to multiple topics does not respect max.poll.records

2023-11-21 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Nee resolved KAFKA-15836. Resolution: Fixed PR merged. > KafkaConsumer subscribes to multiple topics does not resp

[jira] [Created] (KAFKA-15867) Should ConsumerNetworkThread wrap the exception and notify the polling thread?

2023-11-20 Thread Philip Nee (Jira)
Philip Nee created KAFKA-15867: -- Summary: Should ConsumerNetworkThread wrap the exception and notify the polling thread? Key: KAFKA-15867 URL: https://issues.apache.org/jira/browse/KAFKA-15867 Project

[jira] [Created] (KAFKA-15866) Refactor OffsetFetchRequestState Error handling to be more consistent with OffsetCommitRequestState

2023-11-20 Thread Philip Nee (Jira)
Philip Nee created KAFKA-15866: -- Summary: Refactor OffsetFetchRequestState Error handling to be more consistent with OffsetCommitRequestState Key: KAFKA-15866 URL: https://issues.apache.org/jira/browse/KAFKA-15866

[jira] [Created] (KAFKA-15865) Ensure consumer.poll() execute autocommit callback

2023-11-20 Thread Philip Nee (Jira)
Philip Nee created KAFKA-15865: -- Summary: Ensure consumer.poll() execute autocommit callback Key: KAFKA-15865 URL: https://issues.apache.org/jira/browse/KAFKA-15865 Project: Kafka Issue Type

[jira] [Resolved] (KAFKA-15174) Ensure the correct thread is executing the callbacks

2023-11-20 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Nee resolved KAFKA-15174. Resolution: Fixed > Ensure the correct thread is executing the callba

[jira] [Created] (KAFKA-15836) KafkaConsumer subscribe to multiple topics does not respect max.poll.records

2023-11-15 Thread Philip Nee (Jira)
Philip Nee created KAFKA-15836: -- Summary: KafkaConsumer subscribe to multiple topics does not respect max.poll.records Key: KAFKA-15836 URL: https://issues.apache.org/jira/browse/KAFKA-15836 Project

[jira] [Created] (KAFKA-15818) Implement max poll internval

2023-11-13 Thread Philip Nee (Jira)
Philip Nee created KAFKA-15818: -- Summary: Implement max poll internval Key: KAFKA-15818 URL: https://issues.apache.org/jira/browse/KAFKA-15818 Project: Kafka Issue Type: Task

Re: [DISCUSS] Should we continue to merge without a green build? No!

2023-11-13 Thread Philip Nee
Hi David et al, I agree with all the suggestions, but from what I've seen, the flaky tests tend to get ignored, and I'm afraid that disabling them would leave them getting forgotten. If the Jira ticket is accurate, we've got plenty of tickets opened for > 2 years

[jira] [Created] (KAFKA-15803) Update last seen epcoh during commit

2023-11-09 Thread Philip Nee (Jira)
Philip Nee created KAFKA-15803: -- Summary: Update last seen epcoh during commit Key: KAFKA-15803 URL: https://issues.apache.org/jira/browse/KAFKA-15803 Project: Kafka Issue Type: Task

[jira] [Resolved] (KAFKA-15686) Consumer should be able to detect network problem

2023-11-06 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Nee resolved KAFKA-15686. Assignee: Philip Nee Resolution: Won't Fix The user can use the admin client to get

[jira] [Created] (KAFKA-15775) Implement listTopics() and partitionFor() for the AsyncKafkaConsumer

2023-11-02 Thread Philip Nee (Jira)
Philip Nee created KAFKA-15775: -- Summary: Implement listTopics() and partitionFor() for the AsyncKafkaConsumer Key: KAFKA-15775 URL: https://issues.apache.org/jira/browse/KAFKA-15775 Project: Kafka

[jira] [Resolved] (KAFKA-15679) Client support for new consumer configs

2023-11-02 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Nee resolved KAFKA-15679. Resolution: Fixed > Client support for new consumer conf

[jira] [Resolved] (KAFKA-15531) Ensure coordinator node is removed upon disconnection exception

2023-11-02 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Nee resolved KAFKA-15531. Resolution: Fixed > Ensure coordinator node is removed upon disconnection except

[jira] [Resolved] (KAFKA-15316) CommitRequestManager not calling RequestState callbacks

2023-11-02 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Nee resolved KAFKA-15316. Resolution: Fixed > CommitRequestManager not calling RequestState callba

[jira] [Resolved] (KAFKA-15562) Ensure fetch offset and commit offset handler handles both timeout and various error types

2023-11-02 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Nee resolved KAFKA-15562. Resolution: Fixed > Ensure fetch offset and commit offset handler handles both time

[jira] [Created] (KAFKA-15773) Group protocol configuration should be validated

2023-11-02 Thread Philip Nee (Jira)
Philip Nee created KAFKA-15773: -- Summary: Group protocol configuration should be validated Key: KAFKA-15773 URL: https://issues.apache.org/jira/browse/KAFKA-15773 Project: Kafka Issue Type

[jira] [Resolved] (KAFKA-15631) Do not send new heartbeat request while another one in-flight

2023-10-30 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Nee resolved KAFKA-15631. Resolution: Not A Problem > Do not send new heartbeat request while another one in-fli

[jira] [Resolved] (KAFKA-15550) OffsetsForTimes validation for negative timestamps in new consumer

2023-10-27 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Nee resolved KAFKA-15550. Resolution: Fixed > OffsetsForTimes validation for negative timestamps in new consu

[jira] [Created] (KAFKA-15697) Add local assignor and ensure it cannot be used with server side assignor

2023-10-26 Thread Philip Nee (Jira)
Philip Nee created KAFKA-15697: -- Summary: Add local assignor and ensure it cannot be used with server side assignor Key: KAFKA-15697 URL: https://issues.apache.org/jira/browse/KAFKA-15697 Project: Kafka

Re: [DISCUSS] How to detect (and prevent) complex bugs in Kafka?

2023-10-24 Thread Philip Nee
Hi Divij, I have limited experience tackling bugs in Kafka, but for the issues I've fixed are usually found through..: 1. flaky unit/integration/system tests 2. Insufficient test coverage: When trying to add tests... 3. Related to #2 - Found through production at scale I have heard Kraft

[jira] [Created] (KAFKA-15642) Ensure offset fetcher behaves correctly when the request is Timeout

2023-10-18 Thread Philip Nee (Jira)
Philip Nee created KAFKA-15642: -- Summary: Ensure offset fetcher behaves correctly when the request is Timeout Key: KAFKA-15642 URL: https://issues.apache.org/jira/browse/KAFKA-15642 Project: Kafka

Re: [DISCUSS] KIP-983: Full speed async processing during rebalance

2023-10-14 Thread Philip Nee
; > Best, > > David > > > > Le ven. 13 oct. 2023 à 20:44, Erik van Oosten .invalid> > > a écrit : > > > >> Thanks Philip, > >> > >> No worries, I am not in a hurry. Knowing this is not forgotten is enough > >> for me. If there is an

Re: [DISCUSS] KIP-983: Full speed async processing during rebalance

2023-10-13 Thread Philip Nee
Hi Erik, Sorry for the delay, I have not finished reviewing the KIP, but I also have not forgotten about it! In general, KIP review process can be lengthy, so I think mailing list is the best bet to get the committer's attention. P On Fri, Oct 13, 2023 at 10:55 AM Erik van Oosten wrote: > Hi

[jira] [Created] (KAFKA-15592) Member does not need to always try to join a group when a groupId is configured

2023-10-11 Thread Philip Nee (Jira)
Philip Nee created KAFKA-15592: -- Summary: Member does not need to always try to join a group when a groupId is configured Key: KAFKA-15592 URL: https://issues.apache.org/jira/browse/KAFKA-15592 Project

[jira] [Created] (KAFKA-15588) Purge the unsent offset commits/fetches when the member is fenced

2023-10-11 Thread Philip Nee (Jira)
Philip Nee created KAFKA-15588: -- Summary: Purge the unsent offset commits/fetches when the member is fenced Key: KAFKA-15588 URL: https://issues.apache.org/jira/browse/KAFKA-15588 Project: Kafka

[jira] [Resolved] (KAFKA-15278) Implement client support for KIP-848 ConsumerGroupHeartbeat protocol RPC

2023-10-09 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Nee resolved KAFKA-15278. Resolution: Fixed > Implement client support for KIP-848 ConsumerGroupHeartbeat protocol

[jira] [Created] (KAFKA-15562) CommitRequestManager needs to test different error handling

2023-10-06 Thread Philip Nee (Jira)
Philip Nee created KAFKA-15562: -- Summary: CommitRequestManager needs to test different error handling Key: KAFKA-15562 URL: https://issues.apache.org/jira/browse/KAFKA-15562 Project: Kafka

[jira] [Created] (KAFKA-15553) Review committed offset refresh logic

2023-10-05 Thread Philip Nee (Jira)
Philip Nee created KAFKA-15553: -- Summary: Review committed offset refresh logic Key: KAFKA-15553 URL: https://issues.apache.org/jira/browse/KAFKA-15553 Project: Kafka Issue Type: Bug

[jira] [Created] (KAFKA-15551) Evaluate conditions for short circuiting consumer API calls

2023-10-05 Thread Philip Nee (Jira)
Philip Nee created KAFKA-15551: -- Summary: Evaluate conditions for short circuiting consumer API calls Key: KAFKA-15551 URL: https://issues.apache.org/jira/browse/KAFKA-15551 Project: Kafka

[jira] [Created] (KAFKA-15548) Handling close() properly

2023-10-04 Thread Philip Nee (Jira)
Philip Nee created KAFKA-15548: -- Summary: Handling close() properly Key: KAFKA-15548 URL: https://issues.apache.org/jira/browse/KAFKA-15548 Project: Kafka Issue Type: Bug Components

[jira] [Created] (KAFKA-15534) Propagate client response time when timeout to the request handler

2023-10-03 Thread Philip Nee (Jira)
Philip Nee created KAFKA-15534: -- Summary: Propagate client response time when timeout to the request handler Key: KAFKA-15534 URL: https://issues.apache.org/jira/browse/KAFKA-15534 Project: Kafka

[jira] [Created] (KAFKA-15533) Ensure HeartbeatRequestManager only send out some fields once

2023-10-03 Thread Philip Nee (Jira)
Philip Nee created KAFKA-15533: -- Summary: Ensure HeartbeatRequestManager only send out some fields once Key: KAFKA-15533 URL: https://issues.apache.org/jira/browse/KAFKA-15533 Project: Kafka

[jira] [Created] (KAFKA-15531) Ensure coordinator node is removed upon disconnection exception

2023-10-03 Thread Philip Nee (Jira)
Philip Nee created KAFKA-15531: -- Summary: Ensure coordinator node is removed upon disconnection exception Key: KAFKA-15531 URL: https://issues.apache.org/jira/browse/KAFKA-15531 Project: Kafka

Re: [ANNOUNCE] New Kafka PMC Member: Justine Olshan

2023-09-22 Thread Philip Nee
Congrats Justine! On Fri, Sep 22, 2023 at 7:07 PM Luke Chen wrote: > Hi, Everyone, > > Justine Olshan has been a Kafka committer since Dec. 2022. She has been > very active and instrumental to the community since becoming a committer. > It's my pleasure to announce that Justine is now a member

Re: [DISCUSS] KIP-714: Client metrics and observability

2023-09-22 Thread Philip Nee
Hi Andrew - Question on top of your answers: Do you think the client should actively search for a broker that supports this RPC? As previously mentioned, the broker uses the leastLoadedNode to find its first connection (am I correct?), and what if that broker doesn't support the metric push? P

Re: [ANNOUNCE] New committer: Lucas Brutschy

2023-09-21 Thread Philip Nee
Nice! On Thu, Sep 21, 2023 at 9:21 AM Kirk True wrote: > Congratulations, Lucas! > > > On Sep 21, 2023, at 9:16 AM, Sagar wrote: > > > > Congrats Lucas ! > > > > On Thu, 21 Sep 2023 at 9:15 PM, Bruno Cadonna > wrote: > > > >> Hi all, > >> > >> The PMC of Apache Kafka is pleased to announce a

[jira] [Created] (KAFKA-15475) Timeout request might retry forever even if the user API times out in AsyncConsumer

2023-09-18 Thread Philip Nee (Jira)
Philip Nee created KAFKA-15475: -- Summary: Timeout request might retry forever even if the user API times out in AsyncConsumer Key: KAFKA-15475 URL: https://issues.apache.org/jira/browse/KAFKA-15475

[jira] [Created] (KAFKA-15474) AbstractCoordinator.testWakeupAfterSyncGroupReceivedExternalCompletion seems flaky

2023-09-18 Thread Philip Nee (Jira)
Philip Nee created KAFKA-15474: -- Summary: AbstractCoordinator.testWakeupAfterSyncGroupReceivedExternalCompletion seems flaky Key: KAFKA-15474 URL: https://issues.apache.org/jira/browse/KAFKA-15474

Re: [VOTE] KIP-714: Client metrics and observability

2023-09-13 Thread Philip Nee
e > pushes is best, but if you choose the “wrong” node to start with, it’s not > ideal. > > Thanks, > Andrew > > > On 8 Sep 2023, at 19:29, Philip Nee wrote: > > > > Hey Andrew - > > > > +1 but I don't have a binding vote! > > > > It t

Re: [VOTE] KIP-714: Client metrics and observability

2023-09-08 Thread Philip Nee
Hey Andrew - +1 but I don't have a binding vote! It took me a while to go through the KIP. Here are some of my notes during the reading: *Metrics* - Should we care about the client's leader epoch? There is a case where the user recreates the topic, but the consumer thinks it is still the same

Re: What are the biggest issues with Apache Kafka?

2023-08-25 Thread Philip Nee
Hi Arpit: Here's a good starting point for Kraft-related issues: https://issues.apache.org/jira/browse/KAFKA-15401?jql=project%20%3D%20KAFKA%20AND%20component%20in%20(core%2C%20kraft) The recent Jenkin's builds have also been extremely flaky - This greatly impacts productivity. This would also

Re: What are the biggest issues with Apache Kafka?

2023-08-10 Thread Philip Nee
Hey Liam, I think apache jira would be a great place to find lower hanging fruits. We also have lots of flaky tests to resolve. What is you familiarity with Kafka? If you are new, i would suggest looking at the demos and examples in the repo, and maybe try to improve them. There is also a lot

[jira] [Created] (KAFKA-15333) Flaky build failure throwing Connect Exception: Could not connect to server....

2023-08-10 Thread Philip Nee (Jira)
Philip Nee created KAFKA-15333: -- Summary: Flaky build failure throwing Connect Exception: Could not connect to server Key: KAFKA-15333 URL: https://issues.apache.org/jira/browse/KAFKA-15333 Project

[jira] [Created] (KAFKA-15305) The background thread should try to process the remaining task until the shutdown timer is expired

2023-08-04 Thread Philip Nee (Jira)
Philip Nee created KAFKA-15305: -- Summary: The background thread should try to process the remaining task until the shutdown timer is expired Key: KAFKA-15305 URL: https://issues.apache.org/jira/browse/KAFKA-15305

[jira] [Created] (KAFKA-15304) CompletableApplicationEvents aren't being completed when the consumer is closing

2023-08-04 Thread Philip Nee (Jira)
Philip Nee created KAFKA-15304: -- Summary: CompletableApplicationEvents aren't being completed when the consumer is closing Key: KAFKA-15304 URL: https://issues.apache.org/jira/browse/KAFKA-15304 Project

[jira] [Created] (KAFKA-15250) DefaultBackgroundThread is running tight loop

2023-07-25 Thread Philip Nee (Jira)
Philip Nee created KAFKA-15250: -- Summary: DefaultBackgroundThread is running tight loop Key: KAFKA-15250 URL: https://issues.apache.org/jira/browse/KAFKA-15250 Project: Kafka Issue Type: Bug

Re: [DISCUSS] KIP-951: Leader discovery optimisations for the client

2023-07-17 Thread Philip Nee
nse for the Metadata RPC? I think brokers always > have a view of the cluster, although it can be stale,it would always return > a leader(whether old or new). > > Mayank > > On Fri, Jul 14, 2023 at 8:53 PM Philip Nee wrote: > > > Hey Mayank, > > > > Thanks for

Re: [DISCUSS] KIP-951: Leader discovery optimisations for the client

2023-07-14 Thread Philip Nee
Hey Mayank, Thanks for the KIP. I think this is a great proposal, and I'm in favor of this idea. A few comments: 1. Claiming metadata refresh is done asynchronously is misleading. The metadata refresh requires Network Client to be physically polled, which is done in a separate thread in

Re: [VOTE] KIP-944 Support async runtimes in consumer, votes needed!

2023-07-11 Thread Philip Nee
of converting this in to KIP-945. Thanks, P On Tue, Jul 11, 2023 at 10:33 AM Philip Nee wrote: > Hey Erik, > > Sorry for holding up this email for a few days since Colin's response > includes some of my concerns. I'm in favor of this KIP, and I think your > approach seems safe. Of cou

Re: [VOTE] KIP-944 Support async runtimes in consumer, votes needed!

2023-07-11 Thread Philip Nee
Hey Erik, Sorry for holding up this email for a few days since Colin's response includes some of my concerns. I'm in favor of this KIP, and I think your approach seems safe. Of course, I probably missed something therefore I think this KIP needs to cover different use cases to demonstrate it

[jira] [Created] (KAFKA-15175) Assess the use of nio2 asynchronous channel for KafkaConsumer

2023-07-10 Thread Philip Nee (Jira)
Philip Nee created KAFKA-15175: -- Summary: Assess the use of nio2 asynchronous channel for KafkaConsumer Key: KAFKA-15175 URL: https://issues.apache.org/jira/browse/KAFKA-15175 Project: Kafka

Re: KafkaConsumer refactor proposal

2023-07-10 Thread Philip Nee
e reached a conclusion on the above points, perhaps, we can > create JIRAs to address them and work with the community members to resolve > them. > > [1] > > https://github.com/netty/netty/blob/57d536c59e434e88c0d8374d6b03671edb02885f/common/src/main/java/io/netty/util/concurrent

[jira] [Created] (KAFKA-15174) Ensure the correct thread is executing the callbacks

2023-07-10 Thread Philip Nee (Jira)
Philip Nee created KAFKA-15174: -- Summary: Ensure the correct thread is executing the callbacks Key: KAFKA-15174 URL: https://issues.apache.org/jira/browse/KAFKA-15174 Project: Kafka Issue Type

[jira] [Created] (KAFKA-15173) ApplicationEventQueue and BackgroundEventQueue should be bounded

2023-07-10 Thread Philip Nee (Jira)
Philip Nee created KAFKA-15173: -- Summary: ApplicationEventQueue and BackgroundEventQueue should be bounded Key: KAFKA-15173 URL: https://issues.apache.org/jira/browse/KAFKA-15173 Project: Kafka

Re: [ANNOUNCE] New committer: Divij Vaidya

2023-06-13 Thread Philip Nee
Congrats! On Tue, Jun 13, 2023 at 8:17 PM Randall Hauch wrote: > Congratulations! > > On Tue, Jun 13, 2023 at 12:48 PM Matthias J. Sax wrote: > > > Congrats! > > > > On 6/13/23 10:24 AM, Satish Duggana wrote: > > > Congratulations Divij!! > > > > > > On Tue, 13 Jun 2023 at 22:41, Manyanda

Re: [DISCUSS] Regarding Old PRs

2023-06-01 Thread Philip Nee
Hey Josep, 2-3weeks is pretty optimistic, but anything over a year can probably be closed. If there's no response. If they don't or we can't get any answer, we could close the > PRs after 2 to 3 weeks. Regular contributors can't label their own PRs though - it requires a committer/collaborator

Re: KafkaConsumer refactor proposal

2023-05-23 Thread Philip Nee
Hi all! I've added the "KIP-848" and "ctr" tags to the relevant PRs. see: https://github.com/apache/kafka/pulls?q=label%3Actr+sort%3Acreated-asc On Tue, May 9, 2023 at 10:29 AM Philip Nee wrote: > Hey Divij, > > Firstly, THANK YOU for reaching out and providing th

Re: KafkaConsumer refactor proposal

2023-05-09 Thread Philip Nee
uld have the visibility to > identify that a need to scale the background thread processing is required. > > -- > Divij Vaidya > > > > On Sun, Sep 18, 2022 at 11:24 PM Philip Nee wrote: > > > On Sun, Sep 18, 2022 at 6:03 AM Luke Chen wrote: > > > > > Hi Phi

  1   2   >