Re: [PR] KAFKA-2372: Add Kafka-backed storage of Copycat configs. [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #241: URL: https://github.com/apache/kafka/pull/241#issuecomment-1997289420 This also adds some other needed infrastructure for distributed Copycat, most importantly the DistributedHerder, and refactors some code for handling Kafka-backed logs into KafkaBasedLog

Re: [PR] Finer locking in log append [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #43: URL: https://github.com/apache/kafka/pull/43#issuecomment-1997289412 This patch adds finer locking when appending to log. It breaks global append lock into 2 sequential and 1 parallel phase. Basic idea is to allow every thread to "reserve" offsets in

Re: [PR] KAFKA-1997: Follow-up patch, hardcode key/value serializer in mirror maker to byte serializer. [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #120: URL: https://github.com/apache/kafka/pull/120#issuecomment-1997289383 Hardcode the key/value serializer to ByteArraySerializer according to Jun’s comments. -- This is an automated message from the Apache Git Service. To respond to the message, please log

Re: [PR] Updated testing readme [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #187: URL: https://github.com/apache/kafka/pull/187#issuecomment-1997289362 Minor update to point to testing tutorial, and install the correct version of vagrant-hostmanager -- This is an automated message from the Apache Git Service. To respond to the

Re: [PR] KAFKA-1888: rolling upgrade test [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #229: URL: https://github.com/apache/kafka/pull/229#issuecomment-1997289379 @ewencp @gwenshap This needs some refactoring to avoid the duplicated code between replication test and upgrade test, but in shape for initial feedback. I'm interested in feedback

Re: [PR] MINOR: Use the correct processor id in the processor thread name [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #294: URL: https://github.com/apache/kafka/pull/294#issuecomment-1997289341 This restores the behaviour before 1265d7cb7. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] KAFKA-2386; increase timeouts for transient test failure in ConsumerCoordinatorResponseTests [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #107: URL: https://github.com/apache/kafka/pull/107#issuecomment-1997289328 There are two race conditions in the test case "testGenerationIdIncrementsOnRebalance." First, a delay before the second join group request can timeout the initial group and cause the

Re: [PR] KAFKA-2425; Copy latest docs to kafka repo docs/ directory [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #171: URL: https://github.com/apache/kafka/pull/171#issuecomment-1997289306 This PR copies the latest kafka docs to kafka repo docs directory. Here I have copied 0.8.3/ directory contents from svn website repo to kafka/docs repository. Some questions: This PR

Re: [PR] KAFKA-2443 Expose windowSize on Rate; KAFKA-2567 - Throttle time should not return NaN [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #213: URL: https://github.com/apache/kafka/pull/213#issuecomment-1997289295 This is a followup ticket from KAFKA-2084 to improve the windowSize calculation in Quotas. I've made the following changes: 1. Added a windowSize function on Rate 2. Calling

Re: [PR] KAFKA-2348; Drop support for Scala 2.9 [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #87: URL: https://github.com/apache/kafka/pull/87#issuecomment-1997289287 `testAll` passed locally. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] KAFKA-2514: change default JVM options in kafka-run-class.sh. [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #218: URL: https://github.com/apache/kafka/pull/218#issuecomment-1997289271 GC is set to G1 collector. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] KAFKA-1375: Fix formatting in README.md [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #24: URL: https://github.com/apache/kafka/pull/24#issuecomment-1997289253 This patch fixes formatting of instructions for using particular version of Scala when running a Gradle build task. -- This is an automated message from the Apache Git Service. To

Re: [PR] Remove non-functional variable definition in log4j.properties [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #36: URL: https://github.com/apache/kafka/pull/36#issuecomment-1997287446 In log4j.properties, a property kafka.logs.dir was defined. However, modifying this property has no effect because log4j.properties does not support variable substitution in this manner.

Re: [PR] KAFKA-1811 Ensuring registered broker host:port is unique [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #168: URL: https://github.com/apache/kafka/pull/168#issuecomment-1997289252 Adds a ZKLock recipe implementation to guarantee that the host:port pair is unique among the brokers registered on ZooKeeper. -- This is an automated message from the Apache Git

Re: [PR] KAFKA-2459: connection backoff, timeouts and retries [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #290: URL: https://github.com/apache/kafka/pull/290#issuecomment-1997289222 This fix applies to three JIRAs, since they are all connected. KAFKA-2459Connection backoff/blackout period should start when a connection is disconnected, not when the connection

Re: [PR] KAFKA-2558: ServerShutdownTest is failing intermittently [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #224: URL: https://github.com/apache/kafka/pull/224#issuecomment-1997289223 See jira for a description. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] Fix to run hadoop-consumer with hadoop 2 [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #18: URL: https://github.com/apache/kafka/pull/18#issuecomment-1997287426 Changes to use hadoop-core-1.2.1.jar in hadoop-consumer Changes to import from maven the packages needed to run hadoop-consumer in HDFS -- This is an automated message from the Apache

Re: [PR] KAFKA-2620: Introduce Scalariform [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #284: URL: https://github.com/apache/kafka/pull/284#issuecomment-1997289201 This is a WIP. There will likely need to be discussion around a rule set we want to use and if we actually want to include this in the build. This could also be used as a one time rebase.

Re: [PR] KAFKA-2555: Infinite recursive function call when call commitSync in … [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #221: URL: https://github.com/apache/kafka/pull/221#issuecomment-1997287384 @hachikuji @ewencp I found this problem when adding new consumer to mirror maker which commits offset in the rebalance callback. It is not clear to me why we are triggering rebalance for

Re: [PR] Rack aware replication [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #15: URL: https://github.com/apache/kafka/pull/15#issuecomment-1997289171 Adding rack-aware replication option. rack-id defaults to -1. use the max-rack-replication option when creating a topic to distribute replicas such that no more than max-rack-replication

Re: [PR] KAFKA-2098: gradle files [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #54: URL: https://github.com/apache/kafka/pull/54#issuecomment-1997287366 gradle files, tiny footprint.lets have it in.thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] KAFKA-2408 ConsoleConsumerService direct log output to file [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #123: URL: https://github.com/apache/kafka/pull/123#issuecomment-1997287362 console consumer writes to System.out, while (some) log4j loggers operate in other threads. This occasionally led to funky interleaved output which disrupted parsing of consumed

Re: [PR] KAFKA-2098: gradle files [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #54: URL: https://github.com/apache/kafka/pull/54#issuecomment-1997287349 gradle files, tiny footprint.lets have it in.thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] KAFKA-1371 Ignore build output dirs [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #21: URL: https://github.com/apache/kafka/pull/21#issuecomment-1997287316 This patch extends git ignore patterns to all build and .gradle dirs -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [PR] KAFKA-2588 ReplicaManager partitionCount metric should actually be re… [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #254: URL: https://github.com/apache/kafka/pull/254#issuecomment-1997287255 …plicaCount -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] MINOR: Set `sendTime` in `doSend` instead of `InFlightRequests.add` and rename method names for consistency [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #264: URL: https://github.com/apache/kafka/pull/264#issuecomment-1997287466 @hachikuji @MayureshGharat @jjkoshy Thoughts? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] KAFKA-2428 [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #282: URL: https://github.com/apache/kafka/pull/282#issuecomment-1997287443 Add sanity test in kafkaConsumer for the timeouts. This is a followup ticket for Kafka-2120. -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [PR] Remove non-functional variable definition in log4j.properties [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #36: URL: https://github.com/apache/kafka/pull/36#issuecomment-1997287454 In log4j.properties, a property kafka.logs.dir was defined. However, modifying this property has no effect because log4j.properties does not support variable substitution in this manner.

Re: [PR] commitOffsets can be passed the offsets to commit [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #10: URL: https://github.com/apache/kafka/pull/10#issuecomment-1997287252 This adds another version of `commitOffsets` that takes the offsets to commit as a parameter. Without this change, getting correct user code is very hard. Despite kafka's at-least-once

Re: [PR] Fix to run hadoop-consumer with hadoop 2 [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #18: URL: https://github.com/apache/kafka/pull/18#issuecomment-1997287431 Changes to use hadoop-core-1.2.1.jar in hadoop-consumer Changes to import from maven the packages needed to run hadoop-consumer in HDFS -- This is an automated message from the Apache

Re: [PR] KAFKA-2428 [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #282: URL: https://github.com/apache/kafka/pull/282#issuecomment-1997287420 Add sanity test in kafkaConsumer for the timeouts. This is a followup ticket for Kafka-2120. -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [PR] Modified the async producer so it re-queues failed batches. [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #7: URL: https://github.com/apache/kafka/pull/7#issuecomment-1997287224 I'm working on an application that needs the throughput offered by an async producer but also needs to handle send failures gracefully like a sync producer. I modified the ProducerSendThread so

Re: [PR] KAFKA-2555: Infinite recursive function call when call commitSync in … [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #221: URL: https://github.com/apache/kafka/pull/221#issuecomment-1997287374 @hachikuji @ewencp I found this problem when adding new consumer to mirror maker which commits offset in the rebalance callback. It is not clear to me why we are triggering rebalance for

Re: [PR] KAFKA-2408 ConsoleConsumerService direct log output to file [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #123: URL: https://github.com/apache/kafka/pull/123#issuecomment-1997287369 console consumer writes to System.out, while (some) log4j loggers operate in other threads. This occasionally led to funky interleaved output which disrupted parsing of consumed

Re: [PR] KAFKA-2072 [WIP]: Add StopReplica request/response to o.a.k.common.requests and replace the usage in core module [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #141: URL: https://github.com/apache/kafka/pull/141#issuecomment-1997287352 Migration is done but this PR will need to be rebased on #110. I have copied some code (ef669a5) for now. I'd appreciate feedback on it mainly around how I handle things in the

Re: [PR] commitOffsets can be passed the offsets to commit [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #10: URL: https://github.com/apache/kafka/pull/10#issuecomment-1997287258 This adds another version of `commitOffsets` that takes the offsets to commit as a parameter. Without this change, getting correct user code is very hard. Despite kafka's at-least-once

Re: [PR] KAFKA-2072 [WIP]: Add StopReplica request/response to o.a.k.common.requests and replace the usage in core module [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #141: URL: https://github.com/apache/kafka/pull/141#issuecomment-1997287356 Migration is done but this PR will need to be rebased on #110. I have copied some code (ef669a5) for now. I'd appreciate feedback on it mainly around how I handle things in the

Re: [PR] Small change to API doc for seekToEnd() to clarify lazy evaluation. [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #199: URL: https://github.com/apache/kafka/pull/199#issuecomment-1997287317 Small clarification to docs. Current behaviour could confuse when doing something like: consumer.seekToEnd() consumer.send(msg) consumer.poll() //would return msg as seek evaluates

Re: [PR] KAFKA-1371 Ignore build output dirs [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #21: URL: https://github.com/apache/kafka/pull/21#issuecomment-1997287320 This patch extends git ignore patterns to all build and .gradle dirs -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [PR] Small change to API doc for seekToEnd() to clarify lazy evaluation. [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #199: URL: https://github.com/apache/kafka/pull/199#issuecomment-1997287326 Small clarification to docs. Current behaviour could confuse when doing something like: consumer.seekToEnd() consumer.send(msg) consumer.poll() //would return msg as seek evaluates

Re: [PR] KAFKA-2588 ReplicaManager partitionCount metric should actually be re… [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #254: URL: https://github.com/apache/kafka/pull/254#issuecomment-1997287263 …plicaCount -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] Unmap before resizing [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #6: URL: https://github.com/apache/kafka/pull/6#issuecomment-1997287212 While I was studying how MappedByteBuffer works, I saw a sharing runtime exception on Windows. I applied what I learned to generate a patch which uses an internal open JDK API to solve this

Re: [PR] Unmap before resizing [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #6: URL: https://github.com/apache/kafka/pull/6#issuecomment-1997287204 While I was studying how MappedByteBuffer works, I saw a sharing runtime exception on Windows. I applied what I learned to generate a patch which uses an internal open JDK API to solve this

Re: [PR] Modified the async producer so it re-queues failed batches. [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #7: URL: https://github.com/apache/kafka/pull/7#issuecomment-1997287219 I'm working on an application that needs the throughput offered by an async producer but also needs to handle send failures gracefully like a sync producer. I modified the ProducerSendThread so

Re: [PR] KAFKA-948 : Update ReplicaStateMachine.scala [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #5: URL: https://github.com/apache/kafka/pull/5#issuecomment-1997287199 KAFKA-948 When the broker which is the leader for a partition is down, the ISR list in the LeaderAndISR path is updated. But if the broker , which is not a leader of the partition is down,

Re: [PR] zkclient and scalatest library updates [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #3: URL: https://github.com/apache/kafka/pull/3#issuecomment-1997287192 Following https://issues.apache.org/jira/browse/KAFKA-826 I forked the code and included fixes for 2 bugs I reported, https://issues.apache.org/jira/browse/KAFKA-807 and

Re: [PR] KAFKA-948 : Update ReplicaStateMachine.scala [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #5: URL: https://github.com/apache/kafka/pull/5#issuecomment-1997287201 KAFKA-948 When the broker which is the leader for a partition is down, the ISR list in the LeaderAndISR path is updated. But if the broker , which is not a leader of the partition is down,

Re: [PR] zkclient and scalatest library updates [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #3: URL: https://github.com/apache/kafka/pull/3#issuecomment-1997287184 Following https://issues.apache.org/jira/browse/KAFKA-826 I forked the code and included fixes for 2 bugs I reported, https://issues.apache.org/jira/browse/KAFKA-807 and

Re: [PR] Switch to using scala 2.9.2 [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #1: URL: https://github.com/apache/kafka/pull/1#issuecomment-1997287164 Compiled and used fine. I had issues with the tests though. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] KAFKA-294 [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #2: URL: https://github.com/apache/kafka/pull/2#issuecomment-1997287165 This issue can be caused by a non-existing path but also a misunderstanding from the config file. A short example will help the user. -- This is an automated message from the Apache Git

Re: [PR] Switch to using scala 2.9.2 [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #1: URL: https://github.com/apache/kafka/pull/1#issuecomment-1997287163 Compiled and used fine. I had issues with the tests though. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] Remove non-functional variable definition in log4j.properties [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #36: URL: https://github.com/apache/kafka/pull/36#issuecomment-1997285368 In log4j.properties, a property kafka.logs.dir was defined. However, modifying this property has no effect because log4j.properties does not support variable substitution in this manner.

Re: [PR] KAFKA-2428 [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #282: URL: https://github.com/apache/kafka/pull/282#issuecomment-1997285337 Add sanity test in kafkaConsumer for the timeouts. This is a followup ticket for Kafka-2120. -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [PR] KAFKA-294 [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #2: URL: https://github.com/apache/kafka/pull/2#issuecomment-1997287162 This issue can be caused by a non-existing path but also a misunderstanding from the config file. A short example will help the user. -- This is an automated message from the Apache Git

Re: [PR] Fix to run hadoop-consumer with hadoop 2 [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #18: URL: https://github.com/apache/kafka/pull/18#issuecomment-1997285328 Changes to use hadoop-core-1.2.1.jar in hadoop-consumer Changes to import from maven the packages needed to run hadoop-consumer in HDFS -- This is an automated message from the Apache

Re: [PR] KAFKA-2555: Infinite recursive function call when call commitSync in … [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #221: URL: https://github.com/apache/kafka/pull/221#issuecomment-1997285309 @hachikuji @ewencp I found this problem when adding new consumer to mirror maker which commits offset in the rebalance callback. It is not clear to me why we are triggering rebalance for

Re: [PR] KAFKA-2408 ConsoleConsumerService direct log output to file [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #123: URL: https://github.com/apache/kafka/pull/123#issuecomment-1997285301 console consumer writes to System.out, while (some) log4j loggers operate in other threads. This occasionally led to funky interleaved output which disrupted parsing of consumed

Re: [PR] KAFKA-2072 [WIP]: Add StopReplica request/response to o.a.k.common.requests and replace the usage in core module [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #141: URL: https://github.com/apache/kafka/pull/141#issuecomment-1997285287 Migration is done but this PR will need to be rebased on #110. I have copied some code (ef669a5) for now. I'd appreciate feedback on it mainly around how I handle things in the

Re: [PR] KAFKA-2408 ConsoleConsumerService direct log output to file [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #123: URL: https://github.com/apache/kafka/pull/123#issuecomment-1997285303 console consumer writes to System.out, while (some) log4j loggers operate in other threads. This occasionally led to funky interleaved output which disrupted parsing of consumed

Re: [PR] KAFKA-2098: gradle files [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #54: URL: https://github.com/apache/kafka/pull/54#issuecomment-1997285291 gradle files, tiny footprint.lets have it in.thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] MINOR: Set `sendTime` in `doSend` instead of `InFlightRequests.add` and rename method names for consistency [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #264: URL: https://github.com/apache/kafka/pull/264#issuecomment-1997285398 @hachikuji @MayureshGharat @jjkoshy Thoughts? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] KAFKA-2098: gradle files [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #54: URL: https://github.com/apache/kafka/pull/54#issuecomment-1997285293 gradle files, tiny footprint.lets have it in.thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] KAFKA-2479: Transient exception [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #279: URL: https://github.com/apache/kafka/pull/279#issuecomment-1997285376 Transient exception handling. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] KAFKA-1371 Ignore build output dirs [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #21: URL: https://github.com/apache/kafka/pull/21#issuecomment-1997285261 This patch extends git ignore patterns to all build and .gradle dirs -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [PR] Small change to API doc for seekToEnd() to clarify lazy evaluation. [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #199: URL: https://github.com/apache/kafka/pull/199#issuecomment-1997285248 Small clarification to docs. Current behaviour could confuse when doing something like: consumer.seekToEnd() consumer.send(msg) consumer.poll() //would return msg as seek evaluates

Re: [PR] Small change to API doc for seekToEnd() to clarify lazy evaluation. [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #199: URL: https://github.com/apache/kafka/pull/199#issuecomment-1997285266 Small clarification to docs. Current behaviour could confuse when doing something like: consumer.seekToEnd() consumer.send(msg) consumer.poll() //would return msg as seek evaluates

Re: [PR] MINOR: Set `sendTime` in `doSend` instead of `InFlightRequests.add` and rename method names for consistency [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #264: URL: https://github.com/apache/kafka/pull/264#issuecomment-1997285364 @hachikuji @MayureshGharat @jjkoshy Thoughts? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] commitOffsets can be passed the offsets to commit [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #10: URL: https://github.com/apache/kafka/pull/10#issuecomment-1997285210 This adds another version of `commitOffsets` that takes the offsets to commit as a parameter. Without this change, getting correct user code is very hard. Despite kafka's at-least-once

Re: [PR] KAFKA-2588 ReplicaManager partitionCount metric should actually be re… [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #254: URL: https://github.com/apache/kafka/pull/254#issuecomment-1997285213 …plicaCount -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] KAFKA-2428 [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #282: URL: https://github.com/apache/kafka/pull/282#issuecomment-1997285362 Add sanity test in kafkaConsumer for the timeouts. This is a followup ticket for Kafka-2120. -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [PR] Unmap before resizing [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #6: URL: https://github.com/apache/kafka/pull/6#issuecomment-1997285171 While I was studying how MappedByteBuffer works, I saw a sharing runtime exception on Windows. I applied what I learned to generate a patch which uses an internal open JDK API to solve this

Re: [PR] commitOffsets can be passed the offsets to commit [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #10: URL: https://github.com/apache/kafka/pull/10#issuecomment-1997285186 This adds another version of `commitOffsets` that takes the offsets to commit as a parameter. Without this change, getting correct user code is very hard. Despite kafka's at-least-once

Re: [PR] Remove non-functional variable definition in log4j.properties [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #36: URL: https://github.com/apache/kafka/pull/36#issuecomment-1997285343 In log4j.properties, a property kafka.logs.dir was defined. However, modifying this property has no effect because log4j.properties does not support variable substitution in this manner.

Re: [PR] Modified the async producer so it re-queues failed batches. [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #7: URL: https://github.com/apache/kafka/pull/7#issuecomment-1997285166 I'm working on an application that needs the throughput offered by an async producer but also needs to handle send failures gracefully like a sync producer. I modified the ProducerSendThread so

Re: [PR] KAFKA-948 : Update ReplicaStateMachine.scala [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #5: URL: https://github.com/apache/kafka/pull/5#issuecomment-1997285164 KAFKA-948 When the broker which is the leader for a partition is down, the ISR list in the LeaderAndISR path is updated. But if the broker , which is not a leader of the partition is down,

Re: [PR] Fix to run hadoop-consumer with hadoop 2 [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #18: URL: https://github.com/apache/kafka/pull/18#issuecomment-1997285322 Changes to use hadoop-core-1.2.1.jar in hadoop-consumer Changes to import from maven the packages needed to run hadoop-consumer in HDFS -- This is an automated message from the Apache

Re: [PR] zkclient and scalatest library updates [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #3: URL: https://github.com/apache/kafka/pull/3#issuecomment-1997285152 Following https://issues.apache.org/jira/browse/KAFKA-826 I forked the code and included fixes for 2 bugs I reported, https://issues.apache.org/jira/browse/KAFKA-807 and

Re: [PR] KAFKA-948 : Update ReplicaStateMachine.scala [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #5: URL: https://github.com/apache/kafka/pull/5#issuecomment-1997285151 KAFKA-948 When the broker which is the leader for a partition is down, the ISR list in the LeaderAndISR path is updated. But if the broker , which is not a leader of the partition is down,

Re: [PR] zkclient and scalatest library updates [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #3: URL: https://github.com/apache/kafka/pull/3#issuecomment-1997285135 Following https://issues.apache.org/jira/browse/KAFKA-826 I forked the code and included fixes for 2 bugs I reported, https://issues.apache.org/jira/browse/KAFKA-807 and

Re: [PR] KAFKA-294 [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #2: URL: https://github.com/apache/kafka/pull/2#issuecomment-1997285114 This issue can be caused by a non-existing path but also a misunderstanding from the config file. A short example will help the user. -- This is an automated message from the Apache Git

Re: [PR] Switch to using scala 2.9.2 [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #1: URL: https://github.com/apache/kafka/pull/1#issuecomment-1997285131 Compiled and used fine. I had issues with the tests though. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] Switch to using scala 2.9.2 [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #1: URL: https://github.com/apache/kafka/pull/1#issuecomment-1997285116 Compiled and used fine. I had issues with the tests though. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] KAFKA-2072 [WIP]: Add StopReplica request/response to o.a.k.common.requests and replace the usage in core module [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #141: URL: https://github.com/apache/kafka/pull/141#issuecomment-1997285285 Migration is done but this PR will need to be rebased on #110. I have copied some code (ef669a5) for now. I'd appreciate feedback on it mainly around how I handle things in the

Re: [PR] KAFKA-2555: Infinite recursive function call when call commitSync in … [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #221: URL: https://github.com/apache/kafka/pull/221#issuecomment-1997285307 @hachikuji @ewencp I found this problem when adding new consumer to mirror maker which commits offset in the rebalance callback. It is not clear to me why we are triggering rebalance for

Re: [PR] KAFKA-1414: Speedup broker shutdown and startup after hard reset [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #26: URL: https://github.com/apache/kafka/pull/26#issuecomment-1997283368 [KAFKA-1414](https://issues.apache.org/jira/browse/KAFKA-1414) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] KAFKA-2489: add benchmark for new consumer [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #179: URL: https://github.com/apache/kafka/pull/179#issuecomment-1997283366 @ewencp The changes here are smaller than they look - mostly refactoring/cleanup. - ConsumerPerformanceService: added new_consumer flag, and exposed more command-line settings -

Re: [PR] KAFKA-2344; kafka-merge-pr improvements [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #90: URL: https://github.com/apache/kafka/pull/90#issuecomment-1997283320 The first 4 commits are adapted from changes that have been done to the Spark version and the last one is the feature that @gwenshap asked for. -- This is an automated message from the

Re: [PR] Log the real exception which triggered a reconnect [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #138: URL: https://github.com/apache/kafka/pull/138#issuecomment-1997283344 The commit here improves the logging in SimpleConsumer to log the real reason why a reconnect was attempted. Relates to https://issues.apache.org/jira/browse/KAFKA-2221. The same

Re: [PR] KAFKA-2397: leave group request [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #103: URL: https://github.com/apache/kafka/pull/103#issuecomment-1997283328 Let's say every consumer in a group has session timeout s. Currently, if a consumer leaves the group, the worst case time to stabilize the group is 2s (s to detect the consumer failure + s

Re: [PR] KAFKA-2624: Change log message position [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #287: URL: https://github.com/apache/kafka/pull/287#issuecomment-1997283315 Log warning message before truncating log in order to display right offset value for the truncated log. -- This is an automated message from the Apache Git Service. To respond to the

Re: [PR] KAFKA-1371 Ignore build output dirs [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #21: URL: https://github.com/apache/kafka/pull/21#issuecomment-1997285247 This patch extends git ignore patterns to all build and .gradle dirs -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [PR] MINOR: Set `sendTime` in `doSend` instead of `InFlightRequests.add` and rename method names for consistency [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #264: URL: https://github.com/apache/kafka/pull/264#issuecomment-1997283305 @hachikuji @MayureshGharat @jjkoshy Thoughts? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] Remove non-functional variable definition in log4j.properties [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #36: URL: https://github.com/apache/kafka/pull/36#issuecomment-1997283292 In log4j.properties, a property kafka.logs.dir was defined. However, modifying this property has no effect because log4j.properties does not support variable substitution in this manner.

Re: [PR] KAFKA-2408 ConsoleConsumerService direct log output to file [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #123: URL: https://github.com/apache/kafka/pull/123#issuecomment-1997283218 console consumer writes to System.out, while (some) log4j loggers operate in other threads. This occasionally led to funky interleaved output which disrupted parsing of consumed

Re: [PR] KAFKA-2098: gradle files [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #54: URL: https://github.com/apache/kafka/pull/54#issuecomment-1997283209 gradle files, tiny footprint.lets have it in.thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] KAFKA-2072 [WIP]: Add StopReplica request/response to o.a.k.common.requests and replace the usage in core module [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #141: URL: https://github.com/apache/kafka/pull/141#issuecomment-1997283200 Migration is done but this PR will need to be rebased on #110. I have copied some code (ef669a5) for now. I'd appreciate feedback on it mainly around how I handle things in the

Re: [PR] KAFKA-2588 ReplicaManager partitionCount metric should actually be re… [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #254: URL: https://github.com/apache/kafka/pull/254#issuecomment-1997285193 …plicaCount -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] Modified the async producer so it re-queues failed batches. [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #7: URL: https://github.com/apache/kafka/pull/7#issuecomment-1997285184 I'm working on an application that needs the throughput offered by an async producer but also needs to handle send failures gracefully like a sync producer. I modified the ProducerSendThread so

Re: [PR] zkclient and scalatest library updates [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #3: URL: https://github.com/apache/kafka/pull/3#issuecomment-1997283074 Following https://issues.apache.org/jira/browse/KAFKA-826 I forked the code and included fixes for 2 bugs I reported, https://issues.apache.org/jira/browse/KAFKA-807 and

Re: [PR] Unmap before resizing [kafka]

2024-03-14 Thread via GitHub
b-goyal commented on PR #6: URL: https://github.com/apache/kafka/pull/6#issuecomment-1997285154 While I was studying how MappedByteBuffer works, I saw a sharing runtime exception on Windows. I applied what I learned to generate a patch which uses an internal open JDK API to solve this

<    1   2   3   4   >