[GitHub] kafka pull request: KAFKA-2415: Fix transient failure in LogRecove...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/121 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Comment Edited] (KAFKA-2406) ISR propagation should be throttled to avoid overwhelming controller.

[ https://issues.apache.org/jira/browse/KAFKA-2406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14662766#comment-14662766 ] Jiangjie Qin edited comment on KAFKA-2406 at 8/8/15 2:54 AM: -

[jira] [Commented] (KAFKA-2406) ISR propagation should be throttled to avoid overwhelming controller.

[ https://issues.apache.org/jira/browse/KAFKA-2406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14662766#comment-14662766 ] Jiangjie Qin commented on KAFKA-2406: - [~ashishujjain] We are discussing whether to

[GitHub] kafka pull request: Minor: Fixes to Selector's javadoc

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/126 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Commented] (KAFKA-2413) New consumer's subscribe(Topic...) api fails if called more than once

[ https://issues.apache.org/jira/browse/KAFKA-2413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14662548#comment-14662548 ] ASF GitHub Bot commented on KAFKA-2413: --- Github user asfgit closed the pull request

[GitHub] kafka pull request: KAFKA-2413: fix ConsumerCoordinator updateCons...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/122 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

Re: [DISCUSSION] KIP-29 - Add an IsrPropagateIntervalMs config to KafkaConfig

Can you give some information on when I may want to tune this parameter? I'm wary of overwhelming admins with options and not giving any reasonable story on what they can / will achieve when modifying them. On Fri, Aug 7, 2015 at 9:29 AM, Jiangjie Qin j...@linkedin.com.invalid wrote: Hi, I

[jira] [Updated] (KAFKA-2408) (new) system tests: ConsoleConsumerService occasionally fails to register consumed message

[ https://issues.apache.org/jira/browse/KAFKA-2408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Geoffrey Anderson updated KAFKA-2408: - Reviewer: Ewen Cheslack-Postava Description: There have been a few spurious

[jira] [Commented] (KAFKA-2389) CommitType seems not necessary in commit().

[ https://issues.apache.org/jira/browse/KAFKA-2389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14662618#comment-14662618 ] Jiangjie Qin commented on KAFKA-2389: - [~gwenshap], we talked with [~ewencp] about

[jira] [Updated] (KAFKA-2413) New consumer's subscribe(Topic...) api fails if called more than once

[ https://issues.apache.org/jira/browse/KAFKA-2413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gwen Shapira updated KAFKA-2413: Resolution: Fixed Fix Version/s: 0.8.3 Status: Resolved (was: Patch Available)

[jira] [Commented] (KAFKA-2389) CommitType seems not necessary in commit().

[ https://issues.apache.org/jira/browse/KAFKA-2389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14662612#comment-14662612 ] Gwen Shapira commented on KAFKA-2389: - mmm... that actually sounds pretty reasonable

[jira] [Commented] (KAFKA-1835) Kafka new producer needs options to make blocking behavior explicit

[ https://issues.apache.org/jira/browse/KAFKA-1835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14662476#comment-14662476 ] Mayuresh Gharat commented on KAFKA-1835: [~stevenz3wu] KAFKA-2120 has max.block.ms

[jira] [Commented] (KAFKA-2411) remove usage of BlockingChannel in the broker

[ https://issues.apache.org/jira/browse/KAFKA-2411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14662532#comment-14662532 ] Gwen Shapira commented on KAFKA-2411: - [~ijuma] - ping me if you need help, I had lots

[jira] [Commented] (KAFKA-2389) CommitType seems not necessary in commit().

[ https://issues.apache.org/jira/browse/KAFKA-2389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14662543#comment-14662543 ] Jason Gustafson commented on KAFKA-2389: [~becket_qin] I'm definitely in favor of

[jira] [Commented] (KAFKA-2389) CommitType seems not necessary in commit().

[ https://issues.apache.org/jira/browse/KAFKA-2389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14662572#comment-14662572 ] Jason Gustafson commented on KAFKA-2389: [~gwenshap] This only reason we've

[jira] [Commented] (KAFKA-1835) Kafka new producer needs options to make blocking behavior explicit

[ https://issues.apache.org/jira/browse/KAFKA-1835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14662607#comment-14662607 ] Steven Zhen Wu commented on KAFKA-1835: --- [~mgharat] thanks for the clarification.

[jira] [Commented] (KAFKA-2389) CommitType seems not necessary in commit().

[ https://issues.apache.org/jira/browse/KAFKA-2389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14662624#comment-14662624 ] Gwen Shapira commented on KAFKA-2389: - ahhh, got it. CommitType seems not necessary

Re: [DISCUSSION] KIP-29 - Add an IsrPropagateIntervalMs config to KafkaConfig

Maybe Ashish can supply the use-case and tuning advice then :) I'm a -1 on adding new configurations that we can't quite explain. On Fri, Aug 7, 2015 at 3:57 PM, Jiangjie Qin j...@linkedin.com.invalid wrote: Hi Gwen, Completely agree with you. I originally just hard coded it to be 10

[jira] [Commented] (KAFKA-2408) (new) system tests: ConsoleConsumerService occasionally fails to register consumed message

[ https://issues.apache.org/jira/browse/KAFKA-2408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14661403#comment-14661403 ] ASF GitHub Bot commented on KAFKA-2408: --- GitHub user granders opened a pull request:

[jira] [Created] (KAFKA-2416) misleading kafka-console-consumer.sh --property option

Onur Karaman created KAFKA-2416: --- Summary: misleading kafka-console-consumer.sh --property option Key: KAFKA-2416 URL: https://issues.apache.org/jira/browse/KAFKA-2416 Project: Kafka Issue

[DISCUSSION] KIP-29 - Add an IsrPropagateIntervalMs config to KafkaConfig

Hi, I just created KIP-29 to add a new configuration of IsrPropagateIntervalMs to KafkaConfig. The motivation is to batch and throttle the ISR propagation in cases where large amount of ISR change occurs such as a broker gets bounced or failed. Otherwise it might overwhelm the controller.

[jira] [Commented] (KAFKA-2406) ISR propagation should be throttled to avoid overwhelming controller.

[ https://issues.apache.org/jira/browse/KAFKA-2406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14662059#comment-14662059 ] Jiangjie Qin commented on KAFKA-2406: - Just created KIP-29 for adding the config.

[jira] [Commented] (KAFKA-2071) Replace Produce Request/Response with their org.apache.kafka.common.requests equivalents

[ https://issues.apache.org/jira/browse/KAFKA-2071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14662025#comment-14662025 ] David Jacot commented on KAFKA-2071: I have looked at the producer and I think that

Re: Jira Permissions

Granted permission to both. Thanks, Jun On Fri, Aug 7, 2015 at 5:24 AM, Ben Stopford b...@confluent.io wrote: Hey - can I permission to take ownership of Jiras, and edit the wiki also? is there a process for this? Cheers B

[jira] [Commented] (KAFKA-2406) ISR propagation should be throttled to avoid overwhelming controller.

[ https://issues.apache.org/jira/browse/KAFKA-2406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14662018#comment-14662018 ] Jiangjie Qin commented on KAFKA-2406: - [~junrao] I submitted a new patch and it works

[jira] [Updated] (KAFKA-2415) Transient failure in LogRecoveryTest.testHWCheckpointWithFailuresMultipleLogSegments

[ https://issues.apache.org/jira/browse/KAFKA-2415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiangjie Qin updated KAFKA-2415: Status: Patch Available (was: Open) Transient failure in

Spooling support for kafka publishers !

Hi What are the best practises to achieve spooling support on producer end if the kafka cluster is not reachable or degraded. We are thinking to have wrapper on kafka producer which can spool locally if the producer cant talk to kafka cluster. Problem with this approach is, all web servers which

[GitHub] kafka pull request: MINOR: documentation fix in StringDecoder

GitHub user davecromberge opened a pull request: https://github.com/apache/kafka/pull/124 MINOR: documentation fix in StringDecoder You can merge this pull request into a Git repository by running: $ git pull https://github.com/davecromberge/kafka documentation-fix

[GitHub] kafka pull request: Minor: Fixes to Selector's javadoc

GitHub user ijuma opened a pull request: https://github.com/apache/kafka/pull/126 Minor: Fixes to Selector's javadoc You can merge this pull request into a Git repository by running: $ git pull https://github.com/ijuma/kafka minor-selector-javadoc-fixes Alternatively you can

[jira] [Commented] (KAFKA-2159) offsets.topic.segment.bytes and offsets.topic.retention.minutes are ignored

[ https://issues.apache.org/jira/browse/KAFKA-2159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14661949#comment-14661949 ] Manikumar Reddy commented on KAFKA-2159: [~jjkoshy] pinging for review

Re: Review Request 33049: Patch for KAFKA-2084

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33049/ --- (Updated Aug. 7, 2015, 6:28 p.m.) Review request for kafka, Joel Koshy and Jun

[jira] [Commented] (KAFKA-2084) byte rate metrics per client ID (producer and consumer)

[ https://issues.apache.org/jira/browse/KAFKA-2084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14662219#comment-14662219 ] Aditya A Auradkar commented on KAFKA-2084: -- Updated reviewboard

Re: [DISCUSSION] KIP-29 - Add an IsrPropagateIntervalMs config to KafkaConfig

Would it be possible to take a group commit approach rather than configuring a delay. I.e. all changes that come in at once while another operation is occurring automatically get batched. This can sometimes be nicer and more automatic than another tunable parameter... -Jay On Fri, Aug 7, 2015 at

[jira] [Updated] (KAFKA-2406) ISR propagation should be throttled to avoid overwhelming controller.

[ https://issues.apache.org/jira/browse/KAFKA-2406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiangjie Qin updated KAFKA-2406: Status: Patch Available (was: Open) ISR propagation should be throttled to avoid overwhelming

Re: Review Request 33049: Patch for KAFKA-2084

On Aug. 6, 2015, 2:02 a.m., Jun Rao wrote: core/src/main/scala/kafka/server/ClientQuotaManager.scala, lines 208-211 https://reviews.apache.org/r/33049/diff/18/?file=1032147#file1032147line208 Instead of creating a new metric instance, we probably should just reuse the same metric

[jira] [Commented] (KAFKA-1006) Consumer loses messages of a new topic with auto.offset.reset = largest

[ https://issues.apache.org/jira/browse/KAFKA-1006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14662154#comment-14662154 ] Eli Naeher commented on KAFKA-1006: --- This issue has caused us problems trying to write

Re: [DISCUSSION] KIP-29 - Add an IsrPropagateIntervalMs config to KafkaConfig

Hi Jay, I'm not sure I understand the group commit. Can you elaborate a little bit? The ISR change are actually on a per partition base. When a broker fails and come back, there could be thousands of ISR changes in say 30 seconds. In that case how would you group them? In terms of ISR shrinking,

[jira] [Updated] (KAFKA-2084) byte rate metrics per client ID (producer and consumer)

[ https://issues.apache.org/jira/browse/KAFKA-2084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aditya A Auradkar updated KAFKA-2084: - Attachment: KAFKA-2084_2015-08-07_11:27:51.patch byte rate metrics per client ID

[jira] [Updated] (KAFKA-2413) New consumer's subscribe(Topic...) api fails if called more than once

[ https://issues.apache.org/jira/browse/KAFKA-2413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Onur Karaman updated KAFKA-2413: Status: Patch Available (was: Open) New consumer's subscribe(Topic...) api fails if called more

[jira] [Assigned] (KAFKA-2413) New consumer's subscribe(Topic...) api fails if called more than once

[ https://issues.apache.org/jira/browse/KAFKA-2413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Onur Karaman reassigned KAFKA-2413: --- Assignee: Onur Karaman (was: Ashish K Singh) New consumer's subscribe(Topic...) api fails

Re: Review Request 33049: Patch for KAFKA-2084

On Aug. 6, 2015, 4:17 p.m., Jun Rao wrote: core/src/main/scala/kafka/server/ClientQuotaManager.scala, lines 154-201 https://reviews.apache.org/r/33049/diff/18/?file=1032147#file1032147line154 Not sure why the lock is needed. metrics.sensor() is synchronized and alreayd does the

Re: Review Request 33049: Patch for KAFKA-2084

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33049/ --- (Updated Aug. 7, 2015, 6:27 p.m.) Review request for kafka, Joel Koshy and Jun

[GitHub] kafka pull request: Reducing code duplication in DefaultEventHandl...

GitHub user lazyval opened a pull request: https://github.com/apache/kafka/pull/125 Reducing code duplication in DefaultEventHandler No functionality was changed -- just removed code duplication [get advantage of case class copy

[jira] [Commented] (KAFKA-1835) Kafka new producer needs options to make blocking behavior explicit

[ https://issues.apache.org/jira/browse/KAFKA-1835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14662427#comment-14662427 ] Steven Zhen Wu commented on KAFKA-1835: --- [~junrao] I think this is a different