[jira] [Commented] (KAFKA-2758) Improve Offset Commit Behavior

[ https://issues.apache.org/jira/browse/KAFKA-2758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15095678#comment-15095678 ] David Jacot commented on KAFKA-2758: [~guozhang] I'd like to dive into the new consume

[jira] [Resolved] (KAFKA-2843) when consumer got empty messageset, fetchResponse.highWatermark != current_offset?

[ https://issues.apache.org/jira/browse/KAFKA-2843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang resolved KAFKA-2843. -- Resolution: Not A Problem > when consumer got empty messageset, fetchResponse.highWatermark !=

[GitHub] kafka pull request: KAFKA-3091: Broker with an invalid id would no...

GitHub user granthenke opened a pull request: https://github.com/apache/kafka/pull/763 KAFKA-3091: Broker with an invalid id would not start when its id is … …updated to a new valid one You can merge this pull request into a Git repository by running: $ git pull https://git

[jira] [Updated] (KAFKA-3091) Broker with an invalid id would not start when its id is updated to a new valid one

[ https://issues.apache.org/jira/browse/KAFKA-3091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Grant Henke updated KAFKA-3091: --- Status: Patch Available (was: Open) > Broker with an invalid id would not start when its id is update

[jira] [Commented] (KAFKA-3091) Broker with an invalid id would not start when its id is updated to a new valid one

[ https://issues.apache.org/jira/browse/KAFKA-3091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15095644#comment-15095644 ] ASF GitHub Bot commented on KAFKA-3091: --- GitHub user granthenke opened a pull reques

[jira] [Commented] (KAFKA-3012) Avoid reserved.broker.max.id collisions on upgrade

[ https://issues.apache.org/jira/browse/KAFKA-3012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15095543#comment-15095543 ] Grant Henke commented on KAFKA-3012: Through discussion [here|http://search-hadoop.co

[jira] [Updated] (KAFKA-3012) Avoid reserved.broker.max.id collisions on upgrade

[ https://issues.apache.org/jira/browse/KAFKA-3012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Grant Henke updated KAFKA-3012: --- Status: Patch Available (was: Open) > Avoid reserved.broker.max.id collisions on upgrade > --

[jira] [Commented] (KAFKA-3012) Avoid reserved.broker.max.id collisions on upgrade

[ https://issues.apache.org/jira/browse/KAFKA-3012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15095535#comment-15095535 ] ASF GitHub Bot commented on KAFKA-3012: --- GitHub user granthenke opened a pull reques

[GitHub] kafka pull request: KAFKA-3012: Avoid reserved.broker.max.id colli...

GitHub user granthenke opened a pull request: https://github.com/apache/kafka/pull/762 KAFKA-3012: Avoid reserved.broker.max.id collisions on upgrade Provides a configuration to opt out of broker id generation. You can merge this pull request into a Git repository by running: $

[jira] [Commented] (KAFKA-3081) KTable Aggregation Implementation

[ https://issues.apache.org/jira/browse/KAFKA-3081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15095499#comment-15095499 ] ASF GitHub Bot commented on KAFKA-3081: --- GitHub user guozhangwang opened a pull requ

[GitHub] kafka pull request: KAFKA-3081: Non-windowed Table Aggregation

GitHub user guozhangwang opened a pull request: https://github.com/apache/kafka/pull/761 KAFKA-3081: Non-windowed Table Aggregation You can merge this pull request into a Git repository by running: $ git pull https://github.com/guozhangwang/kafka K3081 Alternatively you can r

[jira] [Assigned] (KAFKA-3083) a soft failure in controller may leader a topic partition in an inconsistent state

[ https://issues.apache.org/jira/browse/KAFKA-3083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mayuresh Gharat reassigned KAFKA-3083: -- Assignee: Mayuresh Gharat > a soft failure in controller may leader a topic partition i

[jira] [Commented] (KAFKA-3083) a soft failure in controller may leader a topic partition in an inconsistent state

[ https://issues.apache.org/jira/browse/KAFKA-3083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15095495#comment-15095495 ] Jun Rao commented on KAFKA-3083: This can potentially lead to more serious issues. For exa

Re: [DISCUSS] KIP-36 - Rack aware replica assignment

Allen, It's not ideal to add a new field in json without increasing the version. Also, if we don't fix this issue in 0.9.0, if we ever change the version of json in the future, the consumer in 0.9.0 will break after the broker is upgraded to the new release. So, I suggest that we fix the behavior

Jenkins build is back to normal : kafka-trunk-jdk7 #958

See

[GitHub] kafka pull request: [kafka-1828]add judgement for ConsumerPerforma...

Github user maji2014 closed the pull request at: https://github.com/apache/kafka/pull/38 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Commented] (KAFKA-3092) Rename SinkTask.onPartitionsAssigned/onPartitionsRevoked and Clarify Contract

[ https://issues.apache.org/jira/browse/KAFKA-3092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15095445#comment-15095445 ] Ewen Cheslack-Postava commented on KAFKA-3092: -- Yes, those are exactly the se

Re: [DISCUSS] KIP-36 - Rack aware replica assignment

Agreed. So it seems that for 0.9.1, the only option is to keep the JSON version unchanged. But as part of the PR, I can change the behavior of ZkUtils.getBrokerInfo() to make it compatible with future JSON versions. Thanks, Allen On Tue, Jan 12, 2016 at 2:57 PM, Jun Rao wrote: > Hi, Allen, > >

[jira] [Commented] (KAFKA-3092) Rename SinkTask.onPartitionsAssigned/onPartitionsRevoked and Clarify Contract

[ https://issues.apache.org/jira/browse/KAFKA-3092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15095413#comment-15095413 ] Jason Gustafson commented on KAFKA-3092: [~ewencp] Yeah, that makes sense. Just to

Build failed in Jenkins: kafka-trunk-jdk8 #285

See Changes: [me] KAFKA-3063; LogRecoveryTest causes JVM to exit occasionally -- Started by an SCM change [EnvInject] - Loading node environment variables. Building remotely on H11 (Ubuntu ubuntu

Re: KIP-41: KafkaConsumer Max Records

Hey Aarti, Sorry for the late response. Your help with memory management would be great! Since this does seem to be one of the major outstanding issues for the consumer, I was going to start looking at it in detail. I found that Jay previously filed KAFKA-2063, which basically proposes to add a ma

[GitHub] kafka pull request: POC Producer Interceptor and simple C3 impleme...

Github user apovzner closed the pull request at: https://github.com/apache/kafka/pull/760 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] kafka pull request: POC Producer Interceptor and simple C3 impleme...

GitHub user apovzner reopened a pull request: https://github.com/apache/kafka/pull/760 POC Producer Interceptor and simple C3 implementation of producer interceptor. You can merge this pull request into a Git repository by running: $ git pull https://github.com/confluentinc/k

[GitHub] kafka pull request: POC Producer Interceptor and simple C3 impleme...

Github user apovzner closed the pull request at: https://github.com/apache/kafka/pull/760 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] kafka pull request: POC Producer Interceptor and simple C3 impleme...

GitHub user apovzner opened a pull request: https://github.com/apache/kafka/pull/760 POC Producer Interceptor and simple C3 implementation of producer interceptor. PR is for code review only (pre-KIP POC code). What's missing: 1. We will specify all interceptor classes

[jira] [Commented] (KAFKA-3092) Rename SinkTask.onPartitionsAssigned/onPartitionsRevoked and Clarify Contract

[ https://issues.apache.org/jira/browse/KAFKA-3092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15095314#comment-15095314 ] Ewen Cheslack-Postava commented on KAFKA-3092: -- The change to rename to open/

[jira] [Commented] (KAFKA-3079) org.apache.kafka.common.KafkaException: java.lang.SecurityException: Configuration Error:

[ https://issues.apache.org/jira/browse/KAFKA-3079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15095292#comment-15095292 ] Mohit Anchlia commented on KAFKA-3079: -- I followed this doc: http://docs.confluent.i

[jira] [Created] (KAFKA-3092) Rename SinkTask.onPartitionsAssigned/onPartitionsRevoked and Clarify Contract

Jason Gustafson created KAFKA-3092: -- Summary: Rename SinkTask.onPartitionsAssigned/onPartitionsRevoked and Clarify Contract Key: KAFKA-3092 URL: https://issues.apache.org/jira/browse/KAFKA-3092 Proje

[jira] [Commented] (KAFKA-3063) LogRecoveryTest exits with -1 occasionally

[ https://issues.apache.org/jira/browse/KAFKA-3063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15095278#comment-15095278 ] ASF GitHub Bot commented on KAFKA-3063: --- Github user asfgit closed the pull request

[jira] [Updated] (KAFKA-3063) LogRecoveryTest exits with -1 occasionally

[ https://issues.apache.org/jira/browse/KAFKA-3063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava updated KAFKA-3063: - Resolution: Fixed Status: Resolved (was: Patch Available) Issue resolved

[GitHub] kafka pull request: KAFKA-3063; LogRecoveryTest causes JVM to exit...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/759 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabl

[jira] [Updated] (KAFKA-3012) Avoid reserved.broker.max.id collisions on upgrade

[ https://issues.apache.org/jira/browse/KAFKA-3012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Grant Henke updated KAFKA-3012: --- Summary: Avoid reserved.broker.max.id collisions on upgrade (was: Reduce reserved.broker.max.id colli

Re: [DISCUSS] KIP-36 - Rack aware replica assignment

Hi, Allen, That's a good point. In 0.9.0.0, the old consumer reads broker info directly from ZK and the code throws an exception if the version in json is not 1 or 2. This old consumer will break when we upgrade the broker json to version 3 in ZK in 0.9.1, which will be an issue. We overlooked thi

[jira] [Assigned] (KAFKA-3091) Broker with an invalid id would not start when its id is updated to a new valid one

[ https://issues.apache.org/jira/browse/KAFKA-3091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Grant Henke reassigned KAFKA-3091: -- Assignee: Grant Henke > Broker with an invalid id would not start when its id is updated to a n

[jira] [Commented] (KAFKA-3091) Broker with an invalid id would not start when its id is updated to a new valid one

[ https://issues.apache.org/jira/browse/KAFKA-3091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15095154#comment-15095154 ] Grant Henke commented on KAFKA-3091: I can look at this since its related to KAFKA-301

[jira] [Updated] (KAFKA-3063) LogRecoveryTest exits with -1 occasionally

[ https://issues.apache.org/jira/browse/KAFKA-3063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-3063: --- Status: Patch Available (was: In Progress) > LogRecoveryTest exits with -1 occasionally > ---

[jira] [Commented] (KAFKA-3063) LogRecoveryTest exits with -1 occasionally

[ https://issues.apache.org/jira/browse/KAFKA-3063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15095123#comment-15095123 ] ASF GitHub Bot commented on KAFKA-3063: --- GitHub user ijuma opened a pull request:

[GitHub] kafka pull request: KAFKA-3063; LogRecoveryTest causes JVM to exit...

GitHub user ijuma opened a pull request: https://github.com/apache/kafka/pull/759 KAFKA-3063; LogRecoveryTest causes JVM to exit occasionally Remove deletion of tmp file in `OffsetCheckpoint`'s constructor. This delete causes unintuitive behaviour like `LogRecoveryTest` causing a `

[jira] [Commented] (KAFKA-3079) org.apache.kafka.common.KafkaException: java.lang.SecurityException: Configuration Error:

[ https://issues.apache.org/jira/browse/KAFKA-3079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15095091#comment-15095091 ] Ismael Juma commented on KAFKA-3079: You have configured the ZK client connection to u

[jira] [Updated] (KAFKA-3091) Broker with an invalid id would not start when its id is updated to a new valid one

[ https://issues.apache.org/jira/browse/KAFKA-3091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vahid Hashemian updated KAFKA-3091: --- Description: Setup being used: - Kafka 0.9.0 - Two servers: -- server 1: zookeeper and 3 broke

[jira] [Updated] (KAFKA-3091) Broker with an invalid id would not start when its id is updated to a new valid one

[ https://issues.apache.org/jira/browse/KAFKA-3091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vahid Hashemian updated KAFKA-3091: --- Component/s: (was: core) > Broker with an invalid id would not start when its id is update

[jira] [Created] (KAFKA-3091) Broker with an invalid id would not start when its id is updated to a new valid one

Vahid Hashemian created KAFKA-3091: -- Summary: Broker with an invalid id would not start when its id is updated to a new valid one Key: KAFKA-3091 URL: https://issues.apache.org/jira/browse/KAFKA-3091

Re: [DISCUSS] KIP-36 - Rack aware replica assignment

Hi Jun, That's a good suggestion. However, it does not solve the problem for the clients or thirty party tools that get broker information directly from ZooKeeper. Thanks, Allen On Tue, Jan 12, 2016 at 1:29 PM, Jun Rao wrote: > Allen, > > Another way to do this is the following. > > When inte

Re: [DISCUSS] KIP-36 - Rack aware replica assignment

Allen, Another way to do this is the following. When inter.broker.protocol.version is set to 0.9.0, the broker will write the broker info in ZK using version 2, ignoring the rack info. When inter.broker.protocol.version is set to 0.9.1, the broker will write the broker info in ZK using version 3

[jira] [Commented] (KAFKA-2843) when consumer got empty messageset, fetchResponse.highWatermark != current_offset?

[ https://issues.apache.org/jira/browse/KAFKA-2843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15094983#comment-15094983 ] Guozhang Wang commented on KAFKA-2843: -- I looked through the code again: HW is stored

Re: [DISCUSS] KIP-36 - Rack aware replica assignment

Regarding the JSON version of Broker: I don't why the ZkUtils.getBrokerInfo() restricts the JSON versions it can read. It will throw exception if version is not 1 or 2. Seems to me that it will cause compatibility problem whenever the version needs to be changed and make the upgrade path difficult

Re: Kafka KIP meeting Jan. 12 at 11:00am PST

The following are the notes from today's KIP discussion. - KIP-41: Discussed whether the issue of long processing time between poll calls is a common issue and whether we should revisit the poll api. Also discussed whether the number of records returned in poll calls can be made more

[jira] [Created] (KAFKA-3090) Zookeeper disconnects with "can't find default realm" message

Mohit Anchlia created KAFKA-3090: Summary: Zookeeper disconnects with "can't find default realm" message Key: KAFKA-3090 URL: https://issues.apache.org/jira/browse/KAFKA-3090 Project: Kafka

[jira] [Commented] (KAFKA-3079) org.apache.kafka.common.KafkaException: java.lang.SecurityException: Configuration Error:

[ https://issues.apache.org/jira/browse/KAFKA-3079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15094642#comment-15094642 ] Mohit Anchlia commented on KAFKA-3079: -- It does make a progress and I don't get parse

Build failed in Jenkins: kafka-trunk-jdk7 #957

See Changes: [me] KAFKA-3069: Fix recursion in ZkSecurityMigrator -- [...truncated 1364 lines...] kafka.server.OffsetCommitTest > testOffsetExpiration PASSED kafka.server.OffsetCommitTest > tes

Build failed in Jenkins: kafka-trunk-jdk8 #284

See Changes: [me] KAFKA-3069: Fix recursion in ZkSecurityMigrator -- [...truncated 2008 lines...] java.lang.NullPointerException at hudson.plugins.toolenv.ToolEnvBuildWrapper$1.buildEnvV

[GitHub] kafka-site pull request: MINOR: Update consumer_config, kafka_conf...

Github user asfgit closed the pull request at: https://github.com/apache/kafka-site/pull/8 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is en

[GitHub] kafka-site pull request: MINOR: Update consumer_config, kafka_conf...

Github user ewencp commented on the pull request: https://github.com/apache/kafka-site/pull/8#issuecomment-171009732 LGTM, thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[jira] [Commented] (KAFKA-3069) Fix recursion in ZkSecurityMigrator

[ https://issues.apache.org/jira/browse/KAFKA-3069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15094375#comment-15094375 ] ASF GitHub Bot commented on KAFKA-3069: --- Github user asfgit closed the pull request

[GitHub] kafka pull request: KAFKA-3069: Fix recursion in ZkSecurityMigrato...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/736 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabl

[jira] [Resolved] (KAFKA-3069) Fix recursion in ZkSecurityMigrator

[ https://issues.apache.org/jira/browse/KAFKA-3069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava resolved KAFKA-3069. -- Resolution: Fixed Fix Version/s: (was: 0.9.0.1) 0.9

[jira] [Assigned] (KAFKA-2066) Replace FetchRequest / FetchResponse with their org.apache.kafka.common.requests equivalents

[ https://issues.apache.org/jira/browse/KAFKA-2066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Jacot reassigned KAFKA-2066: -- Assignee: David Jacot > Replace FetchRequest / FetchResponse with their > org.apache.kafka.com

[jira] [Commented] (KAFKA-3086) unused handle method in MirrorMakerMessageHandler

[ https://issues.apache.org/jira/browse/KAFKA-3086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15093843#comment-15093843 ] ASF GitHub Bot commented on KAFKA-3086: --- GitHub user Mszak opened a pull request:

[jira] [Updated] (KAFKA-3086) unused handle method in MirrorMakerMessageHandler

[ https://issues.apache.org/jira/browse/KAFKA-3086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakub Nowak updated KAFKA-3086: --- Status: Patch Available (was: Open) > unused handle method in MirrorMakerMessageHandler > ---

[GitHub] kafka pull request: KAFKA-3086: Remove unused method.

GitHub user Mszak opened a pull request: https://github.com/apache/kafka/pull/758 KAFKA-3086: Remove unused method. You can merge this pull request into a Git repository by running: $ git pull https://github.com/Mszak/kafka kafka-3086 Alternatively you can review and apply th

[jira] [Updated] (KAFKA-3086) unused handle method in MirrorMakerMessageHandler

[ https://issues.apache.org/jira/browse/KAFKA-3086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakub Nowak updated KAFKA-3086: --- Assignee: Jakub Nowak > unused handle method in MirrorMakerMessageHandler > --

[GitHub] kafka pull request: KAFKA-3082: Make LogManager.InitialTaskDelayMs...

GitHub user Mszak opened a pull request: https://github.com/apache/kafka/pull/757 KAFKA-3082: Make LogManager.InitialTaskDelayMs configurable You can merge this pull request into a Git repository by running: $ git pull https://github.com/Mszak/kafka kafka-3082 Alternatively y

[jira] [Updated] (KAFKA-3082) Make LogManager.InitialTaskDelayMs configurable

[ https://issues.apache.org/jira/browse/KAFKA-3082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakub Nowak updated KAFKA-3082: --- Status: Patch Available (was: Open) > Make LogManager.InitialTaskDelayMs configurable > -

[jira] [Commented] (KAFKA-3082) Make LogManager.InitialTaskDelayMs configurable

[ https://issues.apache.org/jira/browse/KAFKA-3082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15093826#comment-15093826 ] ASF GitHub Bot commented on KAFKA-3082: --- GitHub user Mszak opened a pull request:

[jira] [Updated] (KAFKA-3082) Make LogManager.InitialTaskDelayMs configurable

[ https://issues.apache.org/jira/browse/KAFKA-3082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakub Nowak updated KAFKA-3082: --- Assignee: Jakub Nowak > Make LogManager.InitialTaskDelayMs configurable >

Build failed in Jenkins: kafka-trunk-jdk8 #283

See Changes: [me] KAFKA-3077: Enable KafkaLog4jAppender to work with SASL enabled brokers [me] KAFKA-3078: Add ducktape tests for KafkaLog4jAppender producing to SASL -- Started by an SCM change