Build failed in Jenkins: kafka-trunk-jdk8 #1056

See Changes: [becket.qin] KAFKA-4362; Consumer can fail after reassignment of the offsets topic -- [...truncated 14428 lines...] org.apache.kafka.streams.KafkaStreamsTest > shouldReturnFalseO

[jira] [Commented] (KAFKA-4430) Broker logging "Topic and partition to exceptions: [topic,6] -> kafka.common.MessageSizeTooLargeException"

[ https://issues.apache.org/jira/browse/KAFKA-4430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15689102#comment-15689102 ] huxi commented on KAFKA-4430: - A message larger than batch.size but smaller than max.request.s

Build failed in Jenkins: kafka-trunk-jdk7 #1706

See Changes: [becket.qin] KAFKA-4362; Consumer can fail after reassignment of the offsets topic -- [...truncated 14401 lines...] org.apache.kafka.streams.processor.internals.StreamsMetadataStat

[jira] [Commented] (KAFKA-1120) Controller could miss a broker state change

[ https://issues.apache.org/jira/browse/KAFKA-1120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15689028#comment-15689028 ] James Cheng commented on KAFKA-1120: No, because broker 1 was up and running by 22:58:

Re: Kafka consumers are not equally distributed

Could it be because of the partition key ? On Wed, Nov 23, 2016 at 12:33 AM, Ghosh, Achintya (Contractor) < achintya_gh...@comcast.com> wrote: > Hi there, > > We are doing the load test in Kafka with 25tps and first 9 hours it went > fine almost 80K/hr messages were processed after that we see a

[jira] [Commented] (KAFKA-4362) Consumer can fail after reassignment of the offsets topic partition

[ https://issues.apache.org/jira/browse/KAFKA-4362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1566#comment-1566 ] ASF GitHub Bot commented on KAFKA-4362: --- Github user asfgit closed the pull request

[GitHub] kafka pull request #2116: KAFKA-4362 : Consumer can fail after reassignment ...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2116 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Commented] (KAFKA-4430) Broker logging "Topic and partition to exceptions: [topic,6] -> kafka.common.MessageSizeTooLargeException"

[ https://issues.apache.org/jira/browse/KAFKA-4430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15688708#comment-15688708 ] Srinivas Dhruvakumar commented on KAFKA-4430: - Previous Configuration - . Kafk

[jira] [Created] (KAFKA-4433) Kafka Controller Does not send a LeaderAndIsr to old leader of a topicPartition during reassignment, if the old leader is not a part of the new assigned replicas

Mayuresh Gharat created KAFKA-4433: -- Summary: Kafka Controller Does not send a LeaderAndIsr to old leader of a topicPartition during reassignment, if the old leader is not a part of the new assigned replicas Key: KAFKA-4433

[jira] [Commented] (KAFKA-4430) Broker logging "Topic and partition to exceptions: [topic,6] -> kafka.common.MessageSizeTooLargeException"

[ https://issues.apache.org/jira/browse/KAFKA-4430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15688640#comment-15688640 ] Srinivas Dhruvakumar commented on KAFKA-4430: - Thanks will try the above steps

[jira] [Commented] (KAFKA-4430) Broker logging "Topic and partition to exceptions: [topic,6] -> kafka.common.MessageSizeTooLargeException"

[ https://issues.apache.org/jira/browse/KAFKA-4430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15688610#comment-15688610 ] huxi commented on KAFKA-4430: - Seems no debug statements can be enabled, but you could fire up

[jira] [Commented] (KAFKA-4021) system tests need to enable trace level logging for controller and state-change log

[ https://issues.apache.org/jira/browse/KAFKA-4021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15688546#comment-15688546 ] Jun Rao commented on KAFKA-4021: [~ewencp], those trace level logging only happens when th

[jira] [Comment Edited] (KAFKA-1120) Controller could miss a broker state change

[ https://issues.apache.org/jira/browse/KAFKA-1120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15325496#comment-15325496 ] Jun Rao edited comment on KAFKA-1120 at 11/23/16 1:03 AM: -- A bett

[jira] [Commented] (KAFKA-1120) Controller could miss a broker state change

[ https://issues.apache.org/jira/browse/KAFKA-1120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15688448#comment-15688448 ] Jun Rao commented on KAFKA-1120: [~wushujames], it seems that the controller did detect th

[jira] [Comment Edited] (KAFKA-4271) The console consumer fails on Windows with new consumer is used

[ https://issues.apache.org/jira/browse/KAFKA-4271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15684468#comment-15684468 ] Vahid Hashemian edited comment on KAFKA-4271 at 11/23/16 12:21 AM: -

[jira] [Updated] (KAFKA-2273) Add rebalance with a minimal number of reassignments to server-defined strategy list

[ https://issues.apache.org/jira/browse/KAFKA-2273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vahid Hashemian updated KAFKA-2273: --- Status: In Progress (was: Patch Available) KIP-54 voting in progress > Add rebalance with a

[jira] [Commented] (KAFKA-4392) Failed to lock the state directory due to an unexpected exception

[ https://issues.apache.org/jira/browse/KAFKA-4392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15688329#comment-15688329 ] ASF GitHub Bot commented on KAFKA-4392: --- GitHub user guozhangwang reopened a pull re

[jira] [Commented] (KAFKA-4392) Failed to lock the state directory due to an unexpected exception

[ https://issues.apache.org/jira/browse/KAFKA-4392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15688328#comment-15688328 ] ASF GitHub Bot commented on KAFKA-4392: --- Github user guozhangwang closed the pull re

[GitHub] kafka pull request #2121: KAFKA-4392: Fix race condition in directory cleanu...

GitHub user guozhangwang reopened a pull request: https://github.com/apache/kafka/pull/2121 KAFKA-4392: Fix race condition in directory cleanup You can merge this pull request into a Git repository by running: $ git pull https://github.com/guozhangwang/kafka K4392-race-dir-cle

[GitHub] kafka pull request #2121: KAFKA-4392: Fix race condition in directory cleanu...

Github user guozhangwang closed the pull request at: https://github.com/apache/kafka/pull/2121 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature i

[GitHub] kafka-site issue #32: Added section for Kafka Summit on the Events page

Github user guozhangwang commented on the issue: https://github.com/apache/kafka-site/pull/32 LGTM. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if t

[jira] [Updated] (KAFKA-4095) When a topic is deleted and then created with the same name, 'committed' offsets are not reset

[ https://issues.apache.org/jira/browse/KAFKA-4095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vahid Hashemian updated KAFKA-4095: --- Status: Patch Available (was: Open) > When a topic is deleted and then created with the same

[jira] [Commented] (KAFKA-1120) Controller could miss a broker state change

[ https://issues.apache.org/jira/browse/KAFKA-1120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15688295#comment-15688295 ] James Cheng commented on KAFKA-1120: I believe we ran into this today. {noformat} cor

[jira] [Commented] (KAFKA-4095) When a topic is deleted and then created with the same name, 'committed' offsets are not reset

[ https://issues.apache.org/jira/browse/KAFKA-4095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15688291#comment-15688291 ] ASF GitHub Bot commented on KAFKA-4095: --- GitHub user vahidhashemian opened a pull re

[GitHub] kafka pull request #2160: KAFKA-4095: Remove topic offsets and owners from Z...

GitHub user vahidhashemian opened a pull request: https://github.com/apache/kafka/pull/2160 KAFKA-4095: Remove topic offsets and owners from ZK consumer groups upon topic deletion You can merge this pull request into a Git repository by running: $ git pull https://github.com/

[jira] [Commented] (KAFKA-4273) Streams DSL - Add TTL / retention period support for intermediate topics and state stores

[ https://issues.apache.org/jira/browse/KAFKA-4273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15688015#comment-15688015 ] ASF GitHub Bot commented on KAFKA-4273: --- GitHub user dpoldrugo opened a pull request

[GitHub] kafka pull request #2159: KAFKA 4273 - Add TTL support for RocksDB

GitHub user dpoldrugo opened a pull request: https://github.com/apache/kafka/pull/2159 KAFKA 4273 - Add TTL support for RocksDB Since Streams DSL doesn't support fine grained configurations of state stores (it usese only RocksDB) - I have added new StreamsConfig called `rocksdb.tt

Jenkins build is back to normal : kafka-trunk-jdk7 #1705

See

[GitHub] kafka-site pull request #32: Added section for Kafka Summit on the Events pa...

GitHub user derrickdoo opened a pull request: https://github.com/apache/kafka-site/pull/32 Added section for Kafka Summit on the Events page Added section for Kafka Summit on the Events page You can merge this pull request into a Git repository by running: $ git pull https://gi

Build failed in Jenkins: kafka-trunk-jdk8 #1055

See Changes: [wangguoz] KAFKA-4355: Skip topics that have no partitions -- [...truncated 12149 lines...] org.apache.kafka.common.record.RecordTest > testChecksum[190] PASSED org.apache.kafka.co

Re: Restrict consumers from connecting to Kafka cluster

Check the Security section of the documentation, especially authorization (which means also authentication) http://kafka.apache.org/documentation.html Ofir Manor Co-Founder & CTO | Equalum Mobile: +972-54-7801286 | Email: ofir.ma...@equalum.io On Tue, Nov 22, 2016 at 9:13 PM, ravi singh wrote:

Restrict consumers from connecting to Kafka cluster

Is it possible to restrict Kafka consumers from consuming from a given Kafka cluster? -- *Regards,* *Ravi*

Kafka consumers are not equally distributed

Hi there, We are doing the load test in Kafka with 25tps and first 9 hours it went fine almost 80K/hr messages were processed after that we see a lot of lags and we stopped the incoming load. Currently we see 15K/hr messages are processing. We have 40 consumer instances with concurrency 4 and

Re: should HeartbeatThread be a daemon thread?

a similar issue exists with the old client stack as well - https://github.com/apache/kafka/pull/1930 On Mon, Nov 21, 2016 at 9:10 PM, Jason Gustafson wrote: > Hey David, > > It probably should be a daemon thread. Perhaps open a JIRA? > > Thanks, > Jason > > On Mon, Nov 21, 2016 at 2:03 PM, David

[jira] [Commented] (KAFKA-4322) StateRestoreCallback begin and end indication

[ https://issues.apache.org/jira/browse/KAFKA-4322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15687516#comment-15687516 ] Mark Shelton commented on KAFKA-4322: - The logging is the same for us and logging fram

[jira] [Comment Edited] (KAFKA-4322) StateRestoreCallback begin and end indication

[ https://issues.apache.org/jira/browse/KAFKA-4322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15687453#comment-15687453 ] Guozhang Wang edited comment on KAFKA-4322 at 11/22/16 6:09 PM:

[jira] [Commented] (KAFKA-4322) StateRestoreCallback begin and end indication

[ https://issues.apache.org/jira/browse/KAFKA-4322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15687453#comment-15687453 ] Guozhang Wang commented on KAFKA-4322: -- I see. Just for clarification for "own loggin

[GitHub] kafka pull request #2133: KAFKA-4355: Skip topics that have no partitions

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2133 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

Re: [DISCUSS] KIP-93: Improve invalid timestamp handling in Kafka Streams

Regarding the "compatibility" section, I would suggest being a bit more specific about why it is a breaking change. For Streams, it could mean different things: 1. User need code change when switching library dependency on the new version, otherwise it won't compile(I think this is the case for th

[jira] [Commented] (KAFKA-4355) StreamThread intermittently dies with "Topic not found during partition assignment" when broker restarted

[ https://issues.apache.org/jira/browse/KAFKA-4355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15687421#comment-15687421 ] Guozhang Wang commented on KAFKA-4355: -- Resolving the ticket as PR 2133 is merged. [~

[jira] [Resolved] (KAFKA-4355) StreamThread intermittently dies with "Topic not found during partition assignment" when broker restarted

[ https://issues.apache.org/jira/browse/KAFKA-4355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang resolved KAFKA-4355. -- Resolution: Fixed Fix Version/s: 0.10.2.0 > StreamThread intermittently dies with "Topic

[jira] [Commented] (KAFKA-4355) StreamThread intermittently dies with "Topic not found during partition assignment" when broker restarted

[ https://issues.apache.org/jira/browse/KAFKA-4355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15687416#comment-15687416 ] ASF GitHub Bot commented on KAFKA-4355: --- Github user asfgit closed the pull request

[jira] [Commented] (KAFKA-4432) ProducerPerformance.java : Add support to supply custom message payloads.

[ https://issues.apache.org/jira/browse/KAFKA-4432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15687106#comment-15687106 ] ASF GitHub Bot commented on KAFKA-4432: --- GitHub user SandeshKarkera opened a pull re

[jira] [Updated] (KAFKA-4432) ProducerPerformance.java : Add support to supply custom message payloads.

[ https://issues.apache.org/jira/browse/KAFKA-4432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandesh Karkera updated KAFKA-4432: --- Status: Patch Available (was: Open) PR raised: KAFKA-4432: Added support to supply custom mes

Re: [DISCUSS] KIP-87 - Add Compaction Tombstone Flag

Hi Mayuresh, LGTM. Ive just made one small adjustment updating the wire protocol to show the magic byte bump. Do we think we’re good to put to a vote? Is there any other bits needing discussion? Cheers Mike On 21/11/2016, 18:26, "Mayuresh Gharat" wrote: Hi Michael, I have updated t

[GitHub] kafka pull request #2158: KAFKA-4432: Added support to supply custom message...

GitHub user SandeshKarkera opened a pull request: https://github.com/apache/kafka/pull/2158 KAFKA-4432: Added support to supply custom message payloads to perf-producer script. Current implementation of ProducerPerformance creates static payload. This is not very useful in testing

Build failed in Jenkins: kafka-trunk-jdk7 #1704

See Changes: [ismael] MINOR: Typo in KafkaConsumer javadoc -- [...truncated 12141 lines...] org.apache.kafka.common.record.RecordTest > testChecksum[190] PASSED org.apache.kafka.common.record.R

[jira] [Commented] (KAFKA-4432) ProducerPerformance.java : Add support to supply custom message payloads.

[ https://issues.apache.org/jira/browse/KAFKA-4432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15686694#comment-15686694 ] Sandesh Karkera commented on KAFKA-4432: Note: I am working on this but don't have

[jira] [Updated] (KAFKA-4432) ProducerPerformance.java : Add support to supply custom message payloads.

[ https://issues.apache.org/jira/browse/KAFKA-4432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandesh Karkera updated KAFKA-4432: --- Fix Version/s: (was: 0.10.0.2) 0.10.2.0 > ProducerPerformance.java : Ad

[jira] [Commented] (KAFKA-3175) topic not accessible after deletion even when delete.topic.enable is disabled

[ https://issues.apache.org/jira/browse/KAFKA-3175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15686567#comment-15686567 ] Ismael Juma commented on KAFKA-3175: I think this was only merged to trunk, so updated

[jira] [Updated] (KAFKA-3175) topic not accessible after deletion even when delete.topic.enable is disabled

[ https://issues.apache.org/jira/browse/KAFKA-3175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-3175: --- Fix Version/s: (was: 0.10.1.1) 0.10.2.0 > topic not accessible after deletion e

[GitHub] kafka pull request #2157: Typo in Javadoc

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2157 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Commented] (KAFKA-4373) Kafka Consumer API jumping offsets

[ https://issues.apache.org/jira/browse/KAFKA-4373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15686394#comment-15686394 ] Conor Hughes commented on KAFKA-4373: - Thanks for such a the quick reply! After you m

[GitHub] kafka pull request #2157: Typo in Javadoc

GitHub user astubbs opened a pull request: https://github.com/apache/kafka/pull/2157 Typo in Javadoc You can merge this pull request into a Git repository by running: $ git pull https://github.com/astubbs/kafka trunk Alternatively you can review and apply these changes as the

[jira] [Updated] (KAFKA-4432) ProducerPerformance.java : Add support to supply custom message payloads.

[ https://issues.apache.org/jira/browse/KAFKA-4432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandesh Karkera updated KAFKA-4432: --- Description: Currently, kafka-producer-perf-test.sh does not support supplying custom payload

[jira] [Updated] (KAFKA-4432) ProducerPerformance.java : Add support to supply custom message payloads.

[ https://issues.apache.org/jira/browse/KAFKA-4432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandesh Karkera updated KAFKA-4432: --- Description: Currently, kafka-producer-perf-test.sh does not support supplying custom payload

[jira] [Created] (KAFKA-4432) ProducerPerformance.java : Add support to supply custom message payloads.

Sandesh Karkera created KAFKA-4432: -- Summary: ProducerPerformance.java : Add support to supply custom message payloads. Key: KAFKA-4432 URL: https://issues.apache.org/jira/browse/KAFKA-4432 Project:

[GitHub] kafka pull request #2154: HOTFIX: Increased wait time

Github user enothereska closed the pull request at: https://github.com/apache/kafka/pull/2154 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is