Re: Kafka consumer getting duplicate message

Achintya, 1.0.0.M2 is not an official release, so this version number is not particularly meaningful to people on this list. What platform/distribution are you using and how does this map to actual Apache Kafka releases? In general, it is not possible for any system to guarantee exactly once sema

Re: [kafka-clients] [VOTE] 0.10.0.1 RC2

+1 (non-binding). verified quick start and artifacts. On Sat, Aug 6, 2016 at 5:45 AM, Joel Koshy wrote: > +1 (binding) > > Thanks Ismael! > > On Thu, Aug 4, 2016 at 6:54 AM, Ismael Juma wrote: > >> Hello Kafka users, developers and client-developers, >> >> This is the third candidate for the re

Build failed in Jenkins: kafka-0.10.0-jdk7 #184

See -- [...truncated 5720 lines...] org.apache.kafka.streams.kstream.internals.KStreamForeachTest > testForeach PASSED org.apache.kafka.streams.kstream.internals.KTableMapValuesTest > testSendingOldVa

Re: [kafka-clients] [VOTE] 0.10.0.1 RC2

+1 (binding) Thanks Ismael! On Thu, Aug 4, 2016 at 6:54 AM, Ismael Juma wrote: > Hello Kafka users, developers and client-developers, > > This is the third candidate for the release of Apache Kafka 0.10.0.1. This > is a bug fix release and it includes fixes and improvements from 53 JIRAs > (inc

[jira] [Commented] (KAFKA-3809) Auto-generate documentation for topic-level configuration

[ https://issues.apache.org/jira/browse/KAFKA-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15410293#comment-15410293 ] ASF GitHub Bot commented on KAFKA-3809: --- GitHub user wushujames reopened a pull requ

[jira] [Commented] (KAFKA-3809) Auto-generate documentation for topic-level configuration

[ https://issues.apache.org/jira/browse/KAFKA-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15410292#comment-15410292 ] ASF GitHub Bot commented on KAFKA-3809: --- Github user wushujames closed the pull requ

[GitHub] kafka pull request #1527: KAFKA-3809: Auto-generate documentation for topic-...

GitHub user wushujames reopened a pull request: https://github.com/apache/kafka/pull/1527 KAFKA-3809: Auto-generate documentation for topic-level configuration @ijuma said that it would make sense to split out this work from KAFKA-3234, since KAFKA-3234 had both a mechanical change

[GitHub] kafka pull request #1527: KAFKA-3809: Auto-generate documentation for topic-...

Github user wushujames closed the pull request at: https://github.com/apache/kafka/pull/1527 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Commented] (KAFKA-3971) Consumers drop from coordinator and cannot reconnet

[ https://issues.apache.org/jira/browse/KAFKA-3971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15410285#comment-15410285 ] Jason Gustafson commented on KAFKA-3971: [~wonlay] All consumer instances sharing

Re: [VOTE] 0.10.0.1 RC2

+1 (non-binding) Tried the build and ran the quick start successfully on Ubuntu, Mac, and Windows. --Vahid From: Ismael Juma To: us...@kafka.apache.org, dev@kafka.apache.org, kafka-clients Date: 08/04/2016 06:55 AM Subject:[VOTE] 0.10.0.1 RC2 Sent by:isma...@gmail

[jira] [Commented] (KAFKA-3665) Default ssl.endpoint.identification.algorithm should be https

[ https://issues.apache.org/jira/browse/KAFKA-3665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15410268#comment-15410268 ] Jun Rao commented on KAFKA-3665: [~Ryan P], thanks for the explanation. In the common case

[jira] [Commented] (KAFKA-3971) Consumers drop from coordinator and cannot reconnet

[ https://issues.apache.org/jira/browse/KAFKA-3971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15410199#comment-15410199 ] Lei Wang commented on KAFKA-3971: - yes, that's right. > Consumers drop from coordinator a

[jira] [Commented] (KAFKA-3971) Consumers drop from coordinator and cannot reconnet

[ https://issues.apache.org/jira/browse/KAFKA-3971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15410194#comment-15410194 ] Jason Gustafson commented on KAFKA-3971: I'm trying to understand your usage a lit

[jira] [Commented] (KAFKA-3971) Consumers drop from coordinator and cannot reconnet

[ https://issues.apache.org/jira/browse/KAFKA-3971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15410170#comment-15410170 ] Lei Wang commented on KAFKA-3971: - we consume 800-1000 different topics in a process. we'r

[GitHub] kafka pull request #1706: MINOR: doc changes for QueueTimeMs JMX metrics.

GitHub user alexlod opened a pull request: https://github.com/apache/kafka/pull/1706 MINOR: doc changes for QueueTimeMs JMX metrics. You can merge this pull request into a Git repository by running: $ git pull https://github.com/alexlod/kafka doc-queuetimems-jmx Alternatively

[jira] [Commented] (KAFKA-3971) Consumers drop from coordinator and cannot reconnet

[ https://issues.apache.org/jira/browse/KAFKA-3971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15410166#comment-15410166 ] Jason Gustafson commented on KAFKA-3971: [~wonlay] Thanks for the report. To clari

[jira] [Comment Edited] (KAFKA-3971) Consumers drop from coordinator and cannot reconnet

[ https://issues.apache.org/jira/browse/KAFKA-3971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15410158#comment-15410158 ] Lei Wang edited comment on KAFKA-3971 at 8/5/16 10:05 PM: -- looks

[jira] [Commented] (KAFKA-3971) Consumers drop from coordinator and cannot reconnet

[ https://issues.apache.org/jira/browse/KAFKA-3971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15410158#comment-15410158 ] Lei Wang commented on KAFKA-3971: - looks like this correlated with the number of consumers

Re: [kafka-clients] [VOTE] 0.10.0.1 RC2

Verified checksums and signatures. Ran quickstart. +1 (non-binding) On Fri, Aug 5, 2016 at 2:51 PM, Jun Rao wrote: > Thanks for running the release. +1 > > Jun > > On Thu, Aug 4, 2016 at 6:54 AM, Ismael Juma wrote: > > > Hello Kafka users, developers and client-developers, > > > > This is the t

Re: [kafka-clients] [VOTE] 0.10.0.1 RC2

Thanks for running the release. +1 Jun On Thu, Aug 4, 2016 at 6:54 AM, Ismael Juma wrote: > Hello Kafka users, developers and client-developers, > > This is the third candidate for the release of Apache Kafka 0.10.0.1. This > is a bug fix release and it includes fixes and improvements from 53 J

Re: Request: Please add me to contributor list.

Done. Cheers. Guozhang On Thu, Aug 4, 2016 at 8:57 PM, chetan singh wrote: > Hello, > > Just trying to follow up on the request to add me to the contributor list. > Please let me know if more information is required. > > Chetan Singh > chetanjbsi...@gmail.com > > On Mon, Aug 1, 2016 at 7:11 AM

Build failed in Jenkins: kafka-0.10.0-jdk7 #183

See Changes: [wangguoz] resolve conflicts from cherry-picking -- [...truncated 5743 lines...] org.apache.kafka.streams.kstream.internals.KStreamTransformValuesTest > testTransform PASSED org.a

Re: [VOTE] 0.10.0.1 RC2

+1 (binding) On Fri, Aug 5, 2016 at 12:29 PM, Grant Henke wrote: > +1 (non-binding) > > On Fri, Aug 5, 2016 at 2:04 PM, Dana Powers wrote: > > > passed kafka-python integration tests, +1 > > > > -Dana > > > > > > On Fri, Aug 5, 2016 at 9:35 AM, Tom Crayford > wrote: > > > Heroku has tested thi

Build failed in Jenkins: kafka-trunk-jdk8 #801

See Changes: [wangguoz] KAFKA-3817: KTableRepartitionMap publish old Change first, for -- [...truncated 8287 lines...] org.apache.kafka.common.metrics.JmxReporterTest > testJmxRegistration START

Build failed in Jenkins: kafka-trunk-jdk7 #1463

See Changes: [wangguoz] KAFKA-3817: KTableRepartitionMap publish old Change first, for non-count -- [...truncated 10982 lines...] org.apache.kafka.streams.state.internals.CompositeReadOnlyKeyVa

Re: [VOTE] 0.10.0.1 RC2

+1 (non-binding) On Fri, Aug 5, 2016 at 2:04 PM, Dana Powers wrote: > passed kafka-python integration tests, +1 > > -Dana > > > On Fri, Aug 5, 2016 at 9:35 AM, Tom Crayford wrote: > > Heroku has tested this using the same performance testing setup we used > to > > evaluate the impact of 0.9 ->

Re: [VOTE] 0.10.0.1 RC2

passed kafka-python integration tests, +1 -Dana On Fri, Aug 5, 2016 at 9:35 AM, Tom Crayford wrote: > Heroku has tested this using the same performance testing setup we used to > evaluate the impact of 0.9 -> 0.10 (see https://engineering. > heroku.com/blogs/2016-05-27-apache-kafka-010-evaluati

[jira] [Updated] (KAFKA-3936) Validate user parameters as early as possible

[ https://issues.apache.org/jira/browse/KAFKA-3936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-3936: - Assignee: Damian Guy (was: Guozhang Wang) > Validate user parameters as early as possible > -

Re: [DISCUSS] KIP-72 Allow Sizing Incoming Request Queue in Bytes

Supporting both configs at the same time to begin with would allow enough time for users to experiment with the settings. If in the future its deemed that a memory bound is far superior to a #req bound (which is my opinion but needs evidence to support) the migration discussion could be deferred to

[GitHub] kafka pull request #1705: KAFKA-3817: KTableRepartitionMap publish old Chang...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1705 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Commented] (KAFKA-3817) KTableRepartitionMap should handle null inputs

[ https://issues.apache.org/jira/browse/KAFKA-3817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15409730#comment-15409730 ] ASF GitHub Bot commented on KAFKA-3817: --- Github user asfgit closed the pull request

Re: [DISCUSS] KIP-72 Allow Sizing Incoming Request Queue in Bytes

Thanks for the proposal, Radai. +1 On Fri, Aug 5, 2016 at 8:38 AM, Ismael Juma wrote: > Hi Radai, > > Thanks for the proposal. I think it makes sense to be able to limit the > request queue by bytes. I haven't made up my mind on whether having both > limits is better than having a single one yet

[jira] [Created] (KAFKA-4023) Add thread id as prefix in Kafka Streams thread logging

Guozhang Wang created KAFKA-4023: Summary: Add thread id as prefix in Kafka Streams thread logging Key: KAFKA-4023 URL: https://issues.apache.org/jira/browse/KAFKA-4023 Project: Kafka Issue T

Re: [VOTE] 0.10.0.1 RC2

Heroku has tested this using the same performance testing setup we used to evaluate the impact of 0.9 -> 0.10 (see https://engineering. heroku.com/blogs/2016-05-27-apache-kafka-010-evaluating- performance-in-distributed-systems/). We see no issues at all with them, so +1 (non-binding) from here.

Re: [DISCUSS] KIP-72 Allow Sizing Incoming Request Queue in Bytes

Hi Radai, Thanks for the proposal. I think it makes sense to be able to limit the request queue by bytes. I haven't made up my mind on whether having both limits is better than having a single one yet, but we probably need to make a call on that before we can start a vote. Just a quick point with

Kafka consumer getting duplicate message

Hi there, We are using Kafka 1.0.0.M2 with Spring and we see a lot of duplicate message is getting received by the Listener onMessage() method . We configured : enable.auto.commit=false session.timeout.ms=15000 factory.getContainerProperties().setSyncCommits(true); factory.setConcurrency(5); So

Re: [DISCUSS] KIP-72 Allow Sizing Incoming Request Queue in Bytes

The proposal makes sense to me. I like that the plan to support both limits simultaneously: queued.max.requests is supported in addition queued.max.bytes (both > respected at the same time). In this case a default value of > queued.max.bytes = -1 would maintain backwards compatible behavior. Esp

Re: [DISCUSS] KIP-72 Allow Sizing Incoming Request Queue in Bytes

This makes good sense to me, and seems to have very low amounts of downside with large amounts of upside. +1 On Thursday, 4 August 2016, radai wrote: > Hello, > > I'd like to initiate a discussion about > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > 72%3A+Allow+Sizing+Incoming+Reques

Re: [VOTE] 0.10.0.1 RC2

Looks good here: +1 > On Aug 4, 2016, at 9:54 AM, Ismael Juma wrote: > > Hello Kafka users, developers and client-developers, > > This is the third candidate for the release of Apache Kafka 0.10.0.1. This > is a bug fix release and it includes fixes and improvements from 53 JIRAs > (including a

[jira] [Comment Edited] (KAFKA-4022) TopicCommand is using default max.message.bytes instead of broker's setting

[ https://issues.apache.org/jira/browse/KAFKA-4022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15409207#comment-15409207 ] Tomasz Trębski edited comment on KAFKA-4022 at 8/5/16 9:33 AM: -

[jira] [Commented] (KAFKA-4022) TopicCommand is using default max.message.bytes instead of broker's setting

[ https://issues.apache.org/jira/browse/KAFKA-4022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15409207#comment-15409207 ] Tomasz Trębski commented on KAFKA-4022: --- Hi, First time that I report anything her

[jira] [Created] (KAFKA-4022) TopicCommand is using default max.message.bytes instead of broker's setting

Tomasz Trębski created KAFKA-4022: - Summary: TopicCommand is using default max.message.bytes instead of broker's setting Key: KAFKA-4022 URL: https://issues.apache.org/jira/browse/KAFKA-4022 Project:

[jira] [Commented] (KAFKA-3518) Transient failure in ReplicationTest.test_replication_with_broker_failure with ConsumerTimeoutException

[ https://issues.apache.org/jira/browse/KAFKA-3518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15409094#comment-15409094 ] Ismael Juma commented on KAFKA-3518: Thanks for the investigation Jun. > Transient fa