Wiki edit permission

2017-03-02 Thread Molnár Bálint
Hi, I would like to create a KIP, please add create permission to the wiki page. My username is baluchicken. Thanks

Re: [DISCUSS] KIP-113: Support replicas movement between log directories

2017-03-02 Thread Dong Lin
Hey Jun, Thanks for all the comments! Please see my answer below. I have updated the KIP to address most of the questions and make the KIP easier to understand. Thanks, Dong On Thu, Mar 2, 2017 at 9:35 AM, Jun Rao wrote: > Hi, Dong, > > Thanks for the KIP. A few comments

[jira] [Assigned] (KAFKA-3182) Failure in kafka.network.SocketServerTest.testSocketsCloseOnShutdown

2017-03-02 Thread Armin Braun (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Armin Braun reassigned KAFKA-3182: -- Assignee: Armin Braun > Failure in kafka.network.SocketServerTest.testSocketsCloseOnShutdown >

[jira] [Commented] (KAFKA-3182) Failure in kafka.network.SocketServerTest.testSocketsCloseOnShutdown

2017-03-02 Thread Armin Braun (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15893845#comment-15893845 ] Armin Braun commented on KAFKA-3182: I'll try to fix it this morning, can easily repro it by running

Re: [DISCUSS] KIP-124: Request rate quotas

2017-03-02 Thread Jun Rao
Hi, Rajini, Consider modeling as n * 100% unit. For 2), the question is what's causing the I/O threads to be saturated. It's unlikely that all users' utilization have increased at the same. A more likely case is that a few isolated users' utilization have increased. If so, after increasing the

[jira] [Commented] (KAFKA-1342) Slow controlled shutdowns can result in stale shutdown requests

2017-03-02 Thread Todd Palino (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15893728#comment-15893728 ] Todd Palino commented on KAFKA-1342: [~wushujames], I'm not sure about increasing the number of

[jira] [Updated] (KAFKA-4738) Remove generic type of class ClientState

2017-03-02 Thread Matthias J. Sax (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-4738: --- Status: Patch Available (was: Open) > Remove generic type of class ClientState >

[jira] [Updated] (KAFKA-4722) Add application.id to StreamThread name

2017-03-02 Thread Matthias J. Sax (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-4722: --- Status: Patch Available (was: Open) https://github.com/apache/kafka/pull/2617 > Add

[jira] [Updated] (KAFKA-4706) Unify StreamsKafkaClient instances

2017-03-02 Thread Matthias J. Sax (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-4706: --- Status: Patch Available (was: Open) https://github.com/apache/kafka/pull/2631 > Unify

[jira] [Assigned] (KAFKA-4631) Refresh consumer metadata more frequently for unknown subscribed topics

2017-03-02 Thread Matthias J. Sax (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax reassigned KAFKA-4631: -- Assignee: Rajini Sivaram (was: Matthias J. Sax) > Refresh consumer metadata more

[jira] [Commented] (KAFKA-4834) Kafka cannot delete topic with ReplicaStateMachine went wrong

2017-03-02 Thread huxi (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15893619#comment-15893619 ] huxi commented on KAFKA-4834: - How did you delete the topic? Seems the partitions' state is still Online which

[GitHub] kafka pull request #2631: KAFKA:4706 - Unify StreamsKafkaClient instances

2017-03-02 Thread sharad-develop
GitHub user sharad-develop opened a pull request: https://github.com/apache/kafka/pull/2631 KAFKA:4706 - Unify StreamsKafkaClient instances KAFKA:4706 - Unify StreamsKafkaClient instances You can merge this pull request into a Git repository by running: $ git pull

[jira] [Created] (KAFKA-4834) Kafka cannot delete topic with ReplicaStateMachine went wrong

2017-03-02 Thread Dan (JIRA)
Dan created KAFKA-4834: -- Summary: Kafka cannot delete topic with ReplicaStateMachine went wrong Key: KAFKA-4834 URL: https://issues.apache.org/jira/browse/KAFKA-4834 Project: Kafka Issue Type: Bug

[GitHub] kafka pull request #2622: KAFKA-4631: Request metadata in consumer if topic/...

2017-03-02 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2622 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Commented] (KAFKA-4669) KafkaProducer.flush hangs when NetworkClient.handleCompletedReceives throws exception

2017-03-02 Thread Hai Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15893563#comment-15893563 ] Hai Lin commented on KAFKA-4669: We have the same issue with Kafka 0.8.2.1 to Kafka 0.9 cluster. We are

[jira] [Commented] (KAFKA-4631) Refresh consumer metadata more frequently for unknown subscribed topics

2017-03-02 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15893567#comment-15893567 ] ASF GitHub Bot commented on KAFKA-4631: --- Github user asfgit closed the pull request at:

[jira] [Resolved] (KAFKA-4631) Refresh consumer metadata more frequently for unknown subscribed topics

2017-03-02 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-4631. Resolution: Fixed Issue resolved by pull request 2622

Jenkins build is back to normal : kafka-trunk-jdk8 #1317

2017-03-02 Thread Apache Jenkins Server
See

Jenkins build is back to normal : kafka-trunk-jdk7 #1981

2017-03-02 Thread Apache Jenkins Server
See

[GitHub] kafka pull request #2630: MINOR: improve hanging ResetIntegrationTest

2017-03-02 Thread mjsax
GitHub user mjsax opened a pull request: https://github.com/apache/kafka/pull/2630 MINOR: improve hanging ResetIntegrationTest You can merge this pull request into a Git repository by running: $ git pull https://github.com/mjsax/kafka minor-reset-integration-test

[GitHub] kafka pull request #2481: Kafka 4706 :Unify StreamsKafkaClient instances

2017-03-02 Thread sharad-develop
Github user sharad-develop closed the pull request at: https://github.com/apache/kafka/pull/2481 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[jira] [Commented] (KAFKA-4828) ProcessorTopologyTestDriver does not work when using .through()

2017-03-02 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15893141#comment-15893141 ] ASF GitHub Bot commented on KAFKA-4828: --- GitHub user hrafzali opened a pull request:

[GitHub] kafka pull request #2629: KAFKA-4828: ProcessorTopologyTestDriver does not w...

2017-03-02 Thread hrafzali
GitHub user hrafzali opened a pull request: https://github.com/apache/kafka/pull/2629 KAFKA-4828: ProcessorTopologyTestDriver does not work when using through This resolves the following issues in the ProcessorTopologyTestDriver: - It should not create an internal changelog

[jira] [Commented] (KAFKA-4828) ProcessorTopologyTestDriver does not work when using .through()

2017-03-02 Thread Hamidreza Afzali (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15893105#comment-15893105 ] Hamidreza Afzali commented on KAFKA-4828: - There are two issues: - ProcessorTopologyTestDriver

Re: KIP-122: Add a tool to Reset Consumer Group Offsets

2017-03-02 Thread Jorge Esteban Quilcate Otoya
Hi, I have incorporated the latest revisions into the KIP and created a PR to check the implementation details. If there are no more issues, the VOTE thread has already started. Looking forward to your comments. Jorge. El mar., 28 feb. 2017 a las 19:46, Vahid S Hashemian (<

[jira] [Resolved] (KAFKA-4800) Streams State transition ASCII diagrams need fixing and polishing

2017-03-02 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang resolved KAFKA-4800. -- Resolution: Fixed Fix Version/s: 0.10.2.1 Issue resolved by pull request 2621

[GitHub] kafka pull request #2621: KAFKA-4800: Streams State transition ASCII diagram...

2017-03-02 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2621 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Commented] (KAFKA-4800) Streams State transition ASCII diagrams need fixing and polishing

2017-03-02 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15893051#comment-15893051 ] ASF GitHub Bot commented on KAFKA-4800: --- Github user asfgit closed the pull request at:

[GitHub] kafka pull request #2626: MINOR: Fix typo in GlobalKTable javadocs

2017-03-02 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2626 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Commented] (KAFKA-4652) Improve test coverage KStreamBuilder

2017-03-02 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15893041#comment-15893041 ] ASF GitHub Bot commented on KAFKA-4652: --- Github user asfgit closed the pull request at:

[GitHub] kafka pull request #2597: KAFKA-4652: Added tests for KStreamBuilder

2017-03-02 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2597 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Resolved] (KAFKA-4652) Improve test coverage KStreamBuilder

2017-03-02 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang resolved KAFKA-4652. -- Resolution: Fixed Issue resolved by pull request 2597

[jira] [Updated] (KAFKA-2358) Cluster collection returning methods should never return null

2017-03-02 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-2358: - Fix Version/s: (was: 0.10.3.0) 0.11.0.0 > Cluster collection returning

Build failed in Jenkins: kafka-trunk-jdk7 #1980

2017-03-02 Thread Apache Jenkins Server
See Changes: [junrao] KAFKA4811; ReplicaFetchThread may fail to create due to existing metric -- [...truncated 843.93 KB...]

[jira] [Commented] (KAFKA-2358) Cluster collection returning methods should never return null

2017-03-02 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15893029#comment-15893029 ] ASF GitHub Bot commented on KAFKA-2358: --- Github user asfgit closed the pull request at:

[jira] [Updated] (KAFKA-2358) Cluster collection returning methods should never return null

2017-03-02 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-2358: - Fix Version/s: 0.10.3.0 > Cluster collection returning methods should never return null >

[GitHub] kafka pull request #96: KAFKA-2358 Cluster collection returning methods neve...

2017-03-02 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/96 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

Re: First contribution

2017-03-02 Thread Guozhang Wang
Hi Marc, You can take a look at this page: https://issues.apache.org/jira/issues/?jql=project%20%3D%20KAFKA%20AND%20labels%20%3D%20newbie%20AND%20status%20%3D%20Open Guozhang On Thu, Mar 2, 2017 at 12:59 PM, Marc Juchli wrote: > Hi all. > > We are a group of master

First contribution

2017-03-02 Thread Marc Juchli
Hi all. We are a group of master students, about to start contributing to the apache Kafka project . Are there any suggestions for an open issue for us to knock our heads on and propose a fix within a reasonable amount of time? Thanks! Marc

Re: [DISCUSS] KIP-124: Request rate quotas

2017-03-02 Thread Rajini Sivaram
Jun, If we use request.percentage as the percentage used in a single I/O thread, the total percentage being allocated will be num.io.threads * 100 for I/O threads and num.network.threads * 100 for network threads. A single quota covering the two as a percentage wouldn't quite work if you want to

[jira] [Commented] (KAFKA-4778) OOM on kafka-streams instances with high numbers of unreaped Record classes

2017-03-02 Thread Dave Thomas (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15892886#comment-15892886 ] Dave Thomas commented on KAFKA-4778: [~guozhang] Yes on rocksDB - how we use it is mostly as a chatty

Re: Is there already a KIP or JIRA to change auto.create.topics.enable to default to false?

2017-03-02 Thread Grant Henke
Feel free to create a KIP based on that discussion and drive the jira. I don't think a KIP exists yet. On Thu, Mar 2, 2017 at 1:28 PM, Jeff Widman wrote: > Thanks, that's the ticket I was thinking of. > > On Thu, Mar 2, 2017 at 11:19 AM, Grant Henke

Re: [DISCUSS] KIP-124: Request rate quotas

2017-03-02 Thread Jun Rao
Another way to express an absolute limit is to use request.percentage, but treat it as the percentage used in a single request handling thread. For now, the request handling threads can be just the io threads. In the future, they can cover the network threads as well. This is similar to how top

Re: Is there already a KIP or JIRA to change auto.create.topics.enable to default to false?

2017-03-02 Thread Jeff Widman
Thanks, that's the ticket I was thinking of. On Thu, Mar 2, 2017 at 11:19 AM, Grant Henke wrote: > I think the idea was that once clients have the ability to create topics, > we would move "auto topic creation" client side and deprecate and > eventually remove the support

[jira] [Resolved] (KAFKA-4833) Transient unit test failure in AbstractCoordinatorTest

2017-03-02 Thread Jiangjie Qin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiangjie Qin resolved KAFKA-4833. - Resolution: Invalid Just noticed that the test was failing on a review branch instead of trunk...

Re: Is there already a KIP or JIRA to change auto.create.topics.enable to default to false?

2017-03-02 Thread Grant Henke
I think the idea was that once clients have the ability to create topics, we would move "auto topic creation" client side and deprecate and eventually remove the support for server side auto create. This simplifies error handling, authorization, and puts the client in control of details like

Re: [VOTE] KIP-119: Drop Support for Scala 2.10 in Kafka 0.11

2017-03-02 Thread Becket Qin
Thanks for the clarification, Ismael. In that case, it is reasonable to drop support for Scala 2.10. LinkedIn is probably fine with this change. I did not notice we have recommended Scala version on the download page. +1 on the KIP On Thu, Mar 2, 2017 at 10:46 AM, Grant Henke

Build failed in Jenkins: kafka-trunk-jdk8 #1316

2017-03-02 Thread Apache Jenkins Server
See Changes: [junrao] KAFKA4811; ReplicaFetchThread may fail to create due to existing metric -- [...truncated 159.64 KB...] kafka.utils.timer.TimerTest >

Is there already a KIP or JIRA to change auto.create.topics.enable to default to false?

2017-03-02 Thread Jeff Widman
I thought I saw mention somewhere of changing the default of auto.create.topics.enable to false. I searched, but couldn't find anything in JIRA... am I imagining things? Now that there's API support for creating topics, the version bump to 0.11.0 seems like a good time to re-evaluate whether

Re: [VOTE] KIP-119: Drop Support for Scala 2.10 in Kafka 0.11

2017-03-02 Thread Grant Henke
+1 On Wed, Mar 1, 2017 at 9:44 PM, Gwen Shapira wrote: > +1 > > On Tue, Feb 28, 2017 at 2:40 AM, Ismael Juma wrote: > > > Hi everyone, > > > > Since the few who responded in the discuss thread were in favour and > there > > were no objections, I would like

[jira] [Updated] (KAFKA-4833) Transient unit test failure in AbstractCoordinatorTest

2017-03-02 Thread Jiangjie Qin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiangjie Qin updated KAFKA-4833: Affects Version/s: 0.10.2.0 > Transient unit test failure in AbstractCoordinatorTest >

[jira] [Updated] (KAFKA-4833) Transient unit test failure in AbstractCoordinatorTest

2017-03-02 Thread Jiangjie Qin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiangjie Qin updated KAFKA-4833: Description: I saw this couple of times when running unit test locally. Haven't looked into this

[jira] [Created] (KAFKA-4833) Transient unit test failure in AbstractCoordinatorTest

2017-03-02 Thread Jiangjie Qin (JIRA)
Jiangjie Qin created KAFKA-4833: --- Summary: Transient unit test failure in AbstractCoordinatorTest Key: KAFKA-4833 URL: https://issues.apache.org/jira/browse/KAFKA-4833 Project: Kafka Issue

[jira] [Resolved] (KAFKA-4811) ReplicaFetchThread may fail to create due to existing metric

2017-03-02 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao resolved KAFKA-4811. Resolution: Fixed Fix Version/s: 0.11.0.0 Issue resolved by pull request 2606

[GitHub] kafka pull request #2606: kafka4811: ReplicaFetchThread may fail to create d...

2017-03-02 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2606 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

Re: [VOTE] KIP-107: Add purgeDataBefore() API in AdminClient

2017-03-02 Thread Dong Lin
Hey Jun, Thanks for this information. I am not aware of this difference between the purge and delete. Given this difference, I will prefer to the existing name of the purge. Ismael, please let me if you are strong about using delete. Thanks, Dong On Thu, Mar 2, 2017 at 9:40 AM, Jun Rao

Re: [DISCUSS] KIP-113: Support replicas movement between log directories

2017-03-02 Thread Jun Rao
Hi, Dong, Thanks for the KIP. A few comments below. 1. For moving data across directories 1.1 I am not sure why we want to use ReplicaFetcherThread to move data around in the leader. ReplicaFetchThread fetches data from socket. For moving data locally, it seems that we want to avoid the socket

Re: [DISCUSS] KIP-112: Handle disk failure for JBOD

2017-03-02 Thread Jun Rao
Hi, Dong, Ok. We can keep LeaderAndIsrRequest as it is in the wiki. Since we need both KIP-112 and KIP-113 to make a compelling case for JBOD, perhaps we should discuss KIP-113 before voting for both? I left some comments in the other thread. Thanks, Jun On Wed, Mar 1, 2017 at 1:58 PM, Dong

Re: [VOTE] KIP-107: Add purgeDataBefore() API in AdminClient

2017-03-02 Thread Jun Rao
Hi, Dong, It seems that delete means removing everything while purge means removing a portion. So, it seems that it's better to be able to distinguish the two? Thanks, Jun On Wed, Mar 1, 2017 at 1:57 PM, Dong Lin wrote: > Hi all, > > I have updated the KIP to include a

[jira] [Commented] (KAFKA-4722) Add application.id to StreamThread name

2017-03-02 Thread Sharad (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15892648#comment-15892648 ] Sharad commented on KAFKA-4722: --- I have enabled github notifications. Also, I have updated the code as per

[GitHub] kafka pull request #2616: KAFKA:4738 - Remove generic type of class ClientSt...

2017-03-02 Thread sharad-develop
GitHub user sharad-develop reopened a pull request: https://github.com/apache/kafka/pull/2616 KAFKA:4738 - Remove generic type of class ClientState Remove generic type of class ClientState and generic T inTaskAssignor. You can merge this pull request into a Git repository by

[GitHub] kafka pull request #2625: KAFKA:4623- Change default unclean.leader.election...

2017-03-02 Thread sharad-develop
GitHub user sharad-develop reopened a pull request: https://github.com/apache/kafka/pull/2625 KAFKA:4623- Change default unclean.leader.election.enabled from True to False KAFKA:4623- Change default unclean.leader.election.enabled from True to False You can merge this pull

[GitHub] kafka pull request #2470: Kafka-4711 :Change Default unclean.leader.election...

2017-03-02 Thread sharad-develop
Github user sharad-develop closed the pull request at: https://github.com/apache/kafka/pull/2470 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

Re: Add me to jira/confluence?

2017-03-02 Thread Guozhang Wang
Done. Guozhang On Thu, Mar 2, 2017 at 9:08 AM, Jeff Widman wrote: > Can someone add me to Jira/confluence for Kafka? > > Don't use this work email, use j...@jeffwidman.com > > Thanks! > -- -- Guozhang

Add me to jira/confluence?

2017-03-02 Thread Jeff Widman
Can someone add me to Jira/confluence for Kafka? Don't use this work email, use j...@jeffwidman.com Thanks!

0.10.2.0 Release Retrospective

2017-03-02 Thread Ewen Cheslack-Postava
Hi all, I think it'd be useful to review & refine our release process, especially in light of the new time based releases. I've created a page under the release plan for 0.10.2.0 and added some of my notes: https://cwiki.apache.org/confluence/display/KAFKA/0.10.2.0+Retrospective If others could

Re: [DISCUSS] KIP-129: Kafka Streams Exactly-Once Semantics

2017-03-02 Thread Jay Kreps
I second the concern on with the one producer per task approach. At a high-level it seems to make sense but I think Damian is exactly right that that cuts against the general design of the producer. Many people have high input partition counts and will have high task counts as a result. I think

Re: Consumption on a explicitly (dynamically) created topic has a 5 minute delay

2017-03-02 Thread Jaikiran Pai
Thank you for pointing me to that JIRA. It indeed is the same issue we discussed in this thread. I'll keep a watch on that JIRA for the code to be merged. -Jaikiran On Thursday 02 March 2017 07:11 PM, Rajini Sivaram wrote: This issue is being addressed in KAFKA-4631. See

[jira] [Comment Edited] (KAFKA-4825) Likely Data Loss in ReassignPartitionsTest System Test

2017-03-02 Thread Ben Stopford (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15890661#comment-15890661 ] Ben Stopford edited comment on KAFKA-4825 at 3/2/17 2:55 PM: - This could be a

[GitHub] kafka pull request #2628: MINOR: Bump version to 0.11.0.0-SNAPSHOT

2017-03-02 Thread ijuma
GitHub user ijuma opened a pull request: https://github.com/apache/kafka/pull/2628 MINOR: Bump version to 0.11.0.0-SNAPSHOT There won't be a 0.10.3.0. You can merge this pull request into a Git repository by running: $ git pull https://github.com/ijuma/kafka

Re: groupBy without auto-repartition topics for Kafka Streams

2017-03-02 Thread Tianji Li
Hi Guys, Thanks so much for your quick replies, very appreciated! Thanks Tianji On Wed, Mar 1, 2017 at 2:53 PM, Matthias J. Sax wrote: > It should be: > > groupBy -> always trigger repartitioning > groupByKey -> maybe trigger repartitioning > > And there will not be two

Re: [DISCUSS] 0.10.3.0/0.11.0.0 release planning

2017-03-02 Thread Ismael Juma
I've updated JIRA versions and the KIP wiki page. Ismael On Thu, Mar 2, 2017 at 12:51 AM, Ismael Juma wrote: > Thanks everyone for the feedback. Since everyone was in favour and we had > covered this particular case in the time-based release plan[1], I went > ahead and

[jira] [Commented] (KAFKA-1368) Upgrade log4j

2017-03-02 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15892282#comment-15892282 ] Ismael Juma commented on KAFKA-1368: Differences in the configuration format:

[jira] [Updated] (KAFKA-3353) Remove deprecated producer configs.

2017-03-02 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-3353: --- Fix Version/s: 0.11.0.0 > Remove deprecated producer configs. > --- >

[jira] [Updated] (KAFKA-3353) Remove deprecated producer configs.

2017-03-02 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-3353: --- Affects Version/s: (was: 0.11.0.0.0) 0.10.2.0 > Remove deprecated producer

[jira] [Updated] (KAFKA-3356) Remove ConsumerOffsetChecker, deprecated in 0.9, in 0.11

2017-03-02 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-3356: --- Affects Version/s: (was: 0.11.0.0.0) 0.10.2.0 > Remove

[jira] [Updated] (KAFKA-3958) To set the maximum no of consumers which can consume from the particular topic.

2017-03-02 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-3958: --- Affects Version/s: (was: 0.11.0.0.0) > To set the maximum no of consumers which can consume from

[jira] [Updated] (KAFKA-4604) Gradle Eclipse plugin creates projects for non project subfolders

2017-03-02 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-4604: --- Affects Version/s: (was: 0.11.0.0.0) > Gradle Eclipse plugin creates projects for non project

[jira] [Updated] (KAFKA-4239) Replication Quotas: kafka-configs.sh should allow the throttled rate to be set using broker wildcards

2017-03-02 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-4239: --- Affects Version/s: (was: 0.11.0.0.0) 0.10.2.0 > Replication Quotas:

[jira] [Updated] (KAFKA-4210) Improve Replication Follower Throttling Stability By Checking Quota During Backoff

2017-03-02 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-4210: --- Affects Version/s: (was: 0.11.0.0.0) 0.10.2.0 > Improve Replication

[jira] [Updated] (KAFKA-4767) KafkaProducer is not joining its IO thread properly

2017-03-02 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-4767: --- Affects Version/s: (was: 0.11.0.0.0) 0.10.2.0 > KafkaProducer is not

[jira] [Updated] (KAFKA-4711) Change Default unclean.leader.election.enabled from True to False (KIP-106)

2017-03-02 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-4711: --- Affects Version/s: (was: 0.11.0.0.0) > Change Default unclean.leader.election.enabled from True

[jira] [Commented] (KAFKA-1368) Upgrade log4j

2017-03-02 Thread Edoardo Comar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15892271#comment-15892271 ] Edoardo Comar commented on KAFKA-1368: -- Yes [~ijuma] we will pick this one up. I work together with

[jira] [Commented] (KAFKA-1368) Upgrade log4j

2017-03-02 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15892265#comment-15892265 ] Ismael Juma commented on KAFKA-1368: [~ecomar], are you interested in picking this up? We could switch

Re: Consumption on a explicitly (dynamically) created topic has a 5 minute delay

2017-03-02 Thread Rajini Sivaram
This issue is being addressed in KAFKA-4631. See https://issues.apache.org/jira/browse/KAFKA-4631 and the discussion in the PR https://github.com/apache/kafka/pull/2622 for details. Regards, Rajini On Thu, Mar 2, 2017 at 4:35 AM, Jaikiran Pai wrote: > For future

[jira] [Updated] (KAFKA-4094) Fix importance labels for Kafka Server config

2017-03-02 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-4094: --- Fix Version/s: (was: 0.11.0.0) > Fix importance labels for Kafka Server config >

[jira] [Updated] (KAFKA-4094) Fix importance labels for Kafka Server config

2017-03-02 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-4094: --- Labels: newbie (was: ) > Fix importance labels for Kafka Server config >

[jira] [Updated] (KAFKA-2507) Replace ControlledShutdown{Request,Response} with org.apache.kafka.common.requests equivalent

2017-03-02 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-2507: --- Fix Version/s: (was: 0.11.0.0) 0.12.0.0 > Replace

[jira] [Updated] (KAFKA-3038) Speeding up partition reassignment after broker failure

2017-03-02 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-3038: --- Fix Version/s: (was: 0.11.0.0) > Speeding up partition reassignment after broker failure >

Allow to replace Zookeeper with different Coordination Service (etcd/consul)

2017-03-02 Thread Molnár Bálint
Hi, I was wonderring to refactor Kafka core code to be able to use different Coordination Service than Zookeeper. I know I will need to create a KIP for that. I think the first part of this task to refactor the classes which are using the ZkUtil methods to use a zookeeper independent trait

Re: [DISCUSS] KIP 130: Expose states of active tasks to KafkaStreams public API

2017-03-02 Thread Damian Guy
Hi Florian, Thanks for the KIP. It seems there is some overlap here with what we already have in KafkaStreams.allMetadata(). This currently returns a Collection where each StreamsMetadata instance holds the state stores and partition assignment for every instance of the KafkaStreams application.

[jira] [Updated] (KAFKA-4832) kafka producer send Async message to the wrong IP cannot to stop producer.close()

2017-03-02 Thread Wang Hong (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wang Hong updated KAFKA-4832: - Description: 1.When I tried to send msg by Async with wrong IP in loop 1400times 10batches. 2.I use

[jira] [Updated] (KAFKA-4832) kafka producer send Async message to the wrong IP cannot to stop producer.close()

2017-03-02 Thread Wang Hong (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wang Hong updated KAFKA-4832: - Description: 1.When I tried to send msg by Async with wrong IP in loop 1400times 10batches. 2.I use

[GitHub] kafka pull request #2627: [WIP]: add a consumer leave group config

2017-03-02 Thread dguy
GitHub user dguy opened a pull request: https://github.com/apache/kafka/pull/2627 [WIP]: add a consumer leave group config Add a new config to ConsumerConfig, `leave.group.on.close`. When set to `false` then the `AbstractCoordinator` will not send the `LeaveGroupRequest`. Default

[jira] [Updated] (KAFKA-4092) retention.bytes should not be allowed to be less than segment.bytes

2017-03-02 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-4092: --- Description: Right now retention.bytes can be as small as the user wants but it doesn't really get

[jira] [Created] (KAFKA-4832) kafka producer send message to the wrong IP cannot to stop producer.close()

2017-03-02 Thread Wang Hong (JIRA)
Wang Hong created KAFKA-4832: Summary: kafka producer send message to the wrong IP cannot to stop producer.close() Key: KAFKA-4832 URL: https://issues.apache.org/jira/browse/KAFKA-4832 Project: Kafka

[jira] [Updated] (KAFKA-4092) retention.bytes should not be allowed to be less than segment.bytes

2017-03-02 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-4092: --- Description: Right now retention.bytes can be as small as the user wants but it doesn't really get

[jira] [Commented] (KAFKA-4092) retention.bytes should not be allowed to be less than segment.bytes

2017-03-02 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15892019#comment-15892019 ] Ismael Juma commented on KAFKA-4092: [~ewencp], yeah, I was thinking about what's the right thing to

[jira] [Updated] (KAFKA-4092) retention.bytes should not be allowed to be less than segment.bytes

2017-03-02 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-4092: --- Fix Version/s: (was: 0.10.2.0) > retention.bytes should not be allowed to be less than

[jira] [Updated] (KAFKA-4823) Creating Kafka Producer on application running on Java older version

2017-03-02 Thread Aneesh (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aneesh updated KAFKA-4823: -- Summary: Creating Kafka Producer on application running on Java older version (was: Creating Kafka Producer on

  1   2   >