Commitlog path while Upgrading kafka server from 0.9 to 0.10.2.0

2017-04-20 Thread dhiraj prajapati
Hi, I want to do a rolling upgrade of kafka server from 0.9 to 0.10.2.0. Should I keep path of the commit logs the same? what is the impact of keeping the path same/different? Thanks in advance.

Re: [VOTE] KIP 130: Expose states of active tasks to KafkaStreams public API

2017-04-20 Thread Guozhang Wang
+1. Thanks a lot for the KIP! Guozhang On Wed, Apr 5, 2017 at 1:57 PM, Florian Hussonnois wrote: > Hi All, > > I would like to start the vote for the KIP-130 : > > https://cwiki.apache.org/confluence/display/KAFKA/KIP+ >

[jira] [Commented] (KAFKA-5006) KeyValueStore.put may throw exception unrelated to the current put attempt

2017-04-20 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15978061#comment-15978061 ] Guozhang Wang commented on KAFKA-5006: -- [~xvrl] That is right. > KeyValueStore.put may throw

[jira] [Updated] (KAFKA-5088) some spelling error in code comment

2017-04-20 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-5088: - Resolution: Fixed Status: Resolved (was: Patch Available) Issue resolved by pull request

[GitHub] kafka pull request #2871: KAFKA-5088: some spelling error in code comment

2017-04-20 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2871 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Commented] (KAFKA-5088) some spelling error in code comment

2017-04-20 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15978059#comment-15978059 ] ASF GitHub Bot commented on KAFKA-5088: --- Github user asfgit closed the pull request at:

Re: [DISCUSS] KIP-114: KTable materialization and improved semantics

2017-04-20 Thread Guozhang Wang
Thanks Eno for the updated KIP. LGTM with just a few minor comments: 1. Regarding the user-facing semantics, I thought we will claim that "KTables generated from functions that do NOT specify a table name will NOT be queryable"; but it seems you're proposing to claim it "may not possible to be

[GitHub] kafka pull request #2881: MINOR: Remove redundant CRC validation for non-com...

2017-04-20 Thread hachikuji
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/2881 MINOR: Remove redundant CRC validation for non-compressed records in older message formats You can merge this pull request into a Git repository by running: $ git pull

Re: Request to be contributor

2017-04-20 Thread Guozhang Wang
Done. Cheers. Guozhang On Wed, Apr 19, 2017 at 5:35 AM, Raul Estrada wrote: > Hi dev team, > > Please add me as contributor in Kafka JIRA. > My JIRA ID is uurl > > Thanks, > Raul > -- -- Guozhang

[jira] [Commented] (KAFKA-5099) Replica Deletion Regression from KIP-101

2017-04-20 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15977937#comment-15977937 ] Ismael Juma commented on KAFKA-5099: Great catch! Seems like our tests need to verify that the space

[jira] [Updated] (KAFKA-5099) Replica Deletion Regression from KIP-101

2017-04-20 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-5099: --- Priority: Blocker (was: Major) > Replica Deletion Regression from KIP-101 >

[jira] [Updated] (KAFKA-5099) Replica Deletion Regression from KIP-101

2017-04-20 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-5099: --- Fix Version/s: 0.11.0.0 > Replica Deletion Regression from KIP-101 >

[jira] [Commented] (KAFKA-5099) Replica Deletion Regression from KIP-101

2017-04-20 Thread Onur Karaman (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15977902#comment-15977902 ] Onur Karaman commented on KAFKA-5099: - My initial guess is that the new epoch checkpoint file isn't

[jira] [Created] (KAFKA-5099) Replica Deletion Regression from KIP-101

2017-04-20 Thread Onur Karaman (JIRA)
Onur Karaman created KAFKA-5099: --- Summary: Replica Deletion Regression from KIP-101 Key: KAFKA-5099 URL: https://issues.apache.org/jira/browse/KAFKA-5099 Project: Kafka Issue Type: Bug

Jenkins build is back to normal : kafka-trunk-jdk8 #1440

2017-04-20 Thread Apache Jenkins Server
See

Jenkins build is back to normal : kafka-trunk-jdk7 #2108

2017-04-20 Thread Apache Jenkins Server
See

Build failed in Jenkins: kafka-0.10.2-jdk7 #149

2017-04-20 Thread Apache Jenkins Server
See Changes: [wangguoz] MINOR: reduce commit interval and cache size for integration test [wangguoz] KAFKA-5095: Adjust accepted overhead for ThreadCacheTest [wangguoz] KAFKA-4980: testReprocessingFromScratch

Re: [DISCUSS] KIP 141 - ProducerRecordBuilder Interface

2017-04-20 Thread Stephane Maarek
Matthias: I was definitely on board with you at first, but Ismael made the comment that for: public ProducerRecord(String topic, K key, V value, Integer partition) public ProducerRecord(String topic, K key, V value, Long timestamp) Integer and Long are way too close in terms of meaning, and

Re: [DISCUSS] KIP-142: Add ListTopicsRequest to efficiently list all the topics in a cluster

2017-04-20 Thread Colin McCabe
If you have thousands of topics and dozens of brokers, your metadata responses could easily get into the megabyte range. At that point it will start affecting your latency statistics. Right now there is absolutely no way for management systems to work around this. They have to make a giant

[GitHub] kafka pull request #2880: New transform: ExtractFields

2017-04-20 Thread dbtucker
GitHub user dbtucker opened a pull request: https://github.com/apache/kafka/pull/2880 New transform: ExtractFields Needed a transform that properly handled extracting fields from multiple different levels in s nested records. You can merge this pull request into a

Build failed in Jenkins: kafka-0.10.2-jdk7 #148

2017-04-20 Thread Apache Jenkins Server
See Changes: [jason] KAFKA-5097; Fix regression in consumer caused by unsafe access to -- [...truncated 770.57 KB...]

[jira] [Commented] (KAFKA-5006) KeyValueStore.put may throw exception unrelated to the current put attempt

2017-04-20 Thread JIRA
[ https://issues.apache.org/jira/browse/KAFKA-5006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15977795#comment-15977795 ] Xavier Léauté commented on KAFKA-5006: -- [~guozhang] that sounds reasonable, I assume it would

[jira] [Resolved] (KAFKA-4980) testReprocessingFromScratch unit test failure

2017-04-20 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang resolved KAFKA-4980. -- Resolution: Fixed Fix Version/s: 0.10.2.1 > testReprocessingFromScratch unit test

[jira] [Commented] (KAFKA-4980) testReprocessingFromScratch unit test failure

2017-04-20 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15977756#comment-15977756 ] Guozhang Wang commented on KAFKA-4980: -- Cherry-picked to 0.10.2 > testReprocessingFromScratch unit

Re: [VOTE] 0.10.2.1 RC2

2017-04-20 Thread Gwen Shapira
This is what happens when you do a last-minute merge :( Will roll a new RC tomorrow morning. Please find all the bugs tonight so we wouldn't need another RC... On Thu, Apr 20, 2017 at 11:35 AM, Jason Gustafson wrote: > Hey Gwen, > > Eno Thereska found a blocking issue here:

Build failed in Jenkins: kafka-trunk-jdk7 #2107

2017-04-20 Thread Apache Jenkins Server
See Changes: [wangguoz] KAFKA-4937: Batch offset fetches in the Consumer [wangguoz] MINOR: improve test stability for Streams broker-compatibility test [jason] KAFKA-5014; NetworkClient.leastLoadedNode should

[jira] [Assigned] (KAFKA-4927) KStreamsTestDriver fails with NPE when KStream.to() sinks are used

2017-04-20 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang reassigned KAFKA-4927: Assignee: Wim Van Leuven > KStreamsTestDriver fails with NPE when KStream.to() sinks are

[jira] [Commented] (KAFKA-4927) KStreamsTestDriver fails with NPE when KStream.to() sinks are used

2017-04-20 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15977724#comment-15977724 ] Guozhang Wang commented on KAFKA-4927: -- [~wimvanleuven] assigning you to this ticket. Moving forward

Re: [DISCUSS] KIP-142: Add ListTopicsRequest to efficiently list all the topics in a cluster

2017-04-20 Thread Jeff Widman
In theory this is nice. However, one possibly silly question... personally I've found that most times when I wanted to list the topics, I normally also wanted to know additional metadata such as the leader of the topic. So I just want to doublecheck that there are enough common use cases that

[jira] [Created] (KAFKA-5098) KafkaProducer.send() blocks and generates TimeoutException if topic name has illegal char

2017-04-20 Thread Jeff Larsen (JIRA)
Jeff Larsen created KAFKA-5098: -- Summary: KafkaProducer.send() blocks and generates TimeoutException if topic name has illegal char Key: KAFKA-5098 URL: https://issues.apache.org/jira/browse/KAFKA-5098

Re: [DISCUSS] KIP-126 - Allow KafkaProducer to batch based on uncompressed size

2017-04-20 Thread Dong Lin
Thanks for the explanation. I agree that it is not easy to have a well-defined and accurate measurement of the split ratio. On Wed, Apr 19, 2017 at 9:38 AM, Becket Qin wrote: > Thanks for the comment, Dong. I think the batch-split-ratio makes sense but > is kind of

[jira] [Commented] (KAFKA-5095) ThreadCacheTest.cacheOverheadsSmallValues fails intermittently

2017-04-20 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15977628#comment-15977628 ] ASF GitHub Bot commented on KAFKA-5095: --- Github user asfgit closed the pull request at:

[jira] [Resolved] (KAFKA-5095) ThreadCacheTest.cacheOverheadsSmallValues fails intermittently

2017-04-20 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang resolved KAFKA-5095. -- Resolution: Fixed Fix Version/s: 0.10.2.1 0.11.0.0 Issue resolved by

[GitHub] kafka pull request #2877: KAFKA-5095: Adjust accepted overhead

2017-04-20 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2877 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Commented] (KAFKA-5006) KeyValueStore.put may throw exception unrelated to the current put attempt

2017-04-20 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15977593#comment-15977593 ] Guozhang Wang commented on KAFKA-5006: -- [~xvrl] Thanks for reporting the issue. After looking into

[jira] [Commented] (KAFKA-5097) KafkaConsumer.poll throws IllegalStateException

2017-04-20 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15977590#comment-15977590 ] ASF GitHub Bot commented on KAFKA-5097: --- Github user enothereska closed the pull request at:

[GitHub] kafka pull request #2876: KAFKA-5097: guard against unassigned partitions

2017-04-20 Thread enothereska
Github user enothereska closed the pull request at: https://github.com/apache/kafka/pull/2876 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[jira] [Commented] (KAFKA-5073) Kafka Streams stuck rebalancing after exception thrown in rebalance listener

2017-04-20 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15977582#comment-15977582 ] ASF GitHub Bot commented on KAFKA-5073: --- Github user asfgit closed the pull request at:

[jira] [Updated] (KAFKA-5073) Kafka Streams stuck rebalancing after exception thrown in rebalance listener

2017-04-20 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-5073: - Resolution: Fixed Fix Version/s: 0.11.0.0 Status: Resolved (was: Patch

[jira] [Updated] (KAFKA-5006) KeyValueStore.put may throw exception unrelated to the current put attempt

2017-04-20 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-5006: - Labels: user-experience (was: ) > KeyValueStore.put may throw exception unrelated to the current

[GitHub] kafka pull request #2856: KAFKA-5073: Kafka Streams stuck rebalancing after ...

2017-04-20 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2856 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Commented] (KAFKA-3502) Build is killed during kafka streams tests due to `pure virtual method called` error

2017-04-20 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15977574#comment-15977574 ] Guozhang Wang commented on KAFKA-3502: -- [~enothereska] could you list the details on java / gradle

[jira] [Assigned] (KAFKA-3502) Build is killed during kafka streams tests due to `pure virtual method called` error

2017-04-20 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang reassigned KAFKA-3502: Assignee: (was: Guozhang Wang) > Build is killed during kafka streams tests due to

[jira] [Updated] (KAFKA-5047) NullPointerException while using GlobalKTable in KafkaStreams

2017-04-20 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-5047: - Resolution: Fixed Fix Version/s: 0.11.0.0 Status: Resolved (was: Patch

[jira] [Commented] (KAFKA-5047) NullPointerException while using GlobalKTable in KafkaStreams

2017-04-20 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15977568#comment-15977568 ] ASF GitHub Bot commented on KAFKA-5047: --- Github user asfgit closed the pull request at:

[GitHub] kafka pull request #2834: KAFKA-5047: NullPointerException while using Globa...

2017-04-20 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2834 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Updated] (KAFKA-5037) Infinite loop if all input topics are unknown at startup

2017-04-20 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-5037: - Labels: user-experience (was: ) > Infinite loop if all input topics are unknown at startup >

[DISCUSS] KIP-142: Add ListTopicsRequest to efficiently list all the topics in a cluster

2017-04-20 Thread Colin McCabe
Hi all, While working on the AdminClient, we determined that we didn't have a way to efficiently list all the topics in the cluster. I wrote up a KIP to add a ListTopicsRequest to make this more efficient in larger clusters with more brokers and topics. Please take a look at the KIP here:

[jira] [Comment Edited] (KAFKA-4795) Confusion around topic deletion

2017-04-20 Thread Edoardo Comar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15977538#comment-15977538 ] Edoardo Comar edited comment on KAFKA-4795 at 4/20/17 9:15 PM: --- [~vahid] I

[jira] [Comment Edited] (KAFKA-4795) Confusion around topic deletion

2017-04-20 Thread Edoardo Comar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15977538#comment-15977538 ] Edoardo Comar edited comment on KAFKA-4795 at 4/20/17 9:13 PM: --- [~vahid] I

[jira] [Commented] (KAFKA-4795) Confusion around topic deletion

2017-04-20 Thread Edoardo Comar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15977538#comment-15977538 ] Edoardo Comar commented on KAFKA-4795: -- [~vahid] I may still see topics ??marked for deletion?? when

[GitHub] kafka pull request #2789: MINOR: reduce commit interval and cache size for i...

2017-04-20 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2789 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Commented] (KAFKA-5014) SSL Channel not ready but tcp is established and the server is hung will not sending metadata

2017-04-20 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15977533#comment-15977533 ] ASF GitHub Bot commented on KAFKA-5014: --- Github user asfgit closed the pull request at:

[GitHub] kafka pull request #2813: KAFKA-5014: NetworkClient.leastLoadedNode should c...

2017-04-20 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2813 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Resolved] (KAFKA-5014) SSL Channel not ready but tcp is established and the server is hung will not sending metadata

2017-04-20 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-5014. Resolution: Fixed Issue resolved by pull request 2813

[jira] [Comment Edited] (KAFKA-5007) Kafka Replica Fetcher Thread- Resource Leak

2017-04-20 Thread Joseph Aliase (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15977495#comment-15977495 ] Joseph Aliase edited comment on KAFKA-5007 at 4/20/17 9:01 PM: --- Issue

[jira] [Comment Edited] (KAFKA-5007) Kafka Replica Fetcher Thread- Resource Leak

2017-04-20 Thread Joseph Aliase (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15977495#comment-15977495 ] Joseph Aliase edited comment on KAFKA-5007 at 4/20/17 9:01 PM: --- Issue

[jira] [Commented] (KAFKA-5016) Consumer hang in poll method while rebalancing is in progress

2017-04-20 Thread Vahid Hashemian (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15977499#comment-15977499 ] Vahid Hashemian commented on KAFKA-5016: [~domenico74] Thanks a lot for sharing the executable

[jira] [Updated] (KAFKA-5007) Kafka Replica Fetcher Thread- Resource Leak

2017-04-20 Thread Joseph Aliase (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Aliase updated KAFKA-5007: - Attachment: jstack-zoo.out jstack-kafka.out lsofkafka.txt

[jira] [Comment Edited] (KAFKA-5007) Kafka Replica Fetcher Thread- Resource Leak

2017-04-20 Thread Joseph Aliase (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15970549#comment-15970549 ] Joseph Aliase edited comment on KAFKA-5007 at 4/20/17 8:50 PM: --- It seems

[GitHub] kafka pull request #2836: MINOR: improve test stability for Streams broker-c...

2017-04-20 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2836 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Resolved] (KAFKA-5097) KafkaConsumer.poll throws IllegalStateException

2017-04-20 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-5097. Resolution: Fixed Issue resolved by pull request 2876

[GitHub] kafka pull request #2769: KAFKA-4937: Batch offset fetches in the Consumer

2017-04-20 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2769 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Resolved] (KAFKA-4937) Batch resetting offsets in Streams' StoreChangelogReader

2017-04-20 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang resolved KAFKA-4937. -- Resolution: Fixed Fix Version/s: 0.11.0.0 Issue resolved by pull request 2769

[jira] [Commented] (KAFKA-4937) Batch resetting offsets in Streams' StoreChangelogReader

2017-04-20 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15977466#comment-15977466 ] ASF GitHub Bot commented on KAFKA-4937: --- Github user asfgit closed the pull request at:

[jira] [Commented] (KAFKA-4667) Connect should create internal topics

2017-04-20 Thread Gwen Shapira (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15977460#comment-15977460 ] Gwen Shapira commented on KAFKA-4667: - Also, note that validating configuration of existing internal

[jira] [Commented] (KAFKA-566) Add last modified time to the TopicMetadataRequest

2017-04-20 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15977395#comment-15977395 ] Jun Rao commented on KAFKA-566: --- [~jozi-k], thanks for your interest. I am not sure if [~sriharsha] is still

[jira] [Work started] (KAFKA-5097) KafkaConsumer.poll throws IllegalStateException

2017-04-20 Thread Eno Thereska (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-5097 started by Eno Thereska. --- > KafkaConsumer.poll throws IllegalStateException >

[jira] [Commented] (KAFKA-617) Refactor KafkaApis.handleOffsetRequest

2017-04-20 Thread Jozef Koval (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15977388#comment-15977388 ] Jozef Koval commented on KAFKA-617: --- Hi [~jkreps], I would like to work on this issue, but

[jira] [Resolved] (KAFKA-451) follower replica may need to backoff the fetching if leader is not ready yet

2017-04-20 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao resolved KAFKA-451. --- Resolution: Not A Problem [~jozi-k], thanks for your interest. As Ismael mentioned, this issue actually has

[GitHub] kafka pull request #2866: MINOR: Improvements to PID snapshot management

2017-04-20 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2866 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Commented] (KAFKA-566) Add last modified time to the TopicMetadataRequest

2017-04-20 Thread Jozef Koval (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15977334#comment-15977334 ] Jozef Koval commented on KAFKA-566: --- Dear [~harsha_ch], [~junrao] and [~nehanarkhede]. Can I ask for

[jira] [Commented] (KAFKA-523) OffsetRequest handler does not handle errors

2017-04-20 Thread Jozef Koval (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15977300#comment-15977300 ] Jozef Koval commented on KAFKA-523: --- The specified method does not exist anymore. Does it mean this issue

[jira] [Commented] (KAFKA-4950) ConcurrentModificationException when iterating over Kafka Metrics

2017-04-20 Thread Dumitru Postoronca (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15977294#comment-15977294 ] Dumitru Postoronca commented on KAFKA-4950: --- [~vahid]: updated description with whole class. The

[jira] [Commented] (KAFKA-454) ProducerSendThread calls ListBuffer.size a whole bunch. That is a O(n) operation

2017-04-20 Thread Jozef Koval (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15977292#comment-15977292 ] Jozef Koval commented on KAFKA-454: --- I think this will not fix as the class is deprecated and will be

[jira] [Updated] (KAFKA-4950) ConcurrentModificationException when iterating over Kafka Metrics

2017-04-20 Thread Dumitru Postoronca (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dumitru Postoronca updated KAFKA-4950: -- Description: It looks like the when calling {{PartitionStates.partitionSet()}}, while

[jira] [Commented] (KAFKA-451) follower replica may need to backoff the fetching if leader is not ready yet

2017-04-20 Thread Jozef Koval (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15977280#comment-15977280 ] Jozef Koval commented on KAFKA-451: --- Hi guys, I would like to work on this issue, but need more info. Can

[jira] [Updated] (KAFKA-4950) ConcurrentModificationException when iterating over Kafka Metrics

2017-04-20 Thread Dumitru Postoronca (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dumitru Postoronca updated KAFKA-4950: -- Description: It looks like the when calling {{PartitionStates.partitionSet()}}, while

Re: [VOTE] 0.10.2.1 RC2

2017-04-20 Thread Jason Gustafson
Hey Gwen, Eno Thereska found a blocking issue here: https://issues.apache.org/jira/browse/KAFKA-5097. Sorry to say, but we probably need another RC. -Jason On Wed, Apr 19, 2017 at 8:04 AM, Swen Moczarski wrote: > Hi Gwen, > thanks for the new release candidate. Did a

[jira] [Updated] (KAFKA-5097) KafkaConsumer.poll throws IllegalStateException

2017-04-20 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson updated KAFKA-5097: --- Priority: Blocker (was: Critical) > KafkaConsumer.poll throws IllegalStateException >

[jira] [Commented] (KAFKA-4291) TopicCommand --describe shows topics marked for deletion as under-replicated and unavailable

2017-04-20 Thread Vahid Hashemian (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15977214#comment-15977214 ] Vahid Hashemian commented on KAFKA-4291: [~mimaison] I opened

[jira] [Commented] (KAFKA-5096) Only log invalid user configs and overwrite with correct one

2017-04-20 Thread Matthias J. Sax (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15977190#comment-15977190 ] Matthias J. Sax commented on KAFKA-5096: [~johnma] Thanks for picking up this Jira. I just update

[jira] [Comment Edited] (KAFKA-5096) Only log invalid user configs and overwrite with correct one

2017-04-20 Thread Matthias J. Sax (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15977190#comment-15977190 ] Matthias J. Sax edited comment on KAFKA-5096 at 4/20/17 6:18 PM: -

[jira] [Updated] (KAFKA-5096) Only log invalid user configs and overwrite with correct one

2017-04-20 Thread Matthias J. Sax (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-5096: --- Description: Streams does not allow to overwrite some config parameters (eg,

Re: [DISCUSS] KIP-138: Change punctuate semantics

2017-04-20 Thread Damian Guy
Hi Michal, Thanks for the KIP. I'd like to propose a bit more of a radical change to the API. 1. deprecate the punctuate method on Processor 2. create a new Functional Interface just for Punctuation, something like: interface Punctuator { void punctuate(long timestamp) } 3. add a new schedule

[jira] [Commented] (KAFKA-4893) async topic deletion conflicts with max topic length

2017-04-20 Thread Vahid Hashemian (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15977130#comment-15977130 ] Vahid Hashemian commented on KAFKA-4893: [~soumabrata] Thanks for checking. The solution has not

[jira] [Commented] (KAFKA-4950) ConcurrentModificationException when iterating over Kafka Metrics

2017-04-20 Thread Vahid Hashemian (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15977116#comment-15977116 ] Vahid Hashemian commented on KAFKA-4950: [~dpostoronca] Would you mind sharing a bit more of your

[jira] [Updated] (KAFKA-5096) Only log invalid user configs and overwrite with correct one

2017-04-20 Thread Matthias J. Sax (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-5096: --- Summary: Only log invalid user configs and overwrite with correct one (was: Check Streams

[jira] [Commented] (KAFKA-5089) JAR mismatch in KafkaConnect leads to NoSuchMethodError in HDP 2.6

2017-04-20 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15977051#comment-15977051 ] Ewen Cheslack-Postava commented on KAFKA-5089: -- Most of that CLASSPATH looks ok -- it's

Re: [DISCUSS] KIP 130: Expose states of active tasks to KafkaStreams public API

2017-04-20 Thread Eno Thereska
Hi there, Can't toString() just stay as is? Thanks Eno > On 20 Apr 2017, at 17:26, Matthias J. Sax wrote: > > Florian, > > I am just wondering: if we keep .toString(), what should the > implementation look like? > > > -Matthias > > On 4/19/17 2:42 PM, Florian

[jira] [Updated] (KAFKA-5092) KIP 141 - ProducerRecordBuilder Interface

2017-04-20 Thread Matthias J. Sax (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-5092: --- Labels: kip (was: ) > KIP 141 - ProducerRecordBuilder Interface >

Re: [DISCUSS] KIP 130: Expose states of active tasks to KafkaStreams public API

2017-04-20 Thread Matthias J. Sax
Florian, I am just wondering: if we keep .toString(), what should the implementation look like? -Matthias On 4/19/17 2:42 PM, Florian Hussonnois wrote: > Hi Matthias, > > So sorry for the delay in replying to you. For now, I think we can keep > KafkaStreams#toString() as it is. > It's always

Re: [DISCUSS] KIP 141 - ProducerRecordBuilder Interface

2017-04-20 Thread Matthias J. Sax
Thanks for the KIP! While I agree, that the current API is not perfect, I am not sure if a builder pattern does make sense here, because it's not too many parameters. IMHO, builder pattern makes sense if there are many optional parameters. For a ProducerRecord, I think there are only 2 optional

[jira] [Created] (KAFKA-5097) KafkaConsumer.poll throws IllegalStateException

2017-04-20 Thread Ismael Juma (JIRA)
Ismael Juma created KAFKA-5097: -- Summary: KafkaConsumer.poll throws IllegalStateException Key: KAFKA-5097 URL: https://issues.apache.org/jira/browse/KAFKA-5097 Project: Kafka Issue Type: Bug

[jira] [Updated] (KAFKA-4755) SimpleBenchmark test fails for streams

2017-04-20 Thread Eno Thereska (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eno Thereska updated KAFKA-4755: Fix Version/s: 0.10.2.1 > SimpleBenchmark test fails for streams >

[jira] [Updated] (KAFKA-4755) SimpleBenchmark test fails for streams

2017-04-20 Thread Eno Thereska (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eno Thereska updated KAFKA-4755: Summary: SimpleBenchmark test fails for streams (was: SimpleBenchmark consume test fails for

[jira] [Reopened] (KAFKA-4980) testReprocessingFromScratch unit test failure

2017-04-20 Thread Matthias J. Sax (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax reopened KAFKA-4980: Reopening this, as the fix should get cherry-picked to {{0.10.2}}:

[jira] [Assigned] (KAFKA-5096) Check Streams user config parameter earlier

2017-04-20 Thread Mariam John (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mariam John reassigned KAFKA-5096: -- Assignee: Mariam John > Check Streams user config parameter earlier >

[jira] [Created] (KAFKA-5096) Check Streams user config parameter earlier

2017-04-20 Thread Matthias J. Sax (JIRA)
Matthias J. Sax created KAFKA-5096: -- Summary: Check Streams user config parameter earlier Key: KAFKA-5096 URL: https://issues.apache.org/jira/browse/KAFKA-5096 Project: Kafka Issue Type:

[jira] [Commented] (KAFKA-5094) Censor SCRAM config change logging

2017-04-20 Thread JIRA
[ https://issues.apache.org/jira/browse/KAFKA-5094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15976744#comment-15976744 ] Johan Ström commented on KAFKA-5094: [~ijuma] My scala is pretty basic.. But from what I think I read,

  1   2   >