Build failed in Jenkins: Kafka-trunk #395

2015-02-18 Thread Apache Jenkins Server
See https://builds.apache.org/job/Kafka-trunk/395/changes Changes: [jjkoshy] KAFKA-1959; Rename group to groupId in TestOffsetManager due to collision with Thread.group in IBM's JDK; reviewed by Joel Koshy and Gwen Shapira [jjkoshy] KAFKA-1960; .gitignore does not exclude test generated files

Re: [jira] [Resolved] (KAFKA-1959) Class CommitThread overwrite group of Thread class causing compile errors

2015-02-18 Thread Tong Li
Joel, thanks so much for merging the two sets. Tong Sent from my iPhone On Feb 18, 2015, at 2:55 AM, Joel Koshy (JIRA) j...@apache.org wrote: [ https://issues.apache.org/jira/browse/KAFKA-1959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy resolved

Build failed in Jenkins: KafkaPreCommit #8

2015-02-18 Thread Apache Jenkins Server
See https://builds.apache.org/job/KafkaPreCommit/8/changes Changes: [jjkoshy] KAFKA-1959; Rename group to groupId in TestOffsetManager due to collision with Thread.group in IBM's JDK; reviewed by Joel Koshy and Gwen Shapira [jjkoshy] KAFKA-1960; .gitignore does not exclude test generated

Re: [DISCUSS] KIP-4 - Command line and centralized administrative operations

2015-02-18 Thread Jay Kreps
Hey Andrii, Generally we can do good error handling without needing custom server-side messages. I.e. generally the client has the context to know that if it got an error that the topic doesn't exist to say Topic X doesn't exist rather than error code 14 (or whatever). Maybe there are specific

Re: [DISCUSS] KIP-4 - Command line and centralized administrative operations

2015-02-18 Thread Andrii Biletskyi
Jay, Re error messages: you are right, in most cases client will have enough context to show descriptive error message. My concern is that we will have to add lots of new error codes for each possible error. Of course, we could reuse some of existing like UknownTopicOrPartitionCode, but we will

[jira] [Commented] (KAFKA-1887) controller error message on shutting the last broker

2015-02-18 Thread Helena Edelson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14326455#comment-14326455 ] Helena Edelson commented on KAFKA-1887: --- I see this consistently on shutdown in

[jira] [Updated] (KAFKA-1852) OffsetCommitRequest can commit offset on unknown topic

2015-02-18 Thread Sriharsha Chintalapani (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sriharsha Chintalapani updated KAFKA-1852: -- Attachment: KAFKA-1852_2015-02-18_13:13:17.patch OffsetCommitRequest can

[jira] [Updated] (KAFKA-1964) testPartitionReassignmentCallback hangs occasionally

2015-02-18 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-1964: --- Attachment: stack.out Saw the test hang once in trunk. Attached is the full stacktrace. Test worker prio=5

[jira] [Updated] (KAFKA-1952) High CPU Usage in 0.8.2 release

2015-02-18 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-1952: --- Resolution: Fixed Status: Resolved (was: Patch Available) Thanks for the review. Committed to

[jira] [Created] (KAFKA-1964) testPartitionReassignmentCallback hangs occasionally

2015-02-18 Thread Jun Rao (JIRA)
Jun Rao created KAFKA-1964: -- Summary: testPartitionReassignmentCallback hangs occasionally Key: KAFKA-1964 URL: https://issues.apache.org/jira/browse/KAFKA-1964 Project: Kafka Issue Type: Bug

[jira] [Comment Edited] (KAFKA-1887) controller error message on shutting the last broker

2015-02-18 Thread Helena Edelson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14326455#comment-14326455 ] Helena Edelson edited comment on KAFKA-1887 at 2/18/15 7:57 PM:

Re: Review Request 29912: Patch for KAFKA-1852

2015-02-18 Thread Sriharsha Chintalapani
On Feb. 13, 2015, 7:01 p.m., Joel Koshy wrote: core/src/main/scala/kafka/server/OffsetManager.scala, line 215 https://reviews.apache.org/r/29912/diff/3/?file=862699#file862699line215 Minor comment. I think this may be better to pass in to the OffsetManager. We should

[jira] [Commented] (KAFKA-1852) OffsetCommitRequest can commit offset on unknown topic

2015-02-18 Thread Sriharsha Chintalapani (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14326566#comment-14326566 ] Sriharsha Chintalapani commented on KAFKA-1852: --- Updated reviewboard

Re: Review Request 29912: Patch for KAFKA-1852

2015-02-18 Thread Sriharsha Chintalapani
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29912/ --- (Updated Feb. 18, 2015, 9:13 p.m.) Review request for kafka. Bugs:

[jira] [Created] (KAFKA-1965) Leaner DelayedItem

2015-02-18 Thread Yasuhiro Matsuda (JIRA)
Yasuhiro Matsuda created KAFKA-1965: --- Summary: Leaner DelayedItem Key: KAFKA-1965 URL: https://issues.apache.org/jira/browse/KAFKA-1965 Project: Kafka Issue Type: Improvement

Re: Review Request 30570: Patch for KAFKA-1914

2015-02-18 Thread Jun Rao
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30570/#review73020 --- Thanks for the patch. A couple of comments. 1. We already measure

[jira] [Updated] (KAFKA-1965) Leaner DelayedItem

2015-02-18 Thread Yasuhiro Matsuda (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yasuhiro Matsuda updated KAFKA-1965: Status: Patch Available (was: Open) Leaner DelayedItem --

[jira] [Updated] (KAFKA-1965) Leaner DelayedItem

2015-02-18 Thread Yasuhiro Matsuda (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yasuhiro Matsuda updated KAFKA-1965: Description: In DelayedItem, which is a superclass of DelayedOperation, both the creation

Re: Review Request 30570: Patch for KAFKA-1914

2015-02-18 Thread Joel Koshy
On Feb. 18, 2015, 10:16 p.m., Jun Rao wrote: Thanks for the patch. A couple of comments. 1. We already measure the total produce/fetch request rate in RequestMetrics. Now, we are duplicating that in BrokerTopics. Is there a way to avoid the duplication? 2. The following unit test

[jira] [Commented] (KAFKA-1788) producer record can stay in RecordAccumulator forever if leader is no available

2015-02-18 Thread Parth Brahmbhatt (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14326657#comment-14326657 ] Parth Brahmbhatt commented on KAFKA-1788: - Can someone from the kafka team respond

[jira] [Updated] (KAFKA-1965) Leaner DelayedItem

2015-02-18 Thread Yasuhiro Matsuda (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yasuhiro Matsuda updated KAFKA-1965: Description: In DelayedItem, which is a superclass of DelayedOperation, both the creation

[jira] [Updated] (KAFKA-1965) Leaner DelayedItem

2015-02-18 Thread Yasuhiro Matsuda (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yasuhiro Matsuda updated KAFKA-1965: Attachment: KAFKA-1965.patch Leaner DelayedItem -- Key:

[jira] [Updated] (KAFKA-1965) Leaner DelayedItem

2015-02-18 Thread Yasuhiro Matsuda (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yasuhiro Matsuda updated KAFKA-1965: Status: Patch Available (was: Open) Leaner DelayedItem --

[jira] [Updated] (KAFKA-1965) Leaner DelayedItem

2015-02-18 Thread Yasuhiro Matsuda (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yasuhiro Matsuda updated KAFKA-1965: Status: Open (was: Patch Available) Leaner DelayedItem --

Re: [VOTE] 0.8.2.1 Candidate 1

2015-02-18 Thread Matt Narrell
+1 On Feb 18, 2015, at 7:56 PM, Jun Rao j...@confluent.io wrote: This is the first candidate for release of Apache Kafka 0.8.2.1. This only fixes one critical issue (KAFKA-1952) in 0.8.2.0. Release Notes for the 0.8.2.1 release

Re: [VOTE] 0.8.2.1 Candidate 1

2015-02-18 Thread Connie Yang
+1 On Feb 18, 2015 7:23 PM, Matt Narrell matt.narr...@gmail.com wrote: +1 On Feb 18, 2015, at 7:56 PM, Jun Rao j...@confluent.io wrote: This is the first candidate for release of Apache Kafka 0.8.2.1. This only fixes one critical issue (KAFKA-1952) in 0.8.2.0. Release Notes for the

[jira] [Commented] (KAFKA-1965) Leaner DelayedItem

2015-02-18 Thread Sriharsha Chintalapani (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14326973#comment-14326973 ] Sriharsha Chintalapani commented on KAFKA-1965: --- [~yasuhiro.matsuda] Changes

Re: Review Request 29301: Patch for KAFKA-1694

2015-02-18 Thread Andrii Biletskyi
On Feb. 3, 2015, 7:14 p.m., Guozhang Wang wrote: core/src/main/scala/kafka/server/TopicCommandHelper.scala, lines 1-17 https://reviews.apache.org/r/29301/diff/7/?file=821380#file821380line1 One general comment: For some topic commands, why use AdminUtils to write ZK

Re: [DISCUSS] KIP-4 - Command line and centralized administrative operations

2015-02-18 Thread Andrii Biletskyi
Hi all, I'm trying to address some of the issues which were mentioned earlier about Admin RQ/RP format. One of those was about batching operations. What if we follow TopicCommand approach and let people specify topic-name by regexp - would that cover most of the use cases? Secondly, is what

[jira] [Commented] (KAFKA-1694) kafka command line and centralized operations

2015-02-18 Thread Andrii Biletskyi (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14326171#comment-14326171 ] Andrii Biletskyi commented on KAFKA-1694: - [~guozhang], yes, initially I did split

[jira] [Issue Comment Deleted] (KAFKA-1694) kafka command line and centralized operations

2015-02-18 Thread Andrii Biletskyi (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrii Biletskyi updated KAFKA-1694: Comment: was deleted (was: [~guozhang], yes, initially I did split it to several patches.

[jira] [Commented] (KAFKA-1694) kafka command line and centralized operations

2015-02-18 Thread Andrii Biletskyi (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14326172#comment-14326172 ] Andrii Biletskyi commented on KAFKA-1694: - [~guozhang], yes, initially I did split

Build failed in Jenkins: Kafka-trunk #396

2015-02-18 Thread Apache Jenkins Server
See https://builds.apache.org/job/Kafka-trunk/396/changes Changes: [junrao] kafka-1952; High CPU Usage in 0.8.2 release; patched by Jun Rao; reviewed by Guozhang Wang, Ewen Cheslack-Postava and Neha Narkhede [jjkoshy] KAFKA-1914; follow-up to address unit test failure

[VOTE] KIP-8: Add a flush() method to the new producer

2015-02-18 Thread Jay Kreps
https://cwiki.apache.org/confluence/display/KAFKA/KIP-8+-+Add+a+flush+method+to+the+producer+API +1 binding -Jay

Re: Review Request 31169: Patch for KAFKA-1729

2015-02-18 Thread Jun Rao
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31169/#review73057 --- core/src/main/scala/kafka/javaapi/OffsetFetchRequest.scala

Build failed in Jenkins: Kafka-trunk #397

2015-02-18 Thread Apache Jenkins Server
See https://builds.apache.org/job/Kafka-trunk/397/changes Changes: [jjkoshy] KAFKA-1914; follow-up to fix SimpleFetchTest; reviewed by Joel Koshy -- [...truncated 1269 lines...] kafka.producer.AsyncProducerTest testPartitionAndCollateEvents PASSED

Re: Review Request 31168: Patch for KAFKA-1914

2015-02-18 Thread Aditya Auradkar
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31168/ --- (Updated Feb. 19, 2015, 12:01 a.m.) Review request for kafka. Bugs:

Re: Review Request 31169: Patch for KAFKA-1729

2015-02-18 Thread Jun Rao
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31169/#review73060 --- Ship it! Ship It! - Jun Rao On Feb. 19, 2015, 1:30 a.m., Joel

Re: [VOTE] KIP-8: Add a flush() method to the new producer

2015-02-18 Thread Joe Stein
+1 binding ~ Joestein On Feb 18, 2015 6:50 PM, Jay Kreps j...@confluent.io wrote: https://cwiki.apache.org/confluence/display/KAFKA/KIP-8+-+Add+a+flush+method+to+the+producer+API +1 binding -Jay

[jira] [Assigned] (KAFKA-1546) Automate replica lag tuning

2015-02-18 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aditya Auradkar reassigned KAFKA-1546: -- Assignee: Aditya Auradkar (was: nicu marasoiu) Automate replica lag tuning

[jira] [Commented] (KAFKA-1546) Automate replica lag tuning

2015-02-18 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14326773#comment-14326773 ] Aditya Auradkar commented on KAFKA-1546: [~nmarasoi] I'm going to assign this to

Re: [DISCUSS] KIP-8 Add a flush method to the new Java producer

2015-02-18 Thread Joel Koshy
Actually, could you clarify this a bit (since I'm not sure which thread you are referring to) - specifically, how would this tie in with the current timeout we have for the producer (for example)? On Tue, Feb 17, 2015 at 02:55:44PM -0800, Jay Kreps wrote: Yeah there was a separate thread on

Build failed in Jenkins: KafkaPreCommit #10

2015-02-18 Thread Apache Jenkins Server
See https://builds.apache.org/job/KafkaPreCommit/10/changes Changes: [jjkoshy] KAFKA-1914; follow-up to fix SimpleFetchTest; reviewed by Joel Koshy -- [...truncated 314 lines...] kafka.log.LogManagerTest testCheckpointRecoveryPoints PASSED

Jenkins build is back to normal : Kafka-trunk #398

2015-02-18 Thread Apache Jenkins Server
See https://builds.apache.org/job/Kafka-trunk/398/changes

[VOTE] 0.8.2.1 Candidate 1

2015-02-18 Thread Jun Rao
This is the first candidate for release of Apache Kafka 0.8.2.1. This only fixes one critical issue (KAFKA-1952) in 0.8.2.0. Release Notes for the 0.8.2.1 release https://people.apache.org/~junrao/kafka-0.8.2.1-candidate1/RELEASE_NOTES.html *** Please download, test and vote by Saturday, Feb 21,

Re: Review Request 31169: Patch for KAFKA-1729

2015-02-18 Thread Joel Koshy
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31169/#review73056 --- core/src/main/scala/kafka/javaapi/OffsetFetchRequest.scala

Review Request 31168: Patch for KAFKA-1914

2015-02-18 Thread Aditya Auradkar
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31168/ --- Review request for kafka. Bugs: KAFKA-1914

Re: Review Request 31168: Patch for KAFKA-1914

2015-02-18 Thread Aditya Auradkar
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31168/ --- (Updated Feb. 18, 2015, 11:55 p.m.) Review request for kafka. Bugs:

[jira] [Updated] (KAFKA-1729) add doc for Kafka-based offset management in 0.8.2

2015-02-18 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy updated KAFKA-1729: -- Attachment: KAFKA-1729.patch add doc for Kafka-based offset management in 0.8.2

Review Request 31169: Patch for KAFKA-1729

2015-02-18 Thread Joel Koshy
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31169/ --- Review request for kafka. Bugs: KAFKA-1729

Re: Review Request 31169: Patch for KAFKA-1729

2015-02-18 Thread Joel Koshy
On Feb. 19, 2015, 1:32 a.m., Jun Rao wrote: core/src/main/scala/kafka/javaapi/OffsetFetchRequest.scala, line 47 https://reviews.apache.org/r/31169/diff/2/?file=868598#file868598line47 Should we remove the comment? Thanks - yes will do that on check-in if everything else looks good

Re: Review Request 31169: Patch for KAFKA-1729

2015-02-18 Thread Jun Rao
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31169/#review73053 --- core/src/main/scala/kafka/javaapi/OffsetFetchRequest.scala

Review Request 31174: Patch for KAFKA-1729

2015-02-18 Thread Joel Koshy
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31174/ --- Review request for kafka. Bugs: KAFKA-1729

[jira] [Commented] (KAFKA-1914) Count TotalProduceRequestRate and TotalFetchRequestRate in BrokerTopicMetrics

2015-02-18 Thread Aditya A Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14326763#comment-14326763 ] Aditya A Auradkar commented on KAFKA-1914: -- Created reviewboard

[jira] [Commented] (KAFKA-1729) add doc for Kafka-based offset management in 0.8.2

2015-02-18 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14326764#comment-14326764 ] Joel Koshy commented on KAFKA-1729: --- Created reviewboard

[jira] [Updated] (KAFKA-1914) Count TotalProduceRequestRate and TotalFetchRequestRate in BrokerTopicMetrics

2015-02-18 Thread Aditya A Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aditya A Auradkar updated KAFKA-1914: - Attachment: KAFKA-1914.patch Count TotalProduceRequestRate and TotalFetchRequestRate in

Re: Review Request 31168: Patch for KAFKA-1914

2015-02-18 Thread Joel Koshy
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31168/#review73035 --- Ship it! Ship It! - Joel Koshy On Feb. 19, 2015, 12:01 a.m.,

Re: Review Request 30809: Patch for KAFKA-1888

2015-02-18 Thread Abhishek Nigam
On Feb. 18, 2015, 12:06 a.m., Mayuresh Gharat wrote: core/src/main/scala/kafka/tools/ContinuousValidationTest.java, line 207 https://reviews.apache.org/r/30809/diff/1/?file=859055#file859055line207 This might end up in infinite loop if something goes wrong with cluster, right?

[jira] [Updated] (KAFKA-1729) add doc for Kafka-based offset management in 0.8.2

2015-02-18 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy updated KAFKA-1729: -- Attachment: KAFKA-1729_2015-02-18_17:30:37.patch add doc for Kafka-based offset management in 0.8.2

Re: Review Request 31169: Patch for KAFKA-1729

2015-02-18 Thread Joel Koshy
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31169/ --- (Updated Feb. 19, 2015, 1:30 a.m.) Review request for kafka. Bugs:

Re: [DISCUSS] KIP-8 Add a flush method to the new Java producer

2015-02-18 Thread Jay Kreps
There have been a couple of rounds of this. Basically a bunch of complaints people have about the producer boil down to their being no limit on how long a request will block if the kafka cluster goes hard down. Some of the discussion was here, I think:

Re: [VOTE] KIP-8: Add a flush() method to the new producer

2015-02-18 Thread Guozhang Wang
+1. On Wed, Feb 18, 2015 at 3:49 PM, Jay Kreps j...@confluent.io wrote: https://cwiki.apache.org/confluence/display/KAFKA/KIP-8+-+Add+a+flush+method+to+the+producer+API +1 binding -Jay -- -- Guozhang