[jira] [Comment Edited] (KAFKA-502) Simplify setup / initialization in replication_basic_test.py

2012-10-04 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13469960#comment-13469960 ] Jun Rao edited comment on KAFKA-502 at 10/5/12 12:43 PM: - Thanks fo

[jira] [Commented] (KAFKA-502) Simplify setup / initialization in replication_basic_test.py

2012-10-04 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13469960#comment-13469960 ] Jun Rao commented on KAFKA-502: --- Thanks for patch v4. A comment and a couple of questions: 4

[jira] [Updated] (KAFKA-510) broker needs to know the replication factor per partition

2012-10-04 Thread Yang Ye (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Ye updated KAFKA-510: -- Attachment: kafka_510_v3.diff Joel, Thanks for comment, this part is changes in patch v3, potentially some oth

[jira] [Updated] (KAFKA-267) Enhance ProducerPerformance to generate unique random Long value for payload

2012-10-04 Thread Yang Ye (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Ye updated KAFKA-267: -- Attachment: kafka-267-v2.patch Refactor a log of code 1. support mutliple topics 2. support various sized mess

[jira] [Commented] (KAFKA-510) broker needs to know the replication factor per partition

2012-10-04 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13469950#comment-13469950 ] Joel Koshy commented on KAFKA-510: -- Looks good - one minor change I forgot to mention is t

[jira] [Commented] (KAFKA-506) Store logical offset in log

2012-10-04 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13469949#comment-13469949 ] Joel Koshy commented on KAFKA-506: -- Rebasing is painful for sure, especially since 0.8 is

[jira] [Commented] (KAFKA-506) Store logical offset in log

2012-10-04 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13469945#comment-13469945 ] Jun Rao commented on KAFKA-506: --- What we can do is to hold off committing other conflicting p

[jira] [Comment Edited] (KAFKA-514) Replication with Leader Failure Test: Log segment files checksum mismatch

2012-10-04 Thread John Fung (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13469833#comment-13469833 ] John Fung edited comment on KAFKA-514 at 10/5/12 10:34 AM: --- This

[jira] [Updated] (KAFKA-514) Replication with Leader Failure Test: Log segment files checksum mismatch

2012-10-04 Thread John Fung (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Fung updated KAFKA-514: Attachment: (was: system_test_output_archive.tar) > Replication with Leader Failure Test: Log segmen

[jira] [Updated] (KAFKA-514) Replication with Leader Failure Test: Log segment files checksum mismatch

2012-10-04 Thread John Fung (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Fung updated KAFKA-514: Attachment: system_test_output_archive.tar.gz > Replication with Leader Failure Test: Log segment files

[jira] [Commented] (KAFKA-508) split out partiondata from fetchresponse and producerrequest

2012-10-04 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13469835#comment-13469835 ] Jun Rao commented on KAFKA-508: --- Joe, thanks for the patch. Looks good overall. I agree with

[jira] [Commented] (KAFKA-514) Replication with Leader Failure Test: Log segment files checksum mismatch

2012-10-04 Thread John Fung (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13469833#comment-13469833 ] John Fung commented on KAFKA-514: - This issue can be reproduced as follows: 1. Download th

[jira] [Updated] (KAFKA-514) Replication with Leader Failure Test: Log segment files checksum mismatch

2012-10-04 Thread John Fung (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Fung updated KAFKA-514: Attachment: testcase_2.tar > Replication with Leader Failure Test: Log segment files checksum mismatch >

[jira] [Updated] (KAFKA-514) Replication with Leader Failure Test: Log segment files checksum mismatch

2012-10-04 Thread John Fung (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Fung updated KAFKA-514: Attachment: system_test_output_archive.tar > Replication with Leader Failure Test: Log segment files che

[jira] [Updated] (KAFKA-502) Simplify setup / initialization in replication_basic_test.py

2012-10-04 Thread John Fung (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Fung updated KAFKA-502: Attachment: kafka-502-v4.patch Thank you Jun for reviewing the patch. Uploaded kafka-502-v4.patch with the

[jira] [Assigned] (KAFKA-496) high level producer send should return a response

2012-10-04 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps reassigned KAFKA-496: --- Assignee: Jay Kreps > high level producer send should return a response >

[jira] [Commented] (KAFKA-506) Store logical offset in log

2012-10-04 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13469794#comment-13469794 ] Jay Kreps commented on KAFKA-506: - It is really hard/error-prone to keep this patch alive a

[jira] [Assigned] (KAFKA-409) Refactor DefaultEventHandler

2012-10-04 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps reassigned KAFKA-409: --- Assignee: Jay Kreps > Refactor DefaultEventHandler > - > >

[jira] [Commented] (KAFKA-506) Store logical offset in log

2012-10-04 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13469780#comment-13469780 ] Neha Narkhede commented on KAFKA-506: - I will free up tomorrow after Grace Hopper confe

[jira] [Commented] (KAFKA-508) split out partiondata from fetchresponse and producerrequest

2012-10-04 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13469754#comment-13469754 ] Joel Koshy commented on KAFKA-508: -- +1 - can you remove the extends from the two objects?

[jira] [Commented] (KAFKA-506) Store logical offset in log

2012-10-04 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13469726#comment-13469726 ] Jay Kreps commented on KAFKA-506: - 50. It can actually only take Int values, so I don't thi

[jira] [Updated] (KAFKA-508) split out partiondata from fetchresponse and producerrequest

2012-10-04 Thread Joe Stein (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Stein updated KAFKA-508: Attachment: KAFKA-508.v2.patch BA patch attached for name change to be ProducerRequestPartitionData and Fe

[jira] [Commented] (KAFKA-508) split out partiondata from fetchresponse and producerrequest

2012-10-04 Thread Joe Stein (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13469660#comment-13469660 ] Joe Stein commented on KAFKA-508: - << Actually, do you think we can also get rid of the par

[jira] [Commented] (KAFKA-506) Store logical offset in log

2012-10-04 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13469655#comment-13469655 ] Jun Rao commented on KAFKA-506: --- Thanks for patch v5. 50. There is still a potential issue

[jira] [Commented] (KAFKA-531) kafka.server.ReplicaManager: java.nio.channels.NonWritableChannelException

2012-10-04 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13469652#comment-13469652 ] Jay Kreps commented on KAFKA-531: - I think the root cause here is that when we re-open a lo

producer changes

2012-10-04 Thread Jay Kreps
Hey guys, Two quick things: 1. There is a proposed change to ProducerData to allow us to retain the partition key with the message here: https://issues.apache.org/jira/browse/KAFKA-544 This would also mean changing the Encoder/Decoder interface. Since this impacts the user-facing APIs I think i

[jira] [Updated] (KAFKA-521) Refactor Log subsystem

2012-10-04 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-521: Description: There are a number of items it would be nice to cleanup in the log subsystem: 1. Misc. funky ap

[jira] [Created] (KAFKA-546) Fix commit() in zk consumer

2012-10-04 Thread Jay Kreps (JIRA)
Jay Kreps created KAFKA-546: --- Summary: Fix commit() in zk consumer Key: KAFKA-546 URL: https://issues.apache.org/jira/browse/KAFKA-546 Project: Kafka Issue Type: New Feature Affects Versions: 0

[jira] [Created] (KAFKA-545) Add a Performance Suite for the Log subsystem

2012-10-04 Thread Jay Kreps (JIRA)
Jay Kreps created KAFKA-545: --- Summary: Add a Performance Suite for the Log subsystem Key: KAFKA-545 URL: https://issues.apache.org/jira/browse/KAFKA-545 Project: Kafka Issue Type: New Feature A

[jira] [Created] (KAFKA-544) Retain key in producer

2012-10-04 Thread Jay Kreps (JIRA)
Jay Kreps created KAFKA-544: --- Summary: Retain key in producer Key: KAFKA-544 URL: https://issues.apache.org/jira/browse/KAFKA-544 Project: Kafka Issue Type: Bug Affects Versions: 0.8

[jira] [Commented] (KAFKA-543) Metadata request from DefaultEventHandler.handle repeats same topic over and over

2012-10-04 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13469604#comment-13469604 ] Jun Rao commented on KAFKA-543: --- Thanks for the patch. Good catch. +1 > Meta

[jira] [Updated] (KAFKA-543) Metadata request from DefaultEventHandler.handle repeats same topic over and over

2012-10-04 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-543: Attachment: KAFKA-543.patch This patch dedupes in DefaultEventHandler which fixes the issue. It also proact

[jira] [Commented] (KAFKA-508) split out partiondata from fetchresponse and producerrequest

2012-10-04 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13469566#comment-13469566 ] Joel Koshy commented on KAFKA-508: -- Joe, thanks for the patch. Actually, do you think we

[jira] [Commented] (KAFKA-456) ProducerSendThread calls ListBuffer.size a whole bunch. That is a O(n) operation

2012-10-04 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13469560#comment-13469560 ] Jay Kreps commented on KAFKA-456: - Why not just make this an ArrayBuffer so that there are

[jira] [Updated] (KAFKA-456) ProducerSendThread calls ListBuffer.size a whole bunch. That is a O(n) operation

2012-10-04 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-456: -- Resolution: Fixed Assignee: David Arthur Status: Resolved (was: Patch Available) > ProducerSe

[jira] [Commented] (KAFKA-456) ProducerSendThread calls ListBuffer.size a whole bunch. That is a O(n) operation

2012-10-04 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13469530#comment-13469530 ] Jun Rao commented on KAFKA-456: --- Thanks for the patch. Committed to 0.8. This performance is

[jira] [Updated] (KAFKA-542) Upgrade to metrics 3.x

2012-10-04 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy updated KAFKA-542: - Status: Patch Available (was: Open) I heard back from metrics-user that there is no immediate plan to rel

[jira] [Commented] (KAFKA-541) Use metrics CSV reporter instead of jmx tool for system tests

2012-10-04 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13469527#comment-13469527 ] Joel Koshy commented on KAFKA-541: -- That's right - we would just use metrics.json, althoug

[jira] [Updated] (KAFKA-506) Store logical offset in log

2012-10-04 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-506: Attachment: KAFKA-506-phase-2-v5.patch Rebased again and fixed the above issues to make v5 50. I looked int

[jira] [Commented] (KAFKA-474) support changing host/port of a broker

2012-10-04 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13469522#comment-13469522 ] Jun Rao commented on KAFKA-474: --- Yes, fetcherSourceBroker is actually no longer needed. It wa

[jira] [Commented] (KAFKA-502) Simplify setup / initialization in replication_basic_test.py

2012-10-04 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13469506#comment-13469506 ] Jun Rao commented on KAFKA-502: --- Thanks for patch v3. This looks great. 0. Do all new tests

[jira] [Updated] (KAFKA-456) ProducerSendThread calls ListBuffer.size a whole bunch. That is a O(n) operation

2012-10-04 Thread David Arthur (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Arthur updated KAFKA-456: --- Affects Version/s: (was: 0.7.2) (was: 0.7.1) Status: Patc

[jira] [Updated] (KAFKA-456) ProducerSendThread calls ListBuffer.size a whole bunch. That is a O(n) operation

2012-10-04 Thread David Arthur (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Arthur updated KAFKA-456: --- Attachment: KAFKA-456.patch I looked around in other parts of the code for similar situations (lots of

[jira] [Commented] (KAFKA-474) support changing host/port of a broker

2012-10-04 Thread David Arthur (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13469333#comment-13469333 ] David Arthur commented on KAFKA-474: I took a look at AbstractFetcherManager and the br

[jira] Subscription: outstanding kafka patches

2012-10-04 Thread jira
Issue Subscription Filter: outstanding kafka patches (57 issues) The list of outstanding kafka patches Subscriber: kafka-mailing-list Key Summary KAFKA-542 Upgrade to metrics 3.x https://issues.apache.org/jira/browse/KAFKA-542 KAFKA-532 Multiple controllers can co-e