Re: mailing list moderators

2012-02-16 Thread Jun Rao
w/ their subscription/unsubscribe problems. I want to make sure that more than the mentors can do this so it doesn't fall through the cracks when you guys graduate. Regards, Alan On Feb 16, 2012, at 8:37 AM, Jun Rao wrote: Alan, What do the moderators do? Thanks, Jun

[jira] [Commented] (KAFKA-265) Add a queue of zookeeper notifications in the zookeeper consumer to reduce the number of rebalancing attempts

2012-02-16 Thread Jun Rao (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13209992#comment-13209992 ] Jun Rao commented on KAFKA-265: --- Committed the fix for both Condition and shutdown to trunk

[jira] [Commented] (KAFKA-274) Handle corrupted messages cleanly

2012-02-16 Thread Jun Rao (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13209993#comment-13209993 ] Jun Rao commented on KAFKA-274: --- +1 for the patch. Handle corrupted

[jira] [Commented] (KAFKA-240) implement new producer and consumer request format

2012-02-15 Thread Jun Rao (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13208691#comment-13208691 ] Jun Rao commented on KAFKA-240: --- Thanks Prashanth, excellent patch. Committed Validation v1

[jira] [Updated] (KAFKA-272) add JMX on broker to track bytes/messages per topic

2012-02-15 Thread Jun Rao (Updated) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-272: -- Attachment: kafka-272_v2.patch Upload patch v2. Add bytesOut. MessagesOut is a bit hard since currently we

[jira] [Updated] (KAFKA-272) add JMX on broker to track bytes/messages per topic

2012-02-15 Thread Jun Rao (Updated) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-272: -- Resolution: Fixed Fix Version/s: 0.7.1 Status: Resolved (was: Patch Available) Thanks

[jira] [Updated] (KAFKA-265) Add a queue of zookeeper notifications in the zookeeper consumer to reduce the number of rebalancing attempts

2012-02-15 Thread Jun Rao (Updated) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-265: -- Attachment: kafka-265_await_fix.patch Found a bug. Condition should use await/notify, instead of wait/notify

[jira] [Created] (KAFKA-272) add JMX on broker to track bytes/messages per topic

2012-02-14 Thread Jun Rao (Created) (JIRA)
Affects Versions: 0.7 Reporter: Jun Rao Assignee: Jun Rao -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa

[jira] [Updated] (KAFKA-272) add JMX on broker to track bytes/messages per topic

2012-02-14 Thread Jun Rao (Updated) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-272: -- Status: Patch Available (was: Open) add JMX on broker to track bytes/messages per topic

[jira] [Commented] (KAFKA-240) implement new producer and consumer request format

2012-02-13 Thread Jun Rao (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13206927#comment-13206927 ] Jun Rao commented on KAFKA-240: --- Prashanth, Another thing. We don't enforce that a given

[jira] [Commented] (KAFKA-268) Add another reconnection condition to the syncProducer: the time elapsed since last connection

2012-02-13 Thread Jun Rao (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13206955#comment-13206955 ] Jun Rao commented on KAFKA-268: --- Some comments: 1. SyncProducerConfigShared: change

[jira] [Commented] (KAFKA-270) sync producer / consumer test producing lot of kafka server exceptions not getting the throughput mentioned here http://incubator.apache.org/kafka/performance.html

2012-02-13 Thread Jun Rao (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13207023#comment-13207023 ] Jun Rao commented on KAFKA-270: --- A few things to try: 1. It seems that there is ZK session

[jira] [Updated] (KAFKA-265) Add a queue of zookeeper notifications in the zookeeper consumer to reduce the number of rebalancing attempts

2012-02-09 Thread Jun Rao (Updated) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-265: -- Attachment: kafka-265_v2.patch That's a good idea. Attach patch v2. Add a queue of zookeeper

[jira] [Commented] (KAFKA-47) Create topic support and new ZK data structures for intra-cluster replication

2012-02-08 Thread Jun Rao (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-47?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13203725#comment-13203725 ] Jun Rao commented on KAFKA-47: -- Prashanth, good question. Yes, we could continue using

[jira] [Commented] (KAFKA-262) Bug in the consumer rebalancing logic causes one consumer to release partitions that it does not own

2012-02-08 Thread Jun Rao (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13203748#comment-13203748 ] Jun Rao commented on KAFKA-262: --- Some comments: 1

[jira] [Commented] (KAFKA-262) Bug in the consumer rebalancing logic causes one consumer to release partitions that it does not own

2012-02-08 Thread Jun Rao (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13204140#comment-13204140 ] Jun Rao commented on KAFKA-262: --- 2. If releasePartitionOwnership always just checks

[jira] [Commented] (KAFKA-240) implement new producer and consumer request format

2012-02-08 Thread Jun Rao (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13204170#comment-13204170 ] Jun Rao commented on KAFKA-240: --- Prashanth, thanks for the patch. Looks good overall. 1

[jira] [Updated] (KAFKA-265) Add a queue of zookeeper notifications in the zookeeper consumer to reduce the number of rebalancing attempts

2012-02-08 Thread Jun Rao (Updated) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-265: -- Status: Patch Available (was: Open) patch attached. Add a queue of zookeeper notifications

[jira] [Updated] (KAFKA-265) Add a queue of zookeeper notifications in the zookeeper consumer to reduce the number of rebalancing attempts

2012-02-08 Thread Jun Rao (Updated) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-265: -- Attachment: kafka-265.patch Add a queue of zookeeper notifications in the zookeeper consumer to reduce

[jira] [Commented] (KAFKA-262) Bug in the consumer rebalancing logic causes one consumer to release partitions that it does not own

2012-02-08 Thread Jun Rao (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13204203#comment-13204203 ] Jun Rao commented on KAFKA-262: --- In releasePartitionOwnership

[jira] [Commented] (KAFKA-240) implement new producer and consumer request format

2012-02-08 Thread Jun Rao (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13204258#comment-13204258 ] Jun Rao commented on KAFKA-240: --- Also, for the builder for FetchRequest. My concern

[jira] [Commented] (KAFKA-253) Refactor the async producer to have only one queue instead of one queue per broker in a Kafka cluster

2012-02-07 Thread Jun Rao (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13203209#comment-13203209 ] Jun Rao commented on KAFKA-253: --- Thanks for the review. Committed to 0.8 branch. Will close

[jira] [Updated] (KAFKA-253) Refactor the async producer to have only one queue instead of one queue per broker in a Kafka cluster

2012-02-06 Thread Jun Rao (Updated) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-253: -- Attachment: kafka-253_v5.patch Attach patch v5. Minor changes in DefaultEventHandler to correctly handle

[jira] [Commented] (KAFKA-240) implement new producer and consumer request format

2012-02-03 Thread Jun Rao (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13199901#comment-13199901 ] Jun Rao commented on KAFKA-240: --- Joe, Prashanth, Do you guys have patches that we can review

[jira] [Updated] (KAFKA-261) Corrupted request shuts down the broker

2012-02-02 Thread Jun Rao (Updated) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-261: -- Resolution: Fixed Fix Version/s: 0.7.1 Status: Resolved (was: Patch Available) Just

[jira] [Updated] (KAFKA-253) Refactor the async producer to have only one queue instead of one queue per broker in a Kafka cluster

2012-02-02 Thread Jun Rao (Updated) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-253: -- Attachment: kafka-253_v2.patch Uploaded patch v2. 1. removed 2. made it case class 3.1 changed. 3.2 removed

[jira] [Commented] (KAFKA-251) The ConsumerStats MBean's PartOwnerStats attribute is a string

2012-02-02 Thread Jun Rao (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13199447#comment-13199447 ] Jun Rao commented on KAFKA-251: --- Sorry, just get the chance to look at this. Couldn't apply

[jira] [Commented] (KAFKA-254) A tool to GET Zookeeper partition-offset and output to files

2012-02-02 Thread Jun Rao (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13199458#comment-13199458 ] Jun Rao commented on KAFKA-254: --- Thanks for the patch. A few comments: 1. For consistency

[jira] [Updated] (KAFKA-253) Refactor the async producer to have only one queue instead of one queue per broker in a Kafka cluster

2012-02-02 Thread Jun Rao (Updated) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-253: -- Attachment: kafka-253_v3.patch Attached v3 patch by rebasing to trunk. Refactor the async

[jira] [Updated] (KAFKA-253) Refactor the async producer to have only one queue instead of one queue per broker in a Kafka cluster

2012-02-01 Thread Jun Rao (Updated) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-253: -- Status: Patch Available (was: Open) The patch can go into 0.7 too. Refactor the async

[jira] [Updated] (KAFKA-261) Corrupted request shuts down the broker

2012-02-01 Thread Jun Rao (Updated) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-261: -- Attachment: kafka-261.patch patch attached. Corrupted request shuts down the broker

[jira] [Created] (KAFKA-261) Corrupted request shuts down the broker

2012-02-01 Thread Jun Rao (Created) (JIRA)
Corrupted request shuts down the broker --- Key: KAFKA-261 URL: https://issues.apache.org/jira/browse/KAFKA-261 Project: Kafka Issue Type: Bug Affects Versions: 0.7 Reporter: Jun Rao

Re: ProducerPool - KAFKA-240

2012-01-31 Thread Jun Rao
Joe, For kafka-240, let's just focus on replacing SyncProducer with the new wire protocol. There is a separate jira kafka-253 on producer-side refactoring. We can defer other changes that you want to make there. Thanks, Jun On Mon, Jan 30, 2012 at 9:36 PM, Joe Stein crypt...@gmail.com wrote:

[jira] [Commented] (KAFKA-238) add a getTopicMetaData method in broker and expose it to producer

2012-01-24 Thread Jun Rao (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13192287#comment-13192287 ] Jun Rao commented on KAFKA-238: --- 3. I agree that the API is rarely used. However, each call

[jira] [Commented] (KAFKA-238) add a getTopicMetaData method in broker and expose it to producer

2012-01-24 Thread Jun Rao (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13192450#comment-13192450 ] Jun Rao commented on KAFKA-238: --- +1 on the patch. This patch doesn't wire the getMetaData

[jira] [Commented] (KAFKA-251) The ConsumerStats MBean's PartOwnerStats attribute is a string

2012-01-24 Thread Jun Rao (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13192736#comment-13192736 ] Jun Rao commented on KAFKA-251: --- Thanks for the patch. Some comments: 1

[jira] [Commented] (KAFKA-238) add a getTopicMetaData method in broker and expose it to producer

2012-01-23 Thread Jun Rao (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13191262#comment-13191262 ] Jun Rao commented on KAFKA-238: --- The patch doesn't apply to the 0.8 branch. Could you rebase

[jira] [Commented] (KAFKA-238) add a getTopicMetaData method in broker and expose it to producer

2012-01-23 Thread Jun Rao (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13191350#comment-13191350 ] Jun Rao commented on KAFKA-238: --- Some comments: 1. Use the new Logger helper for new log4j

[jira] [Commented] (KAFKA-41) multi-produce and multi-fetch support with replication

2012-01-23 Thread Jun Rao (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-41?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13191446#comment-13191446 ] Jun Rao commented on KAFKA-41: -- Yes, this may be covered by the new wire protocol change

[jira] [Commented] (KAFKA-240) implement new producer and consumer request format

2012-01-23 Thread Jun Rao (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13191471#comment-13191471 ] Jun Rao commented on KAFKA-240: --- Joe, The changes in github make sense. A couple

[jira] [Commented] (KAFKA-240) implement new producer and consumer request format

2012-01-18 Thread Jun Rao (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13188716#comment-13188716 ] Jun Rao commented on KAFKA-240: --- Joe, I think it could be simpler than that. You can have

[jira] [Updated] (KAFKA-241) ConsumerIterator throws a IllegalStateException after a ConsumerTimeout occurs

2012-01-18 Thread Jun Rao (Updated) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-241: -- Fix Version/s: 0.7.1 Assignee: Jun Rao Status: Patch Available (was: Open) Added a unit

[jira] [Updated] (KAFKA-247) max.message.size and fetch.size defaults should be consistent

2012-01-17 Thread Jun Rao (Updated) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-247: -- Resolution: Fixed Fix Version/s: 0.7.1 Assignee: Pierre-Yves Ritschard Status

[jira] [Resolved] (KAFKA-248) In the class kafka.producer.DefaultPartitionner the ligne key.hashCode % numPartitions may return a negative value. this is not a good thing knowing that partittion is (a

2012-01-16 Thread Jun Rao (Resolved) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao resolved KAFKA-248. --- Resolution: Duplicate Fix Version/s: 0.7 This is already fixed in 0.7. In the class

Re: skype meeting on replication roadmap

2012-01-16 Thread Jun Rao
Prashanth, Here is the meeting minutes. Here is an update on the Skype meeting on Jan 10: Attendees: Neha Narkhede Jay Kreps Joel Koshy Jun Rao Andrew.Psaltis Joe Stein David Ormsbee Jeffrey Damick Patricio Echagüe We went through the high level design of replication as described in kafka-50

[jira] [Created] (KAFKA-246) log configuration values used

2012-01-13 Thread Jun Rao (Created) (JIRA)
log configuration values used - Key: KAFKA-246 URL: https://issues.apache.org/jira/browse/KAFKA-246 Project: Kafka Issue Type: Improvement Reporter: Jun Rao Currently, it's hard to figure out which

[jira] [Updated] (KAFKA-237) create/delete ZK path for a topic in an admin tool

2012-01-13 Thread Jun Rao (Updated) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-237: -- Resolution: Fixed Fix Version/s: 0.8 Status: Resolved (was: Patch Available) thanks

[jira] [Commented] (KAFKA-245) upgrade to zkclient 0.1

2012-01-12 Thread Jun Rao (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13185177#comment-13185177 ] Jun Rao commented on KAFKA-245: --- Do we know the github revision/date corresponding

[jira] [Commented] (KAFKA-232) ConsumerConnector has no access to getOffsetsBefore

2012-01-12 Thread Jun Rao (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13185190#comment-13185190 ] Jun Rao commented on KAFKA-232: --- We discussed in the mailing before about allowing the ZK

[jira] [Updated] (KAFKA-237) create/delete ZK path for a topic in an admin tool

2012-01-12 Thread Jun Rao (Updated) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-237: -- Attachment: kafka-237-v3.patch Attach v3. 5. Return version id if create topic fails

[jira] [Commented] (KAFKA-240) implement new producer and consumer request format

2012-01-11 Thread Jun Rao (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13184455#comment-13184455 ] Jun Rao commented on KAFKA-240: --- Joe, That's a good question. We had some discussions

Skype meeting Jan 10

2012-01-11 Thread Jun Rao
Here is an update on the Skype meeting on Jan 10: Attendees: Neha Narkhede Jay Kreps Joel Koshy Jun Rao Andrew.Psaltis Joe Stein David Ormsbee Jeffrey Damick Patricio Echagüe We went through the high level design of replication as described in kafka-50. We discussed the dependency of the sub

[jira] [Commented] (KAFKA-202) Make the request processing in kafka asynchonous

2012-01-11 Thread Jun Rao (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13184626#comment-13184626 ] Jun Rao commented on KAFKA-202: --- unit tests now pass. +1 on the patch. We should probably

Re: Any chance on getting Kafka published to Maven?

2012-01-09 Thread Jun Rao
If so, we can just use zkclient 0.1 from maven. Could you create a jira for that? Thanks, Jun On Mon, Jan 9, 2012 at 8:01 AM, Pierre-Yves Ritschard p...@spootnik.orgwrote: it is from april 13th On Mon, Jan 9, 2012 at 4:07 PM, Jun Rao jun...@gmail.com wrote: Pierre, The zkclient jar

[jira] [Updated] (KAFKA-237) create/delete ZK path for a topic in an admin tool

2012-01-09 Thread Jun Rao (Updated) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-237: -- Attachment: kafka-237.patch create/delete ZK path for a topic in an admin tool

Re: skype meeting on replication roadmap

2012-01-06 Thread Jun Rao
to take my son to a birthday party tonight (my babe daughter is sick wife has to stay home with her) if there is any chance to move this to Monday? (sorry) if not I understand either way let me know On Thu, Jan 5, 2012 at 2:11 PM, Jun Rao jun...@gmail.com wrote: Hi, everyone, So far

Re: skype meeting on replication roadmap

2012-01-06 Thread Jun Rao
, but not Monday the 9th. On 01/06/2012 11:31 AM, Jun Rao wrote: Joe, I am ok with moving the meeting to Monday. If nobody objects, how about we meet at 2pm PST Monday (Jan 9) instead? This actually will give people a bit more time to read the docs and wiki. Jun

[jira] [Commented] (KAFKA-228) Reduce duplicate messages served by the kafka consumer for uncompressed topics to 0

2012-01-06 Thread Jun Rao (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13181491#comment-13181491 ] Jun Rao commented on KAFKA-228: --- Thanks for the new patch. Overall, looks pretty good. A few

Re: skype meeting on replication roadmap

2012-01-06 Thread Jun Rao
would be Monday, Tuesday, or Friday. Thank you. Dave On Fri, Jan 6, 2012 at 12:34 PM, Jun Rao jun...@gmail.com wrote: How about Tue (Jan 10) at 12 noon? Can everyone make that? If not, another choice will be Tue (Jan 10) at 2:30pm PST. Jun On Fri, Jan 6, 2012 at 9:25

[jira] [Resolved] (KAFKA-200) Support configurable send / receive socket buffer size in server

2012-01-06 Thread Jun Rao (Resolved) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao resolved KAFKA-200. --- Resolution: Fixed Fix Version/s: (was: 0.8) 0.7.1 Thanks, John. Just committed

[jira] [Commented] (KAFKA-236) Make 'autooffset.reset' accept a delay in addition to {smallest,largest}

2012-01-05 Thread Jun Rao (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13180528#comment-13180528 ] Jun Rao commented on KAFKA-236: --- To reset the offset, under the cover, we use

[jira] [Created] (KAFKA-237) create/delete ZK path for a topic in an admin tool

2012-01-05 Thread Jun Rao (Created) (JIRA)
create/delete ZK path for a topic in an admin tool -- Key: KAFKA-237 URL: https://issues.apache.org/jira/browse/KAFKA-237 Project: Kafka Issue Type: Sub-task Reporter: Jun Rao

[jira] [Updated] (KAFKA-237) create/delete ZK path for a topic in an admin tool

2012-01-05 Thread Jun Rao (Updated) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-237: -- Component/s: core create/delete ZK path for a topic in an admin tool

[jira] [Created] (KAFKA-238) add a getTopicMetaData method in broker and expose it to producer

2012-01-05 Thread Jun Rao (Created) (JIRA)
Components: core Reporter: Jun Rao We need a way to propagate the leader and the partition information to the producer so that it can do load balancing and semantic partitioning. One way to do that is to have the producer get the information from ZK directly. This means

[jira] [Commented] (KAFKA-238) add a getTopicMetaData method in broker and expose it to producer

2012-01-05 Thread Jun Rao (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13180677#comment-13180677 ] Jun Rao commented on KAFKA-238: --- We can potentially piggy-back auto topic creation

[jira] [Created] (KAFKA-239) Wire existing producer and consumer to use the new ZK data structure

2012-01-05 Thread Jun Rao (Created) (JIRA)
-task Components: core Reporter: Jun Rao We can assume the leader of a partition is always the first replica. Data will only be stored in the first replica. So, there is no fault-tolerance support yet. Just make the partition logical. -- This message is automatically generated

[jira] [Created] (KAFKA-240) implement new producer and consumer request format

2012-01-05 Thread Jun Rao (Created) (JIRA)
Reporter: Jun Rao We want to change the producer/consumer request/response format according to the discussion in the following wiki: https://cwiki.apache.org/confluence/display/KAFKA/New+Wire+Format+Proposal -- This message is automatically generated by JIRA. If you think it was sent

[jira] [Commented] (KAFKA-232) ConsumerConnector has no access to getOffsetsBefore

2012-01-05 Thread Jun Rao (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13180841#comment-13180841 ] Jun Rao commented on KAFKA-232: --- Do you think you can add meta data in the message itself so

[jira] [Commented] (KAFKA-238) add a getTopicMetaData method in broker and expose it to producer

2012-01-05 Thread Jun Rao (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13180855#comment-13180855 ] Jun Rao commented on KAFKA-238: --- I can see it's useful for the api to support a list

[jira] [Commented] (KAFKA-202) Make the request processing in kafka asynchonous

2012-01-05 Thread Jun Rao (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13181141#comment-13181141 ] Jun Rao commented on KAFKA-202: --- Overall, the patch looks good. Some comments: 1

skype meeting on replication roadmap

2012-01-04 Thread Jun Rao
Hi, We'd like have a meeting discussing the roadmap for replication (our 0.8 release) and how people can contribute. How about a Skype meeting at 2pm Jan 6 (Friday)? If you are interested, please send me your Skype id. Thanks, Jun

Re: skype meeting on replication roadmap

2012-01-04 Thread Jun Rao
Clarification. 2pm is PST. Thanks Andrew for reminding me the geographical diversity of the community. Jun On Wed, Jan 4, 2012 at 10:11 AM, Jun Rao jun...@gmail.com wrote: Hi, We'd like have a meeting discussing the roadmap for replication (our 0.8 release) and how people can contribute

[jira] [Commented] (KAFKA-200) Support configurable send / receive socket buffer size in server

2012-01-04 Thread Jun Rao (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13179977#comment-13179977 ] Jun Rao commented on KAFKA-200: --- John, Thanks for the detailed update. From TCP illustrated

[jira] [Updated] (KAFKA-234) Make backoff time during consumer rebalance configurable

2012-01-04 Thread Jun Rao (Updated) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-234: -- Attachment: kafka-234.patch Make backoff time during consumer rebalance configurable

[jira] [Updated] (KAFKA-234) Make backoff time during consumer rebalance configurable

2012-01-04 Thread Jun Rao (Updated) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-234: -- Status: Patch Available (was: Open) Patch submitted. Make backoff time during consumer

[jira] [Updated] (KAFKA-234) Make backoff time during consumer rebalance configurable

2012-01-04 Thread Jun Rao (Updated) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-234: -- Resolution: Fixed Fix Version/s: 0.7.1 Status: Resolved (was: Patch Available) Thanks

[jira] [Updated] (KAFKA-228) Reduce duplicate messages served by the kafka consumer for uncompressed topics to 0

2012-01-03 Thread Jun Rao (Updated) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-228: -- Attachment: kafka-228_v2.patch Attach patch v2. It implements ConsumerIterator in an alternative way

Re: Partial message when packet size = max size?

2012-01-03 Thread Jun Rao
Aaron, The consumer has to use a fetch size larger than the largest single message (+ overhead, about 10 bytes). Otherwise, the consumer won't be able to decode a single message from a fetched chunk and will get stuck. Thanks, Jun On Tue, Jan 3, 2012 at 2:08 PM, Aaron Raddon arad...@gmail.com

[jira] [Commented] (KAFKA-232) ConsumerConnector has no access to getOffsetsBefore

2012-01-03 Thread Jun Rao (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13179211#comment-13179211 ] Jun Rao commented on KAFKA-232: --- Could you explain in a bit more detail how you plan to use

[jira] [Commented] (KAFKA-48) Implement optional long poll support in fetch request

2012-01-02 Thread Jun Rao (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-48?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13178466#comment-13178466 ] Jun Rao commented on KAFKA-48: -- Taylor, Sorry for the late response. I am not sure that I

[jira] [Created] (KAFKA-231) avoid logging stacktrace directly

2012-01-02 Thread Jun Rao (Created) (JIRA)
avoid logging stacktrace directly - Key: KAFKA-231 URL: https://issues.apache.org/jira/browse/KAFKA-231 Project: Kafka Issue Type: Improvement Components: core Reporter: Jun Rao

[jira] [Updated] (KAFKA-231) avoid logging stacktrace directly

2012-01-02 Thread Jun Rao (Updated) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-231: -- Attachment: KAFKA-231.patch avoid logging stacktrace directly

[jira] [Updated] (KAFKA-231) avoid logging stacktrace directly

2012-01-02 Thread Jun Rao (Updated) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-231: -- Status: Patch Available (was: Open) Patch attached. avoid logging stacktrace directly

Re: compiling and running using intellij

2012-01-01 Thread Jun Rao
? On Thu, Dec 29, 2011 at 10:57 AM, Jun Rao jun...@gmail.com wrote: I am using 9.04 and 10.5. The scala plugin seems to mostly work for both. Thanks, Jun On Sun, Dec 25, 2011 at 12:50 PM, S Ahmed sahmed1...@gmail.com wrote: I have IntelliJ 9.0, will the scala plugin work

Re: compiling and running using intellij

2011-12-29 Thread Jun Rao
I am using 9.04 and 10.5. The scala plugin seems to mostly work for both. Thanks, Jun On Sun, Dec 25, 2011 at 12:50 PM, S Ahmed sahmed1...@gmail.com wrote: I have IntelliJ 9.0, will the scala plugin work? On Sat, Dec 24, 2011 at 5:59 PM, Jun Rao jun...@gmail.com wrote

Re: compiling and running using intellij

2011-12-24 Thread Jun Rao
:22 AM, Jun Rao jun...@gmail.com wrote: Here is my reply to your previous email: For debugging, you can either start Kafka in intellij or attach intellij to a remote jvm running kafka. Jun On Fri, Dec 23, 2011 at 1:18 PM, S Ahmed sahmed1...@gmail.com wrote: So I ran ./sbt idea

Re: compiling and running using intellij

2011-12-23 Thread Jun Rao
Here is my reply to your previous email: For debugging, you can either start Kafka in intellij or attach intellij to a remote jvm running kafka. Jun On Fri, Dec 23, 2011 at 1:18 PM, S Ahmed sahmed1...@gmail.com wrote: So I ran ./sbt idea and was able to open the project in IDEA.

Re: kafka and intellij

2011-12-22 Thread Jun Rao
For debugging, you can either start Kafka in intellij or attach intellij to a remote jvm running kafka. To understand kafka, the best place to add break point is KafkaRequestHandler. Thanks, Jun On Thu, Dec 22, 2011 at 10:25 AM, S Ahmed sahmed1...@gmail.com wrote: easy enough thanks! how do

[jira] [Commented] (KAFKA-228) Reduce duplicate messages served by the kafka consumer for uncompressed topics to 0

2011-12-19 Thread Jun Rao (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13172441#comment-13172441 ] Jun Rao commented on KAFKA-228: --- Thanks for the patch. Some comments below: 1. This jira

[jira] [Updated] (KAFKA-226) SyncProducer connect may return failed connection on reconnect

2011-12-15 Thread Jun Rao (Updated) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-226: -- Status: Patch Available (was: Open) Patch attached. SyncProducer connect may return failed

[jira] [Updated] (KAFKA-226) SyncProducer connect may return failed connection on reconnect

2011-12-15 Thread Jun Rao (Updated) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-226: -- Attachment: KAFKA-226.patch SyncProducer connect may return failed connection on reconnect

[jira] [Updated] (KAFKA-226) SyncProducer connect may return failed connection on reconnect

2011-12-15 Thread Jun Rao (Updated) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-226: -- Resolution: Fixed Status: Resolved (was: Patch Available) committed SyncProducer

[jira] [Updated] (KAFKA-225) Bug in mirroring code causes mirroring to halt

2011-12-13 Thread Jun Rao (Updated) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-225: -- Fix Version/s: 0.7.1 Status: Patch Available (was: Open) Bug in mirroring code causes

[jira] [Updated] (KAFKA-225) Bug in mirroring code causes mirroring to halt

2011-12-13 Thread Jun Rao (Updated) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-225: -- Attachment: KAFKA-225.patch Patch attached. Bug in mirroring code causes mirroring to halt

[jira] [Updated] (KAFKA-225) Bug in mirroring code causes mirroring to halt

2011-12-13 Thread Jun Rao (Updated) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-225: -- Resolution: Fixed Assignee: Jun Rao Status: Resolved (was: Patch Available) just committed

Re: ZooKeeper-based Producer behavior before there are topics

2011-12-12 Thread Jun Rao
. Thanks, Neha On Sun, Dec 11, 2011 at 2:44 AM, Inder Pall inder.p...@gmail.com wrote: why do u want to integrate zk at producer side. can you please elaborate your usecase. is it for load balancing. On Dec 7, 2011 12:35 AM, Jun Rao jun...@gmail.com wrote: David

Re: Retrieve Partition-Id from MessageStream

2011-12-08 Thread Jun Rao
The current design of the high level consumer is that all messages from a single partition (whether it's semantically partitioned or not) are guaranteed to be consumed by one consumer. However, you don't get to choose which consumer gets which partition. The kafka consumer code makes that

Re: Kafka is live in prod @ 100%

2011-12-06 Thread Jun Rao
Hi, Taylor, Thanks for the update. This is great. Could you update your usage in Kafka wiki? Also, do you delete topics online? If so, how do you do that? Jun On Tue, Dec 6, 2011 at 8:30 AM, Taylor Gautier tgaut...@tagged.com wrote: I've already mentioned this before, but I wanted to give a

Re: ZooKeeper-based Producer behavior before there are topics

2011-12-06 Thread Jun Rao
David, There is some logic in ZKBrokerPartitionInfo.bootstrapWithExistingBrokers that deals with the case that the topic is initially empty. Currently, there is no API to get metadata. One has to query ZK directly. We can discuss whether to add metadata apis. Do you want to create a jira for

Re: ZooKeeper-based Producer behavior before there are topics

2011-12-06 Thread Jun Rao
David, Could you also comment whether you want a programming api or a monitoring api? Thanks, Jun On Tue, Dec 6, 2011 at 10:03 AM, David Ormsbee d...@datadoghq.com wrote: Jun, Neha: Thank you for pointing me the right way on the Producer issue. :-) As for the metadata requests, I've

<    3   4   5   6   7   8   9   10   >