Re: Build emails

2012-06-28 Thread Jun Rao
Chris, Thanks for setting this up. For 0.8, unit tests seems\ to pass. It fails at generating java docs, which is weird. Jun On Wed, Jun 27, 2012 at 3:33 PM, Chris Burroughs chris.burrou...@gmail.comwrote: By popular demand build emails now go to kafka-comm...@incubator.apache.org and we

[jira] [Commented] (KAFKA-353) tie producer-side ack with high watermark and progress of replicas

2012-06-28 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13403640#comment-13403640 ] Jun Rao commented on KAFKA-353: --- A few more comments on patch v3: 31

[jira] [Commented] (KAFKA-375) update the site page for the 0.7.1 release download and related changes

2012-06-27 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13402472#comment-13402472 ] Jun Rao commented on KAFKA-375: --- Looks good. The only thing is that message format has

Re: 0.7.1 almost out the door...

2012-06-27 Thread Jun Rao
Joe, Made a comment on the jira. Once that's fixed, we can commit the site change. To update the site in Apache, do the following: login to people.apache.orghttps://email.corp.linkedin.com/owa/redir.aspx?C=63db50a7369049b996e5e4cdb6fc9140URL=http%3a%2f%2fpeople.apache.org%2f; go to

Re: [ANNOUNCEMENT] Apache Kafka 0.7.1-incubating Released

2012-06-27 Thread Jun Rao
Joe, When I clicked the download link, I got permission denied. Thanks, Jun On Wed, Jun 27, 2012 at 5:55 PM, Joe Stein crypt...@gmail.com wrote: Hi. The Apache Kafka team is pleased to announce the release of Kafka 0.7.1-incubating. This is the second incubating release of Apache Kafka,

[jira] [Commented] (KAFKA-353) tie producer-side ack with high watermark and progress of replicas

2012-06-26 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13401447#comment-13401447 ] Jun Rao commented on KAFKA-353: --- 3.4 I overlooked that watcher.collectSatisfiedRequests

[jira] [Created] (KAFKA-376) we need to expose different data to fetch requests from the follower replicas and consumer clients

2012-06-26 Thread Jun Rao (JIRA)
Jun Rao created KAFKA-376: - Summary: we need to expose different data to fetch requests from the follower replicas and consumer clients Key: KAFKA-376 URL: https://issues.apache.org/jira/browse/KAFKA-376

Re: RC3 status

2012-06-25 Thread Jun Rao
- http://incubator.apache.org/guides/releasemanagement.html#understanding-distribution Also, a lot of users have asked the release jar to be available through Maven. It will be great to have that. Thanks, Neha On Sun, Jun 24, 2012 at 8:56 PM, Jun Rao jun...@gmail.com wrote: Joe

[jira] [Commented] (KAFKA-353) tie producer-side ack with high watermark and progress of replicas

2012-06-25 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13400682#comment-13400682 ] Jun Rao commented on KAFKA-353: --- The race condition that Jay raised seems like a potential

[jira] [Updated] (KAFKA-372) Consumer doesn't receive all data if there are multiple segment files

2012-06-25 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-372: -- Attachment: kafka-372_v1.patch There were several issues that caused the problem. 1. Log.nextAppendOffset

[jira] [Commented] (KAFKA-353) tie producer-side ack with high watermark and progress of replicas

2012-06-24 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13400204#comment-13400204 ] Jun Rao commented on KAFKA-353: --- The suggestions for future improvements all make sense. We

Re: RC3 status

2012-06-24 Thread Jun Rao
with respect to IPMC votes Given the following votes, I think our release has passed. Chris Douglas (PPMC/IPMC) + 1 Kevan Miller (IPMC)- 1 Jun Rao (PPMC) + 1 Jakob Homan (PPMC/IPMC) + 1 Please tally the votes and inform general@incubator and kafka-dev. The next step is to change the website with the new

Re: [VOTE] Release Kafka 0.7.1-incubating (Candidate 3)

2012-06-22 Thread Jun Rao
Alan, I compared all dependent jars in 0.7.1 with 0.7.0. Other than upgrades, the only added jar in 0.7.1 is the following. ./core/lib_managed/scala_2.8.0/compile/snappy-java-1.0.4.1.jar Snappy is on Apache License 2.0. So, our licenses should be covered. Thanks, Jun On Thu, Jun 21, 2012 at

[jira] [Updated] (KAFKA-370) Exception java.util.NoSuchElementException: None.get appears inconsistently in Mirror Maker log.

2012-06-22 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-370: -- Attachment: kafka-370_v1.patch Attach patch v1. The issue is that the code relies on the broker ids

[jira] [Commented] (KAFKA-306) broker failure system test broken on replication branch

2012-06-22 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13399439#comment-13399439 ] Jun Rao commented on KAFKA-306: --- Thanks for patch v4. A few more comments: 21.2 KAFA-313

Re: [VOTE] Release Kafka 0.7.1-incubating (Candidate 3)

2012-06-22 Thread Jun Rao
not true. There are a significant number of new jars. That was deemed acceptable and so without a lot of new directives, which don't seem to be clear here, so should this one. Binding +1 on RC3. On Fri, Jun 22, 2012 at 8:24 AM, Jun Rao jun...@gmail.com wrote: Alan, I compared

Re: [VOTE] Release Kafka 0.7.1-incubating (Candidate 3)

2012-06-22 Thread Jun Rao
, Jun Rao wrote: Alan, Here is what I did. I built the 0.7.0 release and the 0.7.1 release. Extract *.jar, sort them, and compare. I reached the same conclusion as Joe: the only new jar is snappy. Other jar differences are all due to jar upgrades (are you referring to those?). Could you

Re: [VOTE] Release Kafka 0.7.1-incubating (Candidate 3)

2012-06-22 Thread Jun Rao
Alan, I found the list of jars that you published during the 0.7.0 release. The new jars in the 0.7.1 release are the following: commons-cli: AL2 commons-el: AL2 commons-net: AL2 commons-io: AL2 commons-compress: AL2 core: Eclipse Public License 1.0 hsqldb: BSD jasper-compiler: AL2

replication status

2012-06-22 Thread Jun Rao
Hi, We closed the following jira this week. A few other jiras have patches and are under review. KAFKA-329: Create topic support (revisit based on v3 design) We have people signed up on the following jiras: KAFKA-339: Use multi-fetch for follower replication KAFKA-350: Improve the highwatermark

[jira] [Commented] (KAFKA-338) controller failover

2012-06-22 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13399838#comment-13399838 ] Jun Rao commented on KAFKA-338: --- Prashanth, Thanks for signing up for this jira. One thing

Re: [VOTE] Release Kafka 0.7.1-incubating (Candidate 3)

2012-06-21 Thread Jun Rao
+1 on the release. Run quickstart and unit tests. All good. Thanks, Jun On Wed, Jun 20, 2012 at 9:19 AM, Joe Stein crypt...@gmail.com wrote: Hello, This is the third candidate for the second incubator release for Apache Kafka, version 0.7.1-incubating. This release fixes the following

[jira] [Commented] (KAFKA-342) revisit the broker startup procedure according to V3 design

2012-06-21 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13399039#comment-13399039 ] Jun Rao commented on KAFKA-342: --- Prashanth, Thanks for volunteering. It turns out

[jira] [Commented] (KAFKA-368) use the pig core jar from maven instead of distributing it

2012-06-19 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13397087#comment-13397087 ] Jun Rao commented on KAFKA-368: --- +1 Just make sure that the pig-0.8.0 jar files are removed

[jira] [Updated] (KAFKA-337) upgrade ZKClient to allow conditional updates in ZK

2012-06-19 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-337: -- Resolution: Fixed Status: Resolved (was: Patch Available) committed to 0.8 upgrade

[jira] [Created] (KAFKA-369) remove ZK dependency on producer

2012-06-19 Thread Jun Rao (JIRA)
Jun Rao created KAFKA-369: - Summary: remove ZK dependency on producer Key: KAFKA-369 URL: https://issues.apache.org/jira/browse/KAFKA-369 Project: Kafka Issue Type: Sub-task Components

Re: [VOTE] Release Kafka 0.7.1-incubating (Candidate 2)

2012-06-18 Thread Jun Rao
Any other voters? Thanks, Jun On Thu, Jun 14, 2012 at 11:42 AM, Joe Stein crypt...@gmail.com wrote: Hello, This is the second candidate for the second incubator release for Apache Kafka, version 0.7.1-incubating. This release fixes the following issues

[jira] [Updated] (KAFKA-366) add jmx beans in broker to track # bytes in consumer

2012-06-18 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-366: -- Status: Patch Available (was: Open) Patch for trunk attached. Will port to 0.8 branch after committing

[jira] [Assigned] (KAFKA-366) add jmx beans in broker to track # bytes in consumer

2012-06-18 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao reassigned KAFKA-366: - Assignee: Jun Rao add jmx beans in broker to track # bytes in consumer

[jira] [Updated] (KAFKA-366) add jmx beans in broker to track # bytes in consumer

2012-06-18 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-366: -- Resolution: Fixed Fix Version/s: 0.7.1 0.8 Status: Resolved (was: Patch

Re: [VOTE] Release Kafka 0.7.1-incubating (Candidate 2)

2012-06-18 Thread Jun Rao
. FYI, I couldn't figure out from the documentation how to start a server after I built kafka. --kevan On Jun 18, 2012, at 1:14 PM, Jun Rao wrote: Any other voters? Thanks, Jun On Thu, Jun 14, 2012 at 11:42 AM, Joe Stein crypt...@gmail.com wrote: Hello, This is the second

Re: [VOTE] Release Kafka 0.7.1-incubating (Candidate 2)

2012-06-16 Thread Jun Rao
Can other people (including mentors) help vote on this release? Thanks, Jun On Thu, Jun 14, 2012 at 11:42 AM, Joe Stein crypt...@gmail.com wrote: Hello, This is the second candidate for the second incubator release for Apache Kafka, version 0.7.1-incubating. This release fixes the

[jira] [Reopened] (KAFKA-348) rebase 0.8 branch from trunk

2012-06-16 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao reopened KAFKA-348: --- Reopen the jira. Found a few missed merges and included them in kafka-348-delta.patch. rebase

video recording for the 1st Kafka user group meeting

2012-06-15 Thread Jun Rao
Thanks for everyone who showed up for the user group meeting last night. It was a good turnout and was very interactive. If you missed the event, the video recording can be found at https://cwiki.apache.org/confluence/display/KAFKA/Kafka+papers+and+presentations Jun

[jira] [Updated] (KAFKA-339) using MultiFetch in the follower

2012-06-15 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-339: -- Attachment: kafka-339_v1.patch using MultiFetch in the follower

Re: Consumer re-design proposal

2012-06-14 Thread Jun Rao
If nobody objects, we can create a separate consumer redesign branch. This way, everyone can see the changes and progress. Thanks, Jun On Mon, Jun 11, 2012 at 4:52 PM, Neha Narkhede neha.narkh...@gmail.comwrote: Hi, Over the past few months, we've received quite a lot of feedback on the

[jira] [Updated] (KAFKA-341) Create a new single host system test to validate all replicas on 0.8 branch

2012-06-14 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-341: -- Resolution: Fixed Fix Version/s: 0.8 Status: Resolved (was: Patch Available) John, thanks

[jira] [Commented] (KAFKA-365) change copyright in NOTICE to 2012

2012-06-14 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13295182#comment-13295182 ] Jun Rao commented on KAFKA-365: --- +1 on the patch. change copyright

[jira] [Commented] (KAFKA-335) Implement an embedded controller

2012-06-13 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13294758#comment-13294758 ] Jun Rao commented on KAFKA-335: --- A few more comments for v5 patch. 41. KafkaController

Re: [DISCUSSION] KAFKA 0.7.1 RELEASE

2012-06-13 Thread Jun Rao
in there then it can get merged. I have created the branch for the release. On Mon, Jun 11, 2012 at 10:56 AM, Jun Rao jun...@gmail.com wrote: Joe, Thanks for driving this. For 1), KAFKA-345 and KAFKA-346 (depends on KAFAK-345) are being worked on. It would be nice to include them in 0.7.1

[jira] [Commented] (KAFKA-335) Implement an embedded controller

2012-06-12 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13293702#comment-13293702 ] Jun Rao commented on KAFKA-335: --- Thanks for patch v2. Some more comments: 21

[jira] [Commented] (KAFKA-335) Implement an embedded controller

2012-06-12 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13293728#comment-13293728 ] Jun Rao commented on KAFKA-335: --- For 4.3, could you create a separate jira to track

[jira] [Commented] (KAFKA-341) Create a new single host system test to validate all replicas on 0.8 branch

2012-06-12 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13293745#comment-13293745 ] Jun Rao commented on KAFKA-341: --- Thanks for patch v2. I kept getting the following exception

[jira] [Commented] (KAFKA-306) broker failure system test broken on replication branch

2012-06-12 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13293752#comment-13293752 ] Jun Rao commented on KAFKA-306: --- Sorry, just got to review this. Trunk has moved, could you

[jira] [Commented] (KAFKA-329) Remove the watches/broker for new topics and partitions and change create topic admin API to send start replica state change to all brokers

2012-06-12 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13293764#comment-13293764 ] Jun Rao commented on KAFKA-329: --- Thanks for draft v2. Some quick comments: 21

[jira] [Commented] (KAFKA-341) Create a new single host system test to validate all replicas on 0.8 branch

2012-06-12 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13293958#comment-13293958 ] Jun Rao commented on KAFKA-341: --- For sleep time, could we make it more adaptive? Basically

Re: [DISCUSSION] KAFKA 0.7.1 RELEASE

2012-06-11 Thread Jun Rao
Joe, Thanks for driving this. For 1), KAFKA-345 and KAFKA-346 (depends on KAFAK-345) are being worked on. It would be nice to include them in 0.7.1. However, it involves a consumer API change and we have to be a bit more careful. So, deferring it to 0.7.2 maybe ok too. It really depends on how

[jira] [Commented] (KAFKA-329) Remove the watches/broker for new topics and partitions and change create topic admin API to send start replica state change to all brokers

2012-06-11 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13292872#comment-13292872 ] Jun Rao commented on KAFKA-329: --- Prashanth, Thanks for patch. Some comments: 1

Re: Kafka user group meeting

2012-06-08 Thread Jun Rao
Anyone else plan to attend the Kafka user group meeting on Jun 14? I'd like to get a rough head count to arrange facility and food, etc. Thanks, Jun On Thu, Jun 7, 2012 at 4:20 PM, Jun Rao jun...@gmail.com wrote: How about having a Kafka user group meeting on Jun 14, say from 7:00pm to 8

Re: [DISCUSS] Graduate from incubator?

2012-06-08 Thread Jun Rao
Actually, any committers outside of LinkedIn is interested in drive an 0.7.1 release? That will help diversifying the community. It should be much easier than the 0.7.0 release since most of the licensing issues have been resolved. Thanks, Jun On Wed, Jun 6, 2012 at 5:46 PM, Joel Koshy

Re: Kafka user group meeting

2012-06-08 Thread Jun Rao
Ok, I have created the following event for the 1st Apache Kafka user group meeting. Feel free to send it around. See you on Jun 14. http://linkd.in/Kpn6XH Thanks, Jun On Fri, Jun 8, 2012 at 10:24 AM, Jun Rao jun...@gmail.com wrote: Yes. Date: Jun 14 Time: 7:00pm-8:30pm Location: LinkedIn

kafka replication status

2012-06-08 Thread Jun Rao
Hi, As of this week, we closed KAFKA-336 (for controller to send requests to brokers). We have people signed up on the following jiras: KAFKA-335: Embedded controller KAFKA-329: Create topic support (revisit based on v3 design) KAFKA-339: Use multi-fetch for follower replication The following

[jira] [Commented] (KAFKA-329) Remove the watches/broker for new topics and partitions and change create topic admin API to send start replica state change to all brokers

2012-06-07 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13291542#comment-13291542 ] Jun Rao commented on KAFKA-329: --- Prashanth, Do you plan to work on this jira anytime soon

Kafka user group meeting

2012-06-05 Thread Jun Rao
Hi, A few people have expressed interests in a Kafka meetup. We'd be happy to host an Apache Kafka user group meeting at LinkedIn. Would people be interested? If so, what date (evening time) would people prefer and what content would people want to cover/discuss? Thanks, Jun

[jira] [Commented] (KAFKA-349) Create individual Response types for each kind of request and wrap them with BoundedByteBufferSend, remove xxResponseSend types for all requests except FetchReque

2012-06-05 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13289865#comment-13289865 ] Jun Rao commented on KAFKA-349: --- A few more comments for patch v8. 81. LeaderAndISRRequest

[jira] [Resolved] (KAFKA-349) Create individual Response types for each kind of request and wrap them with BoundedByteBufferSend, remove xxResponseSend types for all requests except FetchReques

2012-06-05 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao resolved KAFKA-349. --- Resolution: Fixed Fix Version/s: 0.8 Thanks for patch v9. Just committed to 0.8

[jira] [Updated] (KAFKA-336) add an admin RPC to communicate state changes between the controller and the broker

2012-06-05 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-336: -- Resolution: Fixed Status: Resolved (was: Patch Available) Fixed as part of kafka-349

[jira] [Commented] (KAFKA-362) ZookeeperConsumerConnector needs to connect to new leader after leadership change

2012-06-04 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13288640#comment-13288640 ] Jun Rao commented on KAFKA-362: --- In trunk, a partition only exists in 1 broker. So, once

[jira] [Commented] (KAFKA-349) Create individual Response types for each kind of request and wrap them with BoundedByteBufferSend, remove xxResponseSend types for all requests except FetchReque

2012-06-04 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13288807#comment-13288807 ] Jun Rao commented on KAFKA-349: --- 75. ControllerToBrokerRequestTest should use

[jira] [Commented] (KAFKA-345) Add a listener to ZookeeperConsumerConnector to get notified on rebalance events

2012-06-03 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13288276#comment-13288276 ] Jun Rao commented on KAFKA-345: --- Peter, Regarding #2, naming it as ConsumerListener is fine

[jira] [Commented] (KAFKA-360) Add ability to disable rebalancing in ZooKeeper consumer

2012-06-03 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13288290#comment-13288290 ] Jun Rao commented on KAFKA-360: --- It seems that the main request is to control which partition

[jira] [Commented] (KAFKA-329) Remove the watches/broker for new topics and partitions and change create topic admin API to send start replica state change to all brokers

2012-06-03 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13288296#comment-13288296 ] Jun Rao commented on KAFKA-329: --- We know that we want to change the ZK layout from

[jira] [Created] (KAFKA-362) ZookeeperConsumerConnector needs to connect to new leader after leadership change

2012-06-03 Thread Jun Rao (JIRA)
Jun Rao created KAFKA-362: - Summary: ZookeeperConsumerConnector needs to connect to new leader after leadership change Key: KAFKA-362 URL: https://issues.apache.org/jira/browse/KAFKA-362 Project: Kafka

[jira] [Updated] (KAFKA-46) Commit thread, ReplicaFetcherThread for intra-cluster replication

2012-06-01 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-46?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-46: - Resolution: Fixed Fix Version/s: 0.8 Status: Resolved (was: Patch Available) Commit thread

[jira] [Commented] (KAFKA-349) Create individual Response types for each kind of request and wrap them with BoundedByteBufferSend, remove xxResponseSend types for all requests except FetchReque

2012-06-01 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13287830#comment-13287830 ] Jun Rao commented on KAFKA-349: --- Thanks for patch v7. Some comments: 71. LeaderAndISRRequest

[jira] [Commented] (KAFKA-349) Create individual Response types for each kind of request and wrap them with BoundedByteBufferSend, remove xxResponseSend types for all requests except FetchReque

2012-05-31 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13286737#comment-13286737 ] Jun Rao commented on KAFKA-349: --- Thanks for patch v6. Some new comments: 61. FetchResponse

[jira] [Commented] (KAFKA-46) Commit thread, ReplicaFetcherThread for intra-cluster replication

2012-05-31 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-46?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13287087#comment-13287087 ] Jun Rao commented on KAFKA-46: -- +1 for patch v4. Commit thread

Re: what is your workflow when developing?

2012-05-31 Thread Jun Rao
You can debug both client and server code in an IDE. Other times, enabling debug/trace level logging can be useful. Thanks, Jun On Thu, May 31, 2012 at 7:27 PM, S Ahmed sahmed1...@gmail.com wrote: Curious on how you guys develop and debug locally with the kafka source? Do you run all the

[jira] [Commented] (KAFKA-46) Commit thread, ReplicaFetcherThread for intra-cluster replication

2012-05-30 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-46?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13286011#comment-13286011 ] Jun Rao commented on KAFKA-46: -- + 1 from me for patch v3. Let's see if there are more comments

[jira] [Created] (KAFKA-353) tie producer-side ack with high watermark and progress of replicas

2012-05-30 Thread Jun Rao (JIRA)
Jun Rao created KAFKA-353: - Summary: tie producer-side ack with high watermark and progress of replicas Key: KAFKA-353 URL: https://issues.apache.org/jira/browse/KAFKA-353 Project: Kafka Issue Type

[jira] [Commented] (KAFKA-353) tie producer-side ack with high watermark and progress of replicas

2012-05-30 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13286050#comment-13286050 ] Jun Rao commented on KAFKA-353: --- We need to support different levels of acks required

[jira] [Updated] (KAFKA-348) rebase 0.8 branch from trunk

2012-05-30 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-348: -- Attachment: kafka-348_ProducerSendThread_KafkaApis.patch Joe, Thanks a lot for your help. Really appreciate

[jira] [Commented] (KAFKA-349) Create individual Response types for each kind of request and wrap them with BoundedByteBufferSend, remove xxResponseSend types for all requests except FetchReque

2012-05-29 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13284887#comment-13284887 ] Jun Rao commented on KAFKA-349: --- Also, unit tests seem to fail due to the wire protocol

wiki describing code path for major components

2012-05-29 Thread Jun Rao
Hi, To make it easy for new people to understand Kafka code, it would be useful to document the code path for major components in Kafka. I started a section in this wiki ( https://cwiki.apache.org/confluence/display/KAFKA/Index). I think we will need to cover at least the following components. It

Re: kafka replication work status

2012-05-29 Thread Jun Rao
:) Just so everyone is aware, is there a timeline we'd like to hit as a community for getting the basic message replication stuff (phase 2?) for KAFKA-50? Knowing this will help everyone plan their time out a little better. Regards, Prashanth On Fri, May 25, 2012 at 1:49 PM, Jun Rao jun

[jira] [Commented] (KAFKA-329) Remove the watches/broker for new topics and partitions and change create topic admin API to send start replica state change to all brokers

2012-05-29 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13285322#comment-13285322 ] Jun Rao commented on KAFKA-329: --- The ticket is to revisit topic creation based on the V3

[jira] [Commented] (KAFKA-46) Commit thread, ReplicaFetcherThread for intra-cluster replication

2012-05-28 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-46?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13284570#comment-13284570 ] Jun Rao commented on KAFKA-46: -- Thanks for patch v2. To help people review the code, I

[jira] [Commented] (KAFKA-348) rebase 0.8 branch from trunk

2012-05-28 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13284574#comment-13284574 ] Jun Rao commented on KAFKA-348: --- Joe, PartitionData has to use MessageSet since FetchRequest

Re: [DISCUSS] Graduate from incubator?

2012-05-25 Thread Jun Rao
I support graduation. Is diversity in PPMC a requirement for graduation? Currently, Kafka has diversity in committers, but not in PPMC. Thanks, Jun On Wed, May 23, 2012 at 8:14 PM, Jay Kreps jay.kr...@gmail.com wrote: Hey All, Our little project is going pretty well: we have good

[jira] [Commented] (KAFKA-336) add an admin RPC to communicate state changes between the controller and the broker

2012-05-25 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13283584#comment-13283584 ] Jun Rao commented on KAFKA-336: --- Thanks for patch V3. Some more comments. 21

Re: setters and getters

2012-05-25 Thread Jun Rao
I think separating out the getter and setter makes the implementation cleaner. I am not sure how intuitive it is to use xyz_() as the setter, although it is concise. Thanks, Jun On Tue, May 22, 2012 at 9:13 PM, Jay Kreps jay.kr...@gmail.com wrote: We are a little inconsistent in our use of

[jira] [Commented] (KAFKA-348) rebase 0.8 branch from trunk

2012-05-25 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13283611#comment-13283611 ] Jun Rao commented on KAFKA-348: --- Joe, Thanks for helping out. The patch seems to miss some

kafka replication work status

2012-05-25 Thread Jun Rao
Hi, Just want to give people a quick update on the progress on kafka replication. We have more or less finalized the replication design ( https://cwiki.apache.org/confluence/display/KAFKA/kafka+Detailed+Replication+Design+V3) and I encourage everyone to take a look and send us additional

[jira] [Commented] (KAFKA-348) rebase 0.8 branch from trunk

2012-05-25 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13283756#comment-13283756 ] Jun Rao commented on KAFKA-348: --- For addPartitionTopicInfo(currentTopicRegistry, topicDirs

[jira] [Commented] (KAFKA-349) Create individual Response types for each kind of request and wrap them with BoundedByteBufferSend, remove xxResponseSend types for all requests except FetchReque

2012-05-25 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13283877#comment-13283877 ] Jun Rao commented on KAFKA-349: --- That's a good cleanup patch. Thanks. Some comments: 1. Maybe

Re: rebase 0.8 to trunk changes

2012-05-24 Thread Jun Rao
Joe, Do you still plan to merge trunk to 0.8? If not, I will do the merge after kafka-46 is committed. Thanks, Jun On Sun, Apr 22, 2012 at 9:26 AM, Joe Stein crypt...@gmail.com wrote: Hey folks, I am finally getting around to starting to get changes that have gone to trunk since the 0.8

[jira] [Commented] (KAFKA-46) Commit thread, ReplicaFetcherThread for intra-cluster replication

2012-05-23 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-46?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13281718#comment-13281718 ] Jun Rao commented on KAFKA-46: -- Jay's comments remind another thing: 21. The follower's HW

[jira] [Commented] (KAFKA-336) add an admin RPC to communicate state changes between the controller and the broker

2012-05-22 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13281133#comment-13281133 ] Jun Rao commented on KAFKA-336: --- Thanks for the new patch. Some comments: From previous

[jira] [Commented] (KAFKA-345) Add a listener to ZookeeperConsumerConnector to get notified on rebalance events

2012-05-22 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13281327#comment-13281327 ] Jun Rao commented on KAFKA-345: --- Peter, thanks for the patch. Some comments: 1. It seems

[jira] [Updated] (KAFKA-337) upgrade ZKClient to allow conditional updates in ZK

2012-05-22 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-337: -- Assignee: Jun Rao Affects Version/s: 0.8 Status: Patch Available (was: Open) Since

[jira] [Commented] (KAFKA-46) Commit thread, ReplicaFetcherThread for intra-cluster replication

2012-05-21 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-46?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13280225#comment-13280225 ] Jun Rao commented on KAFKA-46: -- Thanks for the patch. Overall, a very encouraging patch given

[jira] [Created] (KAFKA-347) change number of partitions of a topic online

2012-05-18 Thread Jun Rao (JIRA)
Jun Rao created KAFKA-347: - Summary: change number of partitions of a topic online Key: KAFKA-347 URL: https://issues.apache.org/jira/browse/KAFKA-347 Project: Kafka Issue Type: Sub-task

[jira] [Created] (KAFKA-344) migration tool from 0.7 to 0.8

2012-05-16 Thread Jun Rao (JIRA)
Jun Rao created KAFKA-344: - Summary: migration tool from 0.7 to 0.8 Key: KAFKA-344 URL: https://issues.apache.org/jira/browse/KAFKA-344 Project: Kafka Issue Type: Sub-task Components: core

[jira] [Created] (KAFKA-340) ensure ISR has enough brokers during clean shutdown of a broker

2012-05-15 Thread Jun Rao (JIRA)
Jun Rao created KAFKA-340: - Summary: ensure ISR has enough brokers during clean shutdown of a broker Key: KAFKA-340 URL: https://issues.apache.org/jira/browse/KAFKA-340 Project: Kafka Issue Type

Re: multi-fetch response doc

2012-05-15 Thread Jun Rao
, COMPRESSION and BODY) are missing. Is this behavior y design? Thanks P On Tue, May 15, 2012 at 2:12 PM, Jun Rao jun...@gmail.com wrote: The response relies on the position of each fetch response matching the position of the corresponding fetch request. Jun On Tue, May 15, 2012

[jira] [Created] (KAFKA-342) revisit the broker startup procedure according to V3 design

2012-05-15 Thread Jun Rao (JIRA)
Jun Rao created KAFKA-342: - Summary: revisit the broker startup procedure according to V3 design Key: KAFKA-342 URL: https://issues.apache.org/jira/browse/KAFKA-342 Project: Kafka Issue Type: Sub

[jira] [Created] (KAFKA-343) revisit the become leader and become follower state change operations using V3 design

2012-05-15 Thread Jun Rao (JIRA)
Jun Rao created KAFKA-343: - Summary: revisit the become leader and become follower state change operations using V3 design Key: KAFKA-343 URL: https://issues.apache.org/jira/browse/KAFKA-343 Project: Kafka

[jira] [Comment Edited] (KAFKA-50) kafka intra-cluster replication support

2012-05-15 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-50?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13269821#comment-13269821 ] Jun Rao edited comment on KAFKA-50 at 5/15/12 11:03 PM: I have

[jira] [Commented] (KAFKA-336) add an admin RPC to communicate state changes between the controller and the broker

2012-05-15 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13276377#comment-13276377 ] Jun Rao commented on KAFKA-336: --- Thanks for the patch. Some comments: 1. LeaderAndISRRequest

[jira] [Commented] (KAFKA-46) Commit thread, ReplicaFetcherThread for intra-cluster replication

2012-05-08 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-46?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13271009#comment-13271009 ] Jun Rao commented on KAFKA-46: -- Some comments on the draft. High level: 1. We should consider

[jira] [Created] (KAFKA-335) Implement an embedded controller

2012-05-07 Thread Jun Rao (JIRA)
Jun Rao created KAFKA-335: - Summary: Implement an embedded controller Key: KAFKA-335 URL: https://issues.apache.org/jira/browse/KAFKA-335 Project: Kafka Issue Type: Sub-task Components

<    1   2   3   4   5   6   7   8   9   10   >