Re: Helping out
Sam, Interested in taking a look at KAFKA-959https://issues.apache.org/jira/browse/KAFKA-959 ? Thanks, Jun On Sun, Jun 23, 2013 at 11:01 PM, Sam Meder sam.me...@jivesoftware.comwrote: Hey, I now have roughly a day a week I can dedicate to working on Kafka, so I am looking for issues in the 0.8.1 batch that you think might be good starting points. Input would be much appreciated. Speaking of issues, I think it would be good to either fix https://issues.apache.org/jira/browse/KAFKA-946 for 0.8 or just drop the code from the release. /Sam
Re: Helping out
Jun, Looks like the update would happen around here? On Wed, Jul 3, 2013 at 12:53 PM, Jun Rao jun...@gmail.com wrote: Sam, Interested in taking a look at KAFKA-959https://issues.apache.org/jira/browse/KAFKA-959 ? Thanks, Jun On Sun, Jun 23, 2013 at 11:01 PM, Sam Meder sam.me...@jivesoftware.com wrote: Hey, I now have roughly a day a week I can dedicate to working on Kafka, so I am looking for issues in the 0.8.1 batch that you think might be good starting points. Input would be much appreciated. Speaking of issues, I think it would be good to either fix https://issues.apache.org/jira/browse/KAFKA-946 for 0.8 or just drop the code from the release. /Sam
Re: Helping out
https://github.com/apache/kafka/blob/0.8/core/src/main/scala/kafka/producer/async/DefaultEventHandler.scala#L152 On Wed, Jul 3, 2013 at 2:18 PM, S Ahmed sahmed1...@gmail.com wrote: Jun, Looks like the update would happen around here? On Wed, Jul 3, 2013 at 12:53 PM, Jun Rao jun...@gmail.com wrote: Sam, Interested in taking a look at KAFKA-959https://issues.apache.org/jira/browse/KAFKA-959 ? Thanks, Jun On Sun, Jun 23, 2013 at 11:01 PM, Sam Meder sam.me...@jivesoftware.com wrote: Hey, I now have roughly a day a week I can dedicate to working on Kafka, so I am looking for issues in the 0.8.1 batch that you think might be good starting points. Input would be much appreciated. Speaking of issues, I think it would be good to either fix https://issues.apache.org/jira/browse/KAFKA-946 for 0.8 or just drop the code from the release. /Sam
Re: Helping out
Yes. We will need to think a bit more for the more general case when there are messages for multiple topics. Thanks, Jun On Wed, Jul 3, 2013 at 11:18 AM, S Ahmed sahmed1...@gmail.com wrote: https://github.com/apache/kafka/blob/0.8/core/src/main/scala/kafka/producer/async/DefaultEventHandler.scala#L152 On Wed, Jul 3, 2013 at 2:18 PM, S Ahmed sahmed1...@gmail.com wrote: Jun, Looks like the update would happen around here? On Wed, Jul 3, 2013 at 12:53 PM, Jun Rao jun...@gmail.com wrote: Sam, Interested in taking a look at KAFKA-959https://issues.apache.org/jira/browse/KAFKA-959 ? Thanks, Jun On Sun, Jun 23, 2013 at 11:01 PM, Sam Meder sam.me...@jivesoftware.com wrote: Hey, I now have roughly a day a week I can dedicate to working on Kafka, so I am looking for issues in the 0.8.1 batch that you think might be good starting points. Input would be much appreciated. Speaking of issues, I think it would be good to either fix https://issues.apache.org/jira/browse/KAFKA-946 for 0.8 or just drop the code from the release. /Sam
Re: Helping out
Actually the intern ended up working on offset storage (https://cwiki.apache.org/confluence/display/KAFKA/Offset+Management) in case any one is still interested in quotas. -Jay On Tue, Jun 25, 2013 at 8:54 PM, Jun Rao jun...@gmail.com wrote: As for offset management, we have a summer intern starting this week and our plan is to have the intern work on this. Thanks, Jun On Tue, Jun 25, 2013 at 7:53 AM, Sam Meder sam.me...@jivesoftware.comwrote: On Jun 24, 2013, at 9:41 AM, Jay Kreps jay.kr...@gmail.com wrote: Hey Sam, That's great. I would be happy to act as a point of contact if any questions come up or you need a reviewer for patches. Great, I have a few patches outstanding, so if you could take a look: https://issues.apache.org/jira/browse/KAFKA-943 https://issues.apache.org/jira/browse/KAFKA-946 https://issues.apache.org/jira/browse/KAFKA-956 We haven't had an official discussion of roadmap but trunk already has support for keyed messages and log compaction and a cleanup of the log code. I think the goal would be to make this release as small and quick as possible with any bigger disruptive features going into 0.9. So the plan is to base 0.8.1 on trunk? Here are some ideas of things that I think would fit well: 1. Any good housekeeping issues. 0.8 was a pretty major rewrite and getting that done and stable required ignoring a lot of basic issues that impact the out-of-the-box experience---good errors, documentation, logging, etc. We haven't filed a lot of JIRAs for this, but anything you see that falls under this umbrella would be a good target. Doing a few of these is often a good way to learn the code base. Mostly what I have been trying to do - some classes could probably also use some splitting up. I'll continue down this path. 2. Quotas. Jonathan had proposed working on this, but I don't think he ever got the time. https://issues.apache.org/jira/browse/KAFKA-656 3. Phase 2 of this wiki: https://cwiki.apache.org/confluence/display/KAFKA/Offset+Management For us 3 seems definitely more valuable than 2. We'd also really like to have support for deleting topics (i.e. continue work on https://issues.apache.org/jira/browse/KAFKA-330) and would potentially want to change the build so it creates separate server and client artifacts. /Sam Folks--any other ideas? -Jay On Sun, Jun 23, 2013 at 11:01 PM, Sam Meder sam.me...@jivesoftware.com wrote: Hey, I now have roughly a day a week I can dedicate to working on Kafka, so I am looking for issues in the 0.8.1 batch that you think might be good starting points. Input would be much appreciated. Speaking of issues, I think it would be good to either fix https://issues.apache.org/jira/browse/KAFKA-946 for 0.8 or just drop the code from the release. /Sam
Re: Helping out
Yeah I can take a look at those. The plan for branching is to do all development off trunk and cut branches for releases. 0.8 was sort of a one-off in this respect. Going forward the idea is that all new code will go against trunk. Topic delete would also be a great one. I would recommend we hold off on splitting the jars. I think post 0.8.1 (0.9?) would be a better target. The reason is that we have a bunch of major surgery to do there--move to nio, fix approach to request definition, redo producer and consumer api, etc--and I think it will be a lot easier to do all the surgery at once. -Jay On Tue, Jun 25, 2013 at 7:53 AM, Sam Meder sam.me...@jivesoftware.com wrote: On Jun 24, 2013, at 9:41 AM, Jay Kreps jay.kr...@gmail.com wrote: Hey Sam, That's great. I would be happy to act as a point of contact if any questions come up or you need a reviewer for patches. Great, I have a few patches outstanding, so if you could take a look: https://issues.apache.org/jira/browse/KAFKA-943 https://issues.apache.org/jira/browse/KAFKA-946 https://issues.apache.org/jira/browse/KAFKA-956 We haven't had an official discussion of roadmap but trunk already has support for keyed messages and log compaction and a cleanup of the log code. I think the goal would be to make this release as small and quick as possible with any bigger disruptive features going into 0.9. So the plan is to base 0.8.1 on trunk? Here are some ideas of things that I think would fit well: 1. Any good housekeeping issues. 0.8 was a pretty major rewrite and getting that done and stable required ignoring a lot of basic issues that impact the out-of-the-box experience---good errors, documentation, logging, etc. We haven't filed a lot of JIRAs for this, but anything you see that falls under this umbrella would be a good target. Doing a few of these is often a good way to learn the code base. Mostly what I have been trying to do - some classes could probably also use some splitting up. I'll continue down this path. 2. Quotas. Jonathan had proposed working on this, but I don't think he ever got the time. https://issues.apache.org/jira/browse/KAFKA-656 3. Phase 2 of this wiki: https://cwiki.apache.org/confluence/display/KAFKA/Offset+Management For us 3 seems definitely more valuable than 2. We'd also really like to have support for deleting topics (i.e. continue work on https://issues.apache.org/jira/browse/KAFKA-330) and would potentially want to change the build so it creates separate server and client artifacts. /Sam Folks--any other ideas? -Jay On Sun, Jun 23, 2013 at 11:01 PM, Sam Meder sam.me...@jivesoftware.com wrote: Hey, I now have roughly a day a week I can dedicate to working on Kafka, so I am looking for issues in the 0.8.1 batch that you think might be good starting points. Input would be much appreciated. Speaking of issues, I think it would be good to either fix https://issues.apache.org/jira/browse/KAFKA-946 for 0.8 or just drop the code from the release. /Sam
Re: Helping out
Quotas is a cool idea, is it going to do cross broker communication or it doesn't need to talk to other brokers? On Mon, Jun 24, 2013 at 12:41 PM, Jay Kreps jay.kr...@gmail.com wrote: Hey Sam, That's great. I would be happy to act as a point of contact if any questions come up or you need a reviewer for patches. We haven't had an official discussion of roadmap but trunk already has support for keyed messages and log compaction and a cleanup of the log code. I think the goal would be to make this release as small and quick as possible with any bigger disruptive features going into 0.9. Here are some ideas of things that I think would fit well: 1. Any good housekeeping issues. 0.8 was a pretty major rewrite and getting that done and stable required ignoring a lot of basic issues that impact the out-of-the-box experience---good errors, documentation, logging, etc. We haven't filed a lot of JIRAs for this, but anything you see that falls under this umbrella would be a good target. Doing a few of these is often a good way to learn the code base. 2. Quotas. Jonathan had proposed working on this, but I don't think he ever got the time. https://issues.apache.org/jira/browse/KAFKA-656 3. Phase 2 of this wiki: https://cwiki.apache.org/confluence/display/KAFKA/Offset+Management Folks--any other ideas? -Jay On Sun, Jun 23, 2013 at 11:01 PM, Sam Meder sam.me...@jivesoftware.com wrote: Hey, I now have roughly a day a week I can dedicate to working on Kafka, so I am looking for issues in the 0.8.1 batch that you think might be good starting points. Input would be much appreciated. Speaking of issues, I think it would be good to either fix https://issues.apache.org/jira/browse/KAFKA-946 for 0.8 or just drop the code from the release. /Sam
Re: Helping out
As for offset management, we have a summer intern starting this week and our plan is to have the intern work on this. Thanks, Jun On Tue, Jun 25, 2013 at 7:53 AM, Sam Meder sam.me...@jivesoftware.comwrote: On Jun 24, 2013, at 9:41 AM, Jay Kreps jay.kr...@gmail.com wrote: Hey Sam, That's great. I would be happy to act as a point of contact if any questions come up or you need a reviewer for patches. Great, I have a few patches outstanding, so if you could take a look: https://issues.apache.org/jira/browse/KAFKA-943 https://issues.apache.org/jira/browse/KAFKA-946 https://issues.apache.org/jira/browse/KAFKA-956 We haven't had an official discussion of roadmap but trunk already has support for keyed messages and log compaction and a cleanup of the log code. I think the goal would be to make this release as small and quick as possible with any bigger disruptive features going into 0.9. So the plan is to base 0.8.1 on trunk? Here are some ideas of things that I think would fit well: 1. Any good housekeeping issues. 0.8 was a pretty major rewrite and getting that done and stable required ignoring a lot of basic issues that impact the out-of-the-box experience---good errors, documentation, logging, etc. We haven't filed a lot of JIRAs for this, but anything you see that falls under this umbrella would be a good target. Doing a few of these is often a good way to learn the code base. Mostly what I have been trying to do - some classes could probably also use some splitting up. I'll continue down this path. 2. Quotas. Jonathan had proposed working on this, but I don't think he ever got the time. https://issues.apache.org/jira/browse/KAFKA-656 3. Phase 2 of this wiki: https://cwiki.apache.org/confluence/display/KAFKA/Offset+Management For us 3 seems definitely more valuable than 2. We'd also really like to have support for deleting topics (i.e. continue work on https://issues.apache.org/jira/browse/KAFKA-330) and would potentially want to change the build so it creates separate server and client artifacts. /Sam Folks--any other ideas? -Jay On Sun, Jun 23, 2013 at 11:01 PM, Sam Meder sam.me...@jivesoftware.com wrote: Hey, I now have roughly a day a week I can dedicate to working on Kafka, so I am looking for issues in the 0.8.1 batch that you think might be good starting points. Input would be much appreciated. Speaking of issues, I think it would be good to either fix https://issues.apache.org/jira/browse/KAFKA-946 for 0.8 or just drop the code from the release. /Sam
Re: Helping out
Sam, Also, just want to say thank you for fixing kafka-938. That's an awesome fix! Jun On Sun, Jun 23, 2013 at 11:01 PM, Sam Meder sam.me...@jivesoftware.comwrote: Hey, I now have roughly a day a week I can dedicate to working on Kafka, so I am looking for issues in the 0.8.1 batch that you think might be good starting points. Input would be much appreciated. Speaking of issues, I think it would be good to either fix https://issues.apache.org/jira/browse/KAFKA-946 for 0.8 or just drop the code from the release. /Sam
