Re: patched performance testing issues

2011-11-11 Thread Joe Stein
Hi, just wanted to ping again on this I still can't run the performance test to compare gzip with snappy. I could just be doing something wrong but unclear. Also was not sure if this was still holding up +1 on KAFKA-187 and it getting committed to trunk, no rush I can patch the trunk to use it

Re: [VOTE] Release Kafka 0.7.0 incubating candidate 6

2011-11-11 Thread Alan D. Cabrera
- artifact has been properly signed - I have confirmed that all the jars that are included the distribution are AL 2.0 compatible +1 On Nov 8, 2011, at 1:23 PM, Neha Narkhede wrote: Hi, This is the sixth candidate for the first incubator release for Apache Kafka, version 0.7.0-incubating.

JIRA reports

2011-11-11 Thread Jay Kreps
Hi Everyone, Does anyone know a way to schedule a weekly report in JIRA? I would love for JIRA to email the dev list with the set of open tickets with patches attached to help ensure we keep up on our reviews. Is there a way to do this from within JIRA? -jay

Re: [VOTE] Release Kafka 0.7.0 incubating candidate 6

2011-11-11 Thread Alan D. Cabrera
Never mind. Did a packages and re-ran tests and everything's fine. Regards, Alan On Nov 11, 2011, at 8:18 AM, Alan D. Cabrera wrote: I ran test in sbt and I get compilation errors. Does that matter to you guys? Regards, Alan On Nov 8, 2011, at 1:23 PM, Neha Narkhede wrote: Hi,

[jira] [Updated] (KAFKA-187) Add Snappy Compression as a Codec and refactor CompressionUtil and option on startup to select what the default codec

2011-11-11 Thread Jun Rao (Updated) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-187: -- Attachment: KAFKA-187_v3.patch Patch v2 looks good to me. Attaching Patch v3 with minor changes in

[jira] [Commented] (KAFKA-175) Add helper scripts to wrap the current perf tools

2011-11-11 Thread Jun Rao (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13148646#comment-13148646 ] Jun Rao commented on KAFKA-175: --- Previous comments 1 and 3 are not addressed. 4.

[jira] Account signup

2011-11-11 Thread jira
You have signed up for a JIRA account at: https://issues.apache.org/jira Here are the details of your account: - Username: kafka-mailing-list Email: kafka-dev@incubator.apache.org Full

[jira] Subscription: outstanding kafka patches

2011-11-11 Thread jira
Issue Subscription Filter: outstanding kafka patches (4 issues) The list of outstanding kafka patches Subscriber: kafka-mailing-list Key Summary KAFKA-176 Fix existing perf tools https://issues.apache.org/jira/browse/KAFKA-176 KAFKA-175 Add helper scripts to wrap

[jira] [Updated] (KAFKA-193) use by name parameter helper for logging and trait to include lazy logging and refactor code to use the new LogHelper

2011-11-11 Thread Joe Stein (Updated) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Stein updated KAFKA-193: Fix Version/s: 0.8 Affects Version/s: 0.7 Status: Patch Available (was: Open)

Re: JIRA reports

2011-11-11 Thread Jay Kreps
Cool, I did this. It will go out once a week. Hopefully this will help us keep on top of patches. Currently the JIRA has to be in one of the patch-related states (patch submitted, patch reviewed, etc). One issue I notice is that attaching a patch doesn't actually move things into the patch

Re: [jira] [Updated] (KAFKA-187) Add Snappy Compression as a Codec and refactor CompressionUtil and option on startup to select what the default codec

2011-11-11 Thread Chris Burroughs
On 11/11/2011 01:55 PM, Jeffrey Damick wrote: So with regard to the KAFKA-187https://issues.apache.org/jira/browse/KAFKA-187 what is the stance going to be on supporting new compression methods? Is it expected that all clients 'must' will support them? If not, is there a set of 'required'

Re: Proposal: fixed release schedule

2011-11-11 Thread Jun Rao
Hi, Chris, Thanks for bringing this up. My feeling is that fixed-time release is probably more suitable for a more stable project. At this moment, we are still developing major features in Kafka. The time it takes to add the compression support is going to be quite different from adding the

Re: Proposal: fixed release schedule

2011-11-11 Thread Taylor Gautier
Jun, You make valid points, but I still support Chris' viewpoint. If you go a long way and introduce a lot of big features in between releases you will slow down upgrades of your early adopters. Consider myself as a case study. My team has been working on putting Kafka into production for

Re: [VOTE] Release Kafka 0.7.0 incubating candidate 6

2011-11-11 Thread Joel Koshy
I also ran a mirroring test and an audit. +1 (Although, I'm past the 12pm deadline.) Thanks, Joel On Tue, Nov 8, 2011 at 1:23 PM, Neha Narkhede neha.narkh...@gmail.com wrote: Hi, This is the sixth candidate for the first incubator release for Apache Kafka, version 0.7.0-incubating. There

Re: Proposal: fixed release schedule

2011-11-11 Thread Jun Rao
Hi, Taylor, Are you more worried about bug fixes? If so, we can have some fixed release interval for bug fix releases. Thanks, Jun

Re: JIRA reports

2011-11-11 Thread Joel Koshy
any fix for that. Another way to do it would be if there is a way just to search for things that have patch files attached, but I couldn't figure We could approximate it by filtering on attachment count - give this filter a try:

[jira] [Created] (KAFKA-200) Support configurable send / receive socket buffer size in server

2011-11-11 Thread John Fung (Created) (JIRA)
Support configurable send / receive socket buffer size in server Key: KAFKA-200 URL: https://issues.apache.org/jira/browse/KAFKA-200 Project: Kafka Issue Type: Improvement

[jira] [Updated] (KAFKA-200) Support configurable send / receive socket buffer size in server

2011-11-11 Thread John Fung (Updated) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Fung updated KAFKA-200: Attachment: KAFKA-200.patch Support configurable send / receive socket buffer size in server

Re: Proposal: fixed release schedule

2011-11-11 Thread Taylor Gautier
No - just that big feature changes are more risky than smaller ones and having smaller upgrades helps identify unexpected issues more easily. Having fixed time length releases reduces the tendency for releases to grow larger. Another benefit of frequent small releases is that you'll get good at

Re: Proposal: fixed release schedule

2011-11-11 Thread Jun Rao
Taylor, What's the release cycle that you are looking for? Also, for big features like replication, it may be hard to decompose it into multiple releases. We can probably stage it so that some of the more advanced stuff (e.g., rebalance with new brokers) are released later. However, even to get

[jira] [Resolved] (KAFKA-199) Add a how-to-mirror document

2011-11-11 Thread Joel Koshy (Resolved) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy resolved KAFKA-199. -- Resolution: Later Holding in wiki for now. Add a how-to-mirror document

[jira] [Commented] (KAFKA-199) Add a how-to-mirror document

2011-11-11 Thread Joel Koshy (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13148771#comment-13148771 ] Joel Koshy commented on KAFKA-199: -- That's right - doesn't help scalability, but

Re: Proposal: fixed release schedule

2011-11-11 Thread Taylor Gautier
I think maybe once per month might be reasonable? Or 6 weeks? I don't know exactly what exact timing would make sense. I'm not necessarily suggesting to break up features into smaller pieces and release them in parts. If a feature takes longer than a release cycle to build then it just comes out

Re: Proposal: fixed release schedule

2011-11-11 Thread Jay Kreps
In general I like this style of time-boxed monthly release planning. It does make big disruptive changes harder but it makes small changes much easier because they get out with regularity. It also gives a nice rhythm to things. There are a couple of potential issues for switching to this right

[jira] [Created] (KAFKA-201) Support for mirroring from multiple sources

2011-11-11 Thread Paul Querna (Created) (JIRA)
Support for mirroring from multiple sources --- Key: KAFKA-201 URL: https://issues.apache.org/jira/browse/KAFKA-201 Project: Kafka Issue Type: New Feature Reporter: Paul Querna Currently

[jira] [Updated] (KAFKA-201) Support for mirroring from multiple sources

2011-11-11 Thread Paul Querna (Updated) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Querna updated KAFKA-201: -- Description: Currently the EmbeddedConsumer is configured against a single source mirror. We have a

[jira] [Commented] (KAFKA-48) Implement optional long poll support in fetch request

2011-11-11 Thread Jun Rao (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-48?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13148835#comment-13148835 ] Jun Rao commented on KAFKA-48: -- Just had a chance to look at the patch. Agree in principle this

[jira] [Created] (KAFKA-202) Make the request processing in kafka asynchonous

2011-11-11 Thread Jay Kreps (Created) (JIRA)
Make the request processing in kafka asynchonous Key: KAFKA-202 URL: https://issues.apache.org/jira/browse/KAFKA-202 Project: Kafka Issue Type: New Feature Reporter: Jay Kreps

[jira] [Updated] (KAFKA-48) Implement optional long poll support in fetch request

2011-11-11 Thread Jay Kreps (Updated) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-48?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-48: --- Attachment: (was: KAFKA-48-socket-server-refactor-draft.patch) Implement optional long poll support

[jira] [Updated] (KAFKA-202) Make the request processing in kafka asynchonous

2011-11-11 Thread Jay Kreps (Updated) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-202: Attachment: KAFKA-48-socket-server-refactor-draft.patch Make the request processing in kafka

[jira] [Commented] (KAFKA-193) use by name parameter helper for logging and trait to include lazy logging and refactor code to use the new LogHelper

2011-11-11 Thread Jay Kreps (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13148974#comment-13148974 ] Jay Kreps commented on KAFKA-193: - This looks good to me. One thing I notice is that the

[jira] [Commented] (KAFKA-193) use by name parameter helper for logging and trait to include lazy logging and refactor code to use the new LogHelper

2011-11-11 Thread Joe Stein (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13148985#comment-13148985 ] Joe Stein commented on KAFKA-193: - Well, error and fatal have overloaded for throwable

Re: KAFKA-101

2011-11-11 Thread Taylor Gautier
Jun - to be clear - the issue I have is only with the 0.6 version of Kafka. I can build and compile 0.7 just fine. On Thu, Nov 10, 2011 at 9:44 PM, Jun Rao jun...@gmail.com wrote: Taylor, Could you open up a jira and upload your patch for this sbt issue? Thanks, Jun On Thu, Nov 10,

Re: KAFKA-101

2011-11-11 Thread Jun Rao
Thanks for the clarification, Taylor. Jun On Fri, Nov 11, 2011 at 9:06 PM, Taylor Gautier tgaut...@tagged.com wrote: Jun - to be clear - the issue I have is only with the 0.6 version of Kafka. I can build and compile 0.7 just fine. On Thu, Nov 10, 2011 at 9:44 PM, Jun Rao jun...@gmail.com

Re: KAFKA-101

2011-11-11 Thread Taylor Gautier
Since I had no problems with 0.7 I thought I would have some fun with git bisect to discover which change fixed it, seems it was you on May 9: 11d9f863454ab4ae0570da12e1806dfdfafd2ba9 :-) On Fri, Nov 11, 2011 at 9:12 PM, Jun Rao jun...@gmail.com wrote: Thanks for the clarification, Taylor.