[jira] [Commented] (KAFKA-166) Create a tool to jump JMX data to a csv file to help build out performance tests

2011-10-24 Thread Chris Burroughs (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13134056#comment-13134056 ] Chris Burroughs commented on KAFKA-166: --- Coda Hale's metrics package has built in

[jira] [Commented] (KAFKA-166) Create a tool to jump JMX data to a csv file to help build out performance tests

2011-10-24 Thread Jay Kreps (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13134258#comment-13134258 ] Jay Kreps commented on KAFKA-166: - Yeah, I am not opposed to moving to that metrics package

[jira] [Created] (KAFKA-168) Support locality in consumer partition assignment algorithm

2011-10-24 Thread Jay Kreps (Created) (JIRA)
Support locality in consumer partition assignment algorithm --- Key: KAFKA-168 URL: https://issues.apache.org/jira/browse/KAFKA-168 Project: Kafka Issue Type: New Feature

[jira] [Commented] (KAFKA-168) Support locality in consumer partition assignment algorithm

2011-10-24 Thread Jay Kreps (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13134270#comment-13134270 ] Jay Kreps commented on KAFKA-168: - Ideally we would also support rack locality in the

[jira] [Assigned] (KAFKA-167) Move partition assignment to the broker

2011-10-24 Thread Jay Kreps (Assigned) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps reassigned KAFKA-167: --- Assignee: (was: Jay Kreps) Move partition assignment to the broker

[jira] [Resolved] (KAFKA-164) Config should default to a higher-throughput configuration for log.flush.interval

2011-10-24 Thread Jay Kreps (Resolved) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps resolved KAFKA-164. - Resolution: Fixed Config should default to a higher-throughput configuration for

[jira] [Updated] (KAFKA-150) Confusing NodeExistsException failing kafka broker startup

2011-10-24 Thread Jay Kreps (Updated) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-150: Resolution: Fixed Status: Resolved (was: Patch Available) Confusing NodeExistsException

[jira] [Updated] (KAFKA-133) Publish kafka jar to a public maven repository

2011-10-24 Thread Joshua Hartman (Updated) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshua Hartman updated KAFKA-133: - Labels: patch (was: ) Affects Version/s: 0.6 Status: Patch

[jira] [Commented] (KAFKA-166) Create a tool to jump JMX data to a csv file to help build out performance tests

2011-10-24 Thread Neha Narkhede (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13134325#comment-13134325 ] Neha Narkhede commented on KAFKA-166: - Minor things - Is it HH:mm:ss:SSS or

[jira] [Resolved] (KAFKA-166) Create a tool to jump JMX data to a csv file to help build out performance tests

2011-10-24 Thread Jay Kreps (Resolved) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps resolved KAFKA-166. - Resolution: Fixed Create a tool to jump JMX data to a csv file to help build out performance tests

[jira] [Commented] (KAFKA-168) Support locality in consumer partition assignment algorithm

2011-10-24 Thread Neha Narkhede (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13134353#comment-13134353 ] Neha Narkhede commented on KAFKA-168: - Today, In the rebalancing assignment, we don't

[jira] [Commented] (KAFKA-133) Publish kafka jar to a public maven repository

2011-10-24 Thread Neha Narkhede (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13134356#comment-13134356 ] Neha Narkhede commented on KAFKA-133: - Thanks for the patch Josh ! Well in time for 0.7

perf testing

2011-10-24 Thread Jay Kreps
Neha and I have been doing so work on perf testing. My past experience with these perf and integration testing frameworks is that 1. This kind of testing is extremely important. At least as important as unit testing. A lot of the bugs that are caught in production could be caught be good

[jira] [Commented] (KAFKA-91) zkclient does not show up in pom

2011-10-24 Thread Chris Burroughs (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-91?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13134364#comment-13134364 ] Chris Burroughs commented on KAFKA-91: -- Could I get a review?

[jira] [Commented] (KAFKA-91) zkclient does not show up in pom

2011-10-24 Thread Neha Narkhede (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-91?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13134371#comment-13134371 ] Neha Narkhede commented on KAFKA-91: Have we verified what other dependent jars this

[jira] [Commented] (KAFKA-150) Confusing NodeExistsException failing kafka broker startup

2011-10-24 Thread Neha Narkhede (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13134378#comment-13134378 ] Neha Narkhede commented on KAFKA-150: - Good point. Changed the fixed version to 0.7

[jira] [Updated] (KAFKA-150) Confusing NodeExistsException failing kafka broker startup

2011-10-24 Thread Neha Narkhede (Updated) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neha Narkhede updated KAFKA-150: Fix Version/s: (was: 0.8) 0.7 Confusing NodeExistsException failing

[jira] [Commented] (KAFKA-91) zkclient does not show up in pom

2011-10-24 Thread Chris Burroughs (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-91?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13134381#comment-13134381 ] Chris Burroughs commented on KAFKA-91: -- Our custom build of zkclient isn't in any repo

[jira] [Commented] (KAFKA-133) Publish kafka jar to a public maven repository

2011-10-24 Thread Joshua Hartman (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13134383#comment-13134383 ] Joshua Hartman commented on KAFKA-133: -- I came into Mountain View today on a lng

Re: perf testing

2011-10-24 Thread Neha Narkhede
Thanks for starting this discussion Jay ! dstat - This is the existing perf tool to get read/write/fs stats My only concern about dstat is that it doesn't seem to give some important IO stats like avgrq-sz - size of the write request after merging several writes together into one big write (

[jira] [Commented] (KAFKA-91) zkclient does not show up in pom

2011-10-24 Thread Neha Narkhede (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-91?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13134407#comment-13134407 ] Neha Narkhede commented on KAFKA-91: Oh, I missed that. I tried to apply the patch, but

Re: ApacheCon meetup?

2011-10-24 Thread Neha Narkhede
I've added a meetup proposal for Kafka here -http://wiki.apache.org/apachecon/ApacheMeetupsNa11 Feel free to update the interested people counter there. Also, does Wednesday lunch time work well for the time of this meetup ? Thanks, Neha On Wed, Oct 5, 2011 at 10:29 AM, Jun Rao jun...@gmail.com

[jira] [Commented] (KAFKA-139) cross-compile multiple Scala versions

2011-10-24 Thread Neha Narkhede (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13134433#comment-13134433 ] Neha Narkhede commented on KAFKA-139: - Assigning to Josh Hartman

[jira] [Updated] (KAFKA-139) cross-compile multiple Scala versions

2011-10-24 Thread Neha Narkhede (Updated) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neha Narkhede updated KAFKA-139: Fix Version/s: 0.7 cross-compile multiple Scala versions

[jira] [Commented] (KAFKA-158) go consumer producer to support compression

2011-10-24 Thread Joel Koshy (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13134462#comment-13134462 ] Joel Koshy commented on KAFKA-158: -- I get the following - are these files missing from

[jira] [Commented] (KAFKA-77) Implement group commit for kafka logs

2011-10-24 Thread Neha Narkhede (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-77?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13134637#comment-13134637 ] Neha Narkhede commented on KAFKA-77: I tried running the new producer perf tests on this

[jira] [Commented] (KAFKA-158) go consumer producer to support compression

2011-10-24 Thread Jeffrey Damick (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13134661#comment-13134661 ] Jeffrey Damick commented on KAFKA-158: -- joel: just a guess but it looks like you

[jira] [Updated] (KAFKA-169) Layering violations in Kafka code

2011-10-24 Thread Jay Kreps (Updated) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-169: Attachment: kafka_deps.svg draw_deps.py Here is a script to create a graphviz graph of the

[jira] [Commented] (KAFKA-158) go consumer producer to support compression

2011-10-24 Thread Joel Koshy (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13134749#comment-13134749 ] Joel Koshy commented on KAFKA-158: -- Yes - that was the issue. It is probably obvious to go

layering

2011-10-24 Thread Jay Kreps
Hey guys, I am seeing a few places where code is kind of growing together at the seams in a little. Here are two examples: 1. The kafka.log package is meant to be a stand-alone implementation of a log. It shouldn't really be aware that it is inside a kafka server, or connected to