[jira] [Updated] (KAFKA-527) Compression support does numerous byte copies

2015-03-08 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-527: Assignee: Yasuhiro Matsuda (was: Jay Kreps) Compression support does numerous byte copies

[jira] [Commented] (KAFKA-1313) Support adding replicas to existing topic partitions

2015-03-08 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14352251#comment-14352251 ] Neha Narkhede commented on KAFKA-1313: -- I missed the fact that we had this option in

[jira] [Updated] (KAFKA-1884) Print metadata response errors

2015-03-08 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neha Narkhede updated KAFKA-1884: - Reviewer: Guozhang Wang [~guozhang] It seems you were helping out earlier. Print metadata

[jira] [Commented] (KAFKA-527) Compression support does numerous byte copies

2015-03-08 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=1435#comment-1435 ] Neha Narkhede commented on KAFKA-527: - bq. I think in the end we will deprecate

[jira] [Commented] (KAFKA-1313) Support adding replicas to existing topic partitions

2015-03-08 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14352242#comment-14352242 ] Neha Narkhede commented on KAFKA-1313: -- Actually, I thought of this more like a

[jira] [Comment Edited] (KAFKA-1313) Support adding replicas to existing topic partitions

2015-03-08 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14352242#comment-14352242 ] Neha Narkhede edited comment on KAFKA-1313 at 3/8/15 8:42 PM: --

[jira] [Commented] (KAFKA-2004) Write Kafka messages directly to HDFS

2015-03-08 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14352237#comment-14352237 ] Neha Narkhede commented on KAFKA-2004: -- Kafka uses local storage for latency and

[jira] [Commented] (KAFKA-1313) Support adding replicas to existing topic partitions

2015-03-08 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14352246#comment-14352246 ] Jay Kreps commented on KAFKA-1313: -- This is actually in the alter topic command today, it

Re: [KIP-DISCUSSION] KIP-13 Quotas

2015-03-08 Thread Jay Kreps
Hey Adi, Great write-up. Here are some comments: 1. I don't think you need a way to disable quotas on a per-client basis, that is just the equivalent of setting the quota to be infinite, right? 2. I agree that the configuration problem is a general part of doing dynamic configuration, and it is

[jira] [Commented] (KAFKA-527) Compression support does numerous byte copies

2015-03-08 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14352227#comment-14352227 ] Jay Kreps commented on KAFKA-527: - The clients already use MemoryRecords, so 0.8.2 and

[jira] [Commented] (KAFKA-1993) Enable topic deletion as default

2015-03-08 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14352263#comment-14352263 ] Neha Narkhede commented on KAFKA-1993: -- [~gwenshap] Thanks for the patch. Given the

[jira] [Updated] (KAFKA-1416) Unify sendMessages/getMessages in unit tests

2015-03-08 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neha Narkhede updated KAFKA-1416: - Reviewer: Guozhang Wang [~guozhang] Since you were helping out... Unify

Re: TL;DR; of the first KIP hangout

2015-03-08 Thread Jay Kreps
I took a pass on these and tried to update our summary table based on Gwen's summary. There are a couple that are just waiting on the owner to hold their vote. There are a number that are still very sparse on detail. I really want to encourage everyone to take a look at the python PEPs, I think

[jira] [Assigned] (KAFKA-527) Compression support does numerous byte copies

2015-03-08 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps reassigned KAFKA-527: --- Assignee: Jay Kreps (was: Yasuhiro Matsuda) Compression support does numerous byte copies

[jira] [Commented] (KAFKA-527) Compression support does numerous byte copies

2015-03-08 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14352230#comment-14352230 ] Neha Narkhede commented on KAFKA-527: - Agree with the long term plan. Definitely makes

[jira] [Updated] (KAFKA-2009) Fix UncheckedOffset.removeOffset synchronization and trace logging issue in mirror maker

2015-03-08 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-2009: - Resolution: Fixed Reviewer: Jay Kreps Status: Resolved (was: Patch Available) Committed,

[jira] [Commented] (KAFKA-2010) unit tests sometimes are slow during shutdown

2015-03-08 Thread Sriharsha Chintalapani (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14352329#comment-14352329 ] Sriharsha Chintalapani commented on KAFKA-2010: --- [~junrao] It might be due

[jira] [Commented] (KAFKA-2010) unit tests sometimes are slow during shutdown

2015-03-08 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14352321#comment-14352321 ] Jun Rao commented on KAFKA-2010: The long delay always happens during the shutdown of a

[jira] [Commented] (KAFKA-527) Compression support does numerous byte copies

2015-03-08 Thread Yasuhiro Matsuda (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14352290#comment-14352290 ] Yasuhiro Matsuda commented on KAFKA-527: This patch is mainly aimed at #1 above If

[jira] [Created] (KAFKA-2010) unit tests sometimes are slow during shutdown

2015-03-08 Thread Jun Rao (JIRA)
Jun Rao created KAFKA-2010: -- Summary: unit tests sometimes are slow during shutdown Key: KAFKA-2010 URL: https://issues.apache.org/jira/browse/KAFKA-2010 Project: Kafka Issue Type: Bug

Review Request 31844: Patch for KAFKA-1926

2015-03-08 Thread Tong Li
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31844/ --- Review request for kafka. Bugs: KAFKA-1926

[jira] [Commented] (KAFKA-1926) Replace kafka.utils.Utils with o.a.k.common.utils.Utils

2015-03-08 Thread Tong Li (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14352488#comment-14352488 ] Tong Li commented on KAFKA-1926: Created reviewboard https://reviews.apache.org/r/31844/

[jira] [Updated] (KAFKA-1926) Replace kafka.utils.Utils with o.a.k.common.utils.Utils

2015-03-08 Thread Tong Li (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tong Li updated KAFKA-1926: --- Attachment: KAFKA-1926.patch Replace kafka.utils.Utils with o.a.k.common.utils.Utils

Re: Review Request 31806: Patch for KAFKA-1501

2015-03-08 Thread Eric Olander
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31806/#review75654 ---

problems submitting patch set for review and a bit of solution

2015-03-08 Thread Tong Li
While trying to submit a reviewboard request by running kafka-patch-review, I was getting some errors which do not make any sense. The error claims that a file did not exist in the repository but it did. Here is the error: 1. How the problems look like: tong@u1404:/opt/stack/kafka$

Re: problems submitting patch set for review and a bit of solution

2015-03-08 Thread Manikumar Reddy
Hi, jira.ini file should be in your home directory. Otherwise it prompts you for jira username and password. https://cwiki.apache.org/confluence/display/KAFKA/Patch+submission+and+review Manikumar On Mon, Mar 9, 2015 at 9:15 AM, Tong Li liton...@us.ibm.com wrote: While trying to submit

Re: problems submitting patch set for review and a bit of solution

2015-03-08 Thread Jiangjie Qin
I saw this a couple of weeks ago, too. It seems to be a bug of review board. Last time I waited for 2-3 days and the problem went away. Jiangjie (Becket) Qin On 3/8/15, 8:45 PM, Tong Li liton...@us.ibm.com wrote: While trying to submit a reviewboard request by running kafka-patch-review, I

Review Request 31850: Patch for KAFKA-1660

2015-03-08 Thread Jiangjie Qin
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31850/ --- Review request for kafka. Bugs: KAFKA-1660

[jira] [Commented] (KAFKA-1660) Ability to call close() with a timeout on the Java Kafka Producer.

2015-03-08 Thread Jiangjie Qin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14352515#comment-14352515 ] Jiangjie Qin commented on KAFKA-1660: - Created reviewboard

[jira] [Updated] (KAFKA-1660) Ability to call close() with a timeout on the Java Kafka Producer.

2015-03-08 Thread Jiangjie Qin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiangjie Qin updated KAFKA-1660: Attachment: KAFKA-1660.patch Ability to call close() with a timeout on the Java Kafka Producer.

Re: Review Request 29467: Patch for KAFKA-1660

2015-03-08 Thread Jiangjie Qin
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29467/#review75656 --- I just had a patch submitted for KAFKA-1660. But I do not have

Re: Review Request 31850: Patch for KAFKA-1660

2015-03-08 Thread Jiangjie Qin
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31850/ --- (Updated March 9, 2015, 4:07 a.m.) Review request for kafka. Bugs:

[jira] [Commented] (KAFKA-2010) unit tests sometimes are slow during shutdown

2015-03-08 Thread Gwen Shapira (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14352521#comment-14352521 ] Gwen Shapira commented on KAFKA-2010: - [~junrao]: Out of curiosity, what's the value

Re: Review Request 31850: Patch for KAFKA-1660

2015-03-08 Thread Jiangjie Qin
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31850/ --- (Updated March 9, 2015, 4:14 a.m.) Review request for kafka. Bugs:

[jira] [Commented] (KAFKA-1660) Ability to call close() with a timeout on the Java Kafka Producer.

2015-03-08 Thread Jiangjie Qin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14352525#comment-14352525 ] Jiangjie Qin commented on KAFKA-1660: - Updated reviewboard

[jira] [Updated] (KAFKA-1660) Ability to call close() with a timeout on the Java Kafka Producer.

2015-03-08 Thread Jiangjie Qin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiangjie Qin updated KAFKA-1660: Attachment: KAFKA-1660_2015-03-08_21:14:50.patch Ability to call close() with a timeout on the

[jira] [Commented] (KAFKA-2010) unit tests sometimes are slow during shutdown

2015-03-08 Thread Gwen Shapira (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14352577#comment-14352577 ] Gwen Shapira commented on KAFKA-2010: - [~junrao] can you also check dmesg (sudo dmesg)

KAFKA-2010

2015-03-08 Thread Jun Rao
Found an interesting issue that causes the slowdown of our unit tests. Not sure how to fully explain the issue though. It would be great if someone knows TCP and sockets well can comment on the jira :). Thanks, Jun

Jenkins build is back to normal : Kafka-trunk #419

2015-03-08 Thread Apache Jenkins Server
See https://builds.apache.org/job/Kafka-trunk/419/changes

Re: [DISCUSSION] KIP-15 close(timeout) for producer

2015-03-08 Thread Jay Kreps
Hey Jiangjie, Can you capture the full motivation and use cases for the feature? This mentions your interest in having a way of aborting from inside the Callback. But it doesn't really explain that usage or why other people would want to do that. It also doesn't list the primary use case for

Re: [DISCUSS] KIP-6 - New reassignment partition logic for re-balancing

2015-03-08 Thread Jay Kreps
Hey Joe, This still seems pretty incomplete. It still has most the sections just containing the default text you are supposed to replace. It is really hard to understand what is being proposed and why and how much of the problem we are addressing. For example the motivation section just says

Re: [DISCUSS] KIP-6 - New reassignment partition logic for re-balancing

2015-03-08 Thread Joe Stein
Jay, That makes sense. I think what folks are bringing up all sounds great but I feel can/should be done afterwards as further improvements as the scope for this change has a very specific focus to resolve problems folks have today with --generate (with a patch tested and ready to go ). I should