[jira] [Commented] (KAFKA-595) Decouple producer side compression from server-side compression.

2015-01-15 Thread Manikumar Reddy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14278435#comment-14278435 ] Manikumar Reddy commented on KAFKA-595: --- [~nehanarkhede] We have given support for br

Re: [VOTE] 0.8.2.0 Candidate 1

2015-01-15 Thread Jaikiran Pai
I just downloaded the Kafka binary and am trying this on my 32 bit JVM (Java 7)? Trying to start Zookeeper or Kafka server keeps failing with "Unrecognized VM option 'UseCompressedOops'": ./zookeeper-server-start.sh ../config/zookeeper.properties Unrecognized VM option 'UseCompressedOops' Error

Re: [VOTE] 0.8.2.0 Candidate 1

2015-01-15 Thread Manikumar Reddy
Yes, we can add a check. This option works only with 64 bit jvm. On Jan 15, 2015 6:53 PM, "Jaikiran Pai" wrote: > I just downloaded the Kafka binary and am trying this on my 32 bit JVM > (Java 7)? Trying to start Zookeeper or Kafka server keeps failing with > "Unrecognized VM option 'UseCompre

[jira] [Commented] (KAFKA-1333) Add consumer co-ordinator module to the server

2015-01-15 Thread Andrii Biletskyi (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14278739#comment-14278739 ] Andrii Biletskyi commented on KAFKA-1333: - Hey [~guozhang], We are considering usi

[jira] [Commented] (KAFKA-1649) Protocol documentation does not indicate that ReplicaNotAvailable can be ignored

2015-01-15 Thread Hernan Rivas Inaka (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14278790#comment-14278790 ] Hernan Rivas Inaka commented on KAFKA-1649: --- I managed to get that error on 0.8.

[jira] [Commented] (KAFKA-1333) Add consumer co-ordinator module to the server

2015-01-15 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14278855#comment-14278855 ] Jay Kreps commented on KAFKA-1333: -- Hey [~abiletskyi] A relatively full-fledged Java cons

Re: Need some pointers to writing (real) tests

2015-01-15 Thread Jay Kreps
The integration tests for the producer are in with the server since the dependency is that the server depends on the clients rather than vice versa). Only the mock tests are with the clients. You should be able to add to one of the tests in core/src/test/scala/integration/kafka/api/Producer* On

[jira] [Commented] (KAFKA-1577) Exception in ConnectionQuotas while shutting down

2015-01-15 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14278873#comment-14278873 ] Joel Koshy commented on KAFKA-1577: --- [~german.borbolla] can you make sure you don't have

[jira] [Commented] (KAFKA-1577) Exception in ConnectionQuotas while shutting down

2015-01-15 Thread German Borbolla (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14278880#comment-14278880 ] German Borbolla commented on KAFKA-1577: I will try to reproduce with the release

Jenkins build is back to normal : Kafka-trunk #370

2015-01-15 Thread Apache Jenkins Server
See

[jira] [Comment Edited] (KAFKA-1461) Replica fetcher thread does not implement any back-off behavior

2015-01-15 Thread Sriharsha Chintalapani (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14278895#comment-14278895 ] Sriharsha Chintalapani edited comment on KAFKA-1461 at 1/15/15 4:24 PM:

[jira] [Resolved] (KAFKA-595) Decouple producer side compression from server-side compression.

2015-01-15 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy resolved KAFKA-595. -- Resolution: Implemented Assignee: Manikumar Reddy Yes I think we can close this. > Decouple produc

[jira] [Commented] (KAFKA-1461) Replica fetcher thread does not implement any back-off behavior

2015-01-15 Thread Sriharsha Chintalapani (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14278895#comment-14278895 ] Sriharsha Chintalapani commented on KAFKA-1461: --- [~guozhang] I had the follo

[jira] [Commented] (KAFKA-1333) Add consumer co-ordinator module to the server

2015-01-15 Thread Onur Karaman (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14278905#comment-14278905 ] Onur Karaman commented on KAFKA-1333: - Hey everyone. I spoke with Guozhang about this

[jira] [Commented] (KAFKA-1507) Using GetOffsetShell against non-existent topic creates the topic unintentionally

2015-01-15 Thread Sriharsha Chintalapani (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14278907#comment-14278907 ] Sriharsha Chintalapani commented on KAFKA-1507: --- [~junrao] [~nehanarkhede] I

Re: [VOTE] 0.8.2.0 Candidate 1

2015-01-15 Thread Jun Rao
Thanks for reporting this. I will remove that option in RC2. Jun On Thu, Jan 15, 2015 at 5:21 AM, Jaikiran Pai wrote: > I just downloaded the Kafka binary and am trying this on my 32 bit JVM > (Java 7)? Trying to start Zookeeper or Kafka server keeps failing with > "Unrecognized VM option 'UseC

[jira] [Commented] (KAFKA-1333) Add consumer co-ordinator module to the server

2015-01-15 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14278920#comment-14278920 ] Jay Kreps commented on KAFKA-1333: -- Woooh!!! :-) > Add consumer co-o

[jira] [Commented] (KAFKA-1507) Using GetOffsetShell against non-existent topic creates the topic unintentionally

2015-01-15 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14278929#comment-14278929 ] Jay Kreps commented on KAFKA-1507: -- I think the right way to do this is have a proper cre

Re: Heads up: KAFKA-1697 - remove code related to ack>1 on the broker

2015-01-15 Thread Ewen Cheslack-Postava
Right, so this looks like it could create an issue similar to what's currently being discussed in https://issues.apache.org/jira/browse/KAFKA-1649 where users now get errors under conditions when they previously wouldn't. Old clients won't even know about the error code, so besides failing they won

[jira] [Commented] (KAFKA-1507) Using GetOffsetShell against non-existent topic creates the topic unintentionally

2015-01-15 Thread Sriharsha Chintalapani (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14279012#comment-14279012 ] Sriharsha Chintalapani commented on KAFKA-1507: --- [~jkreps] thanks for the co

Re: Heads up: KAFKA-1697 - remove code related to ack>1 on the broker

2015-01-15 Thread Joe Stein
Looping in the mailing list that the client developers live on because they are all not on dev (though they should be if they want to be helping to build the best client libraries they can). I whole hardily believe that we need to not break existing functionality of the client protocol, ever. The

[jira] [Commented] (KAFKA-1697) remove code related to ack>1 on the broker

2015-01-15 Thread Joe Stein (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14279037#comment-14279037 ] Joe Stein commented on KAFKA-1697: -- With this patch I think we should change the existing

[jira] [Comment Edited] (KAFKA-1697) remove code related to ack>1 on the broker

2015-01-15 Thread Joe Stein (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14279037#comment-14279037 ] Joe Stein edited comment on KAFKA-1697 at 1/15/15 6:21 PM: --- With

Re: [kafka-clients] Re: Heads up: KAFKA-1697 - remove code related to ack>1 on the broker

2015-01-15 Thread Mark Roberts
This would sting a whole lot less if there was a programmatic way to get what server version is in use. Also, how will this work in mixed version clusters (during an upgrade, for example)? > On Jan 15, 2015, at 10:10, Joe Stein wrote: > > Looping in the mailing list that the client developers

Re: Heads up: KAFKA-1697 - remove code related to ack>1 on the broker

2015-01-15 Thread Magnus Edenhill
I very much agree on what Joe is saying, let's use the version field as intended and be very strict with not removing nor altering existing behaviour without bumping the version. Old API versions could be deprecated (documentation only?) immediately and removed completely in the next minor version

[DISCUSS] KIPs

2015-01-15 Thread Jay Kreps
The idea of KIPs came up in a previous discussion but there was no real crisp definition of what they were. Here is an attempt at defining a process: https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals The trick here is to have something light-weight enough that it isn't

[GitHub] kafka pull request: Trunk

2015-01-15 Thread SylviaVargasCTL
GitHub user SylviaVargasCTL opened a pull request: https://github.com/apache/kafka/pull/42 Trunk You can merge this pull request into a Git repository by running: $ git pull https://github.com/apache/kafka trunk Alternatively you can review and apply these changes as the patc

Re: [kafka-clients] Re: Heads up: KAFKA-1697 - remove code related to ack>1 on the broker

2015-01-15 Thread Gwen Shapira
Is the protocol bump caused by the behavior change or the new error code? 1) IMO, error_codes are data, and clients can expect to receive errors that they don't understand (i.e. unknown errors). AFAIK, clients don't break on unknown errors, they are simple more challenging to debug. If we document

RE: [kafka-clients] Re: Heads up: KAFKA-1697 - remove code related to ack>1 on the broker

2015-01-15 Thread Felix GV
Next time the protocol is evolved and new error codes can be introduced, would it make sense to add a new one called "Deprecated" (or "Deprecation" or "DeprecatedOperation" or whatever sounds best)? This would act as a more precise form of "Unknown" error. It could help identify what the problem

Re: [DISCUSS] KIPs

2015-01-15 Thread Joe Stein
Thanks Jay for kicking this off! I think the confluence page you wrote up is a great start. The KIP makes sense to me (at a minimum) if there is going to be any "breaking change". This way Kafka can continue to grow and blossom and we have a process in place if we are going to release a thorn ...

Re: [kafka-clients] Re: Heads up: KAFKA-1697 - remove code related to ack>1 on the broker

2015-01-15 Thread Dana Powers
> clients don't break on unknown errors maybe true for the official java clients, but I dont think the assumption holds true for community-maintained clients and users of those clients. kafka-python generally follows the fail-fast philosophy and raises an exception on any unrecognized error code

[jira] [Commented] (KAFKA-1804) Kafka network thread lacks top exception handler

2015-01-15 Thread Alexey Ozeritskiy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14279396#comment-14279396 ] Alexey Ozeritskiy commented on KAFKA-1804: -- We've written the simple patch for ka

[jira] [Commented] (KAFKA-1476) Get a list of consumer groups

2015-01-15 Thread Onur Karaman (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14279407#comment-14279407 ] Onur Karaman commented on KAFKA-1476: - Updated reviewboard https://reviews.apache.org/

[jira] [Updated] (KAFKA-1476) Get a list of consumer groups

2015-01-15 Thread Onur Karaman (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Onur Karaman updated KAFKA-1476: Attachment: KAFKA-1476_2015-01-15_14:30:04.patch > Get a list of consumer groups > -

Re: Review Request 29831: Patch for KAFKA-1476

2015-01-15 Thread Onur Karaman
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29831/ --- (Updated Jan. 15, 2015, 10:30 p.m.) Review request for kafka. Bugs: KAFKA-147

[jira] [Created] (KAFKA-1866) LogStartOffset gauge throws exceptions after log.delete()

2015-01-15 Thread Gian Merlino (JIRA)
Gian Merlino created KAFKA-1866: --- Summary: LogStartOffset gauge throws exceptions after log.delete() Key: KAFKA-1866 URL: https://issues.apache.org/jira/browse/KAFKA-1866 Project: Kafka Issue T

[jira] [Commented] (KAFKA-1866) LogStartOffset gauge throws exceptions after log.delete()

2015-01-15 Thread Gian Merlino (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14279412#comment-14279412 ] Gian Merlino commented on KAFKA-1866: - At least, I think it has that benefit. I haven'

Re: [kafka-clients] Re: Heads up: KAFKA-1697 - remove code related to ack>1 on the broker

2015-01-15 Thread Ewen Cheslack-Postava
Gwen, I think the only option that wouldn't require a protocol version change is the one where acks > 1 is converted to acks = -1 since it's the only one that doesn't potentially break older clients. The protocol guide says that the expected upgrade path is servers first, then clients, so old clie

[jira] [Commented] (KAFKA-1866) LogStartOffset gauge throws exceptions after log.delete()

2015-01-15 Thread Sriharsha Chintalapani (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14279473#comment-14279473 ] Sriharsha Chintalapani commented on KAFKA-1866: --- [~gian] any steps to reprod

[jira] [Assigned] (KAFKA-1866) LogStartOffset gauge throws exceptions after log.delete()

2015-01-15 Thread Sriharsha Chintalapani (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sriharsha Chintalapani reassigned KAFKA-1866: - Assignee: Sriharsha Chintalapani > LogStartOffset gauge throws exceptions

[jira] [Updated] (KAFKA-1476) Get a list of consumer groups

2015-01-15 Thread Onur Karaman (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Onur Karaman updated KAFKA-1476: Attachment: sample-kafka-consumer-groups-sh-output.txt I've attached a sample run of kafka-consumer-

Re: Review Request 29831: Patch for KAFKA-1476

2015-01-15 Thread Onur Karaman
> On Jan. 13, 2015, 5:36 a.m., Neha Narkhede wrote: > > To make review easier, could you add the output of the command for all > > options for 2 consumer groups that consume 2 or more topics to the JIRA? It > > will make it easier to review. One thing to watch out for is the ease of > > script

[jira] [Commented] (KAFKA-1333) Add consumer co-ordinator module to the server

2015-01-15 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14279518#comment-14279518 ] Guozhang Wang commented on KAFKA-1333: -- [~abiletskyi] sorry for getting late on this.

[jira] [Comment Edited] (KAFKA-1333) Add consumer co-ordinator module to the server

2015-01-15 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14279518#comment-14279518 ] Guozhang Wang edited comment on KAFKA-1333 at 1/15/15 11:34 PM:

[jira] [Commented] (KAFKA-1866) LogStartOffset gauge throws exceptions after log.delete()

2015-01-15 Thread Gian Merlino (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14279529#comment-14279529 ] Gian Merlino commented on KAFKA-1866: - Anything that calls log.delete() should do it.

Re: Latency Tracking Across All Kafka Component

2015-01-15 Thread Guozhang Wang
Hi, At LinkedIn we used an audit module to track the latency / message counts at each "tier" of the pipeline (for your example it will have the producer / local / central / HDFS tiers). Some details can be found on our recent talk slides (slide 41/42): http://www.slideshare.net/GuozhangWang/apach

[jira] [Created] (KAFKA-1867) liveBroker list not updated on a cluster with no topics

2015-01-15 Thread Jun Rao (JIRA)
Jun Rao created KAFKA-1867: -- Summary: liveBroker list not updated on a cluster with no topics Key: KAFKA-1867 URL: https://issues.apache.org/jira/browse/KAFKA-1867 Project: Kafka Issue Type: Bug

[jira] [Updated] (KAFKA-1810) Add IP Filtering / Whitelists-Blacklists

2015-01-15 Thread Jeff Holoman (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Holoman updated KAFKA-1810: Attachment: KAFKA-1810_2015-01-15_19:47:14.patch > Add IP Filtering / Whitelists-Blacklists > -

Re: Review Request 29714: Patch for KAFKA-1810

2015-01-15 Thread Jeff Holoman
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29714/ --- (Updated Jan. 16, 2015, 12:47 a.m.) Review request for kafka. Bugs: KAFKA-181

[jira] [Commented] (KAFKA-1810) Add IP Filtering / Whitelists-Blacklists

2015-01-15 Thread Jeff Holoman (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14279609#comment-14279609 ] Jeff Holoman commented on KAFKA-1810: - Updated reviewboard https://reviews.apache.org/

Re: Review Request 29714: Patch for KAFKA-1810

2015-01-15 Thread Jeff Holoman
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29714/ --- (Updated Jan. 16, 2015, 12:48 a.m.) Review request for kafka. Bugs: KAFKA-181

[jira] [Assigned] (KAFKA-1864) Revisit defaults for the internal offsets topic

2015-01-15 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao reassigned KAFKA-1864: -- Assignee: Jun Rao > Revisit defaults for the internal offsets topic > -

Review Request 29952: Patch for kafka-1864

2015-01-15 Thread Jun Rao
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29952/ --- Review request for kafka. Bugs: kafka-1864 https://issues.apache.org/jira/b

[jira] [Updated] (KAFKA-1864) Revisit defaults for the internal offsets topic

2015-01-15 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-1864: --- Status: Patch Available (was: Open) > Revisit defaults for the internal offsets topic > -

[jira] [Updated] (KAFKA-1864) Revisit defaults for the internal offsets topic

2015-01-15 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-1864: --- Attachment: kafka-1864.patch > Revisit defaults for the internal offsets topic > -

[jira] [Commented] (KAFKA-1864) Revisit defaults for the internal offsets topic

2015-01-15 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14279616#comment-14279616 ] Jun Rao commented on KAFKA-1864: Attached is a patch. A couple of the unit tests fail beca

[jira] [Commented] (KAFKA-1864) Revisit defaults for the internal offsets topic

2015-01-15 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14279615#comment-14279615 ] Jun Rao commented on KAFKA-1864: Created reviewboard https://reviews.apache.org/r/29952/di

[jira] [Created] (KAFKA-1868) ConsoleConsumer shouldn't override dual.commit.enabled to false if not explicitly set

2015-01-15 Thread Jun Rao (JIRA)
Jun Rao created KAFKA-1868: -- Summary: ConsoleConsumer shouldn't override dual.commit.enabled to false if not explicitly set Key: KAFKA-1868 URL: https://issues.apache.org/jira/browse/KAFKA-1868 Project: Kafk

Re: [kafka-clients] Re: Heads up: KAFKA-1697 - remove code related to ack>1 on the broker

2015-01-15 Thread Gwen Shapira
Hi, I got convinced by Joe and Dana that errors are indeed part of the protocol and can't be randomly added. So, it looks like we need to bump version of ProduceRequest in the following way: Version 0 -> accept acks >1. I think we should keep the existing behavior too (i.e. not replace it with -1

Review Request 29953: Patch for kafka-1868

2015-01-15 Thread Jun Rao
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29953/ --- Review request for kafka. Bugs: kafka-1868 https://issues.apache.org/jira/b

[jira] [Updated] (KAFKA-1868) ConsoleConsumer shouldn't override dual.commit.enabled to false if not explicitly set

2015-01-15 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-1868: --- Status: Patch Available (was: Open) > ConsoleConsumer shouldn't override dual.commit.enabled to false if not

[jira] [Updated] (KAFKA-1868) ConsoleConsumer shouldn't override dual.commit.enabled to false if not explicitly set

2015-01-15 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-1868: --- Attachment: kafka-1868.patch > ConsoleConsumer shouldn't override dual.commit.enabled to false if not > expli

[jira] [Commented] (KAFKA-1868) ConsoleConsumer shouldn't override dual.commit.enabled to false if not explicitly set

2015-01-15 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14279626#comment-14279626 ] Jun Rao commented on KAFKA-1868: Created reviewboard https://reviews.apache.org/r/29953/di

Re: [kafka-clients] Re: Heads up: KAFKA-1697 - remove code related to ack>1 on the broker

2015-01-15 Thread Steve Morin
Agree errors should be part of the protocol > On Jan 15, 2015, at 17:59, Gwen Shapira wrote: > > Hi, > > I got convinced by Joe and Dana that errors are indeed part of the > protocol and can't be randomly added. > > So, it looks like we need to bump version of ProduceRequest in the > following

Re: [kafka-clients] Re: Heads up: KAFKA-1697 - remove code related to ack>1 on the broker

2015-01-15 Thread Gwen Shapira
The errors are part of the KIP process now, so I think the clients are safe :) https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals On Thu, Jan 15, 2015 at 5:12 PM, Steve Morin wrote: > Agree errors should be part of the protocol > >> On Jan 15, 2015, at 17:59, Gwen Shap

[jira] [Updated] (KAFKA-1810) Add IP Filtering / Whitelists-Blacklists

2015-01-15 Thread Joe Stein (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Stein updated KAFKA-1810: - Reviewer: Gwen Shapira > Add IP Filtering / Whitelists-Blacklists > -

Re: Review Request 29952: Patch for kafka-1864

2015-01-15 Thread Joel Koshy
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29952/#review68381 --- core/src/main/scala/kafka/server/OffsetManager.scala

Re: Review Request 29953: Patch for kafka-1868

2015-01-15 Thread Joel Koshy
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29953/#review68383 --- Ship it! Ship It! - Joel Koshy On Jan. 16, 2015, 1:02 a.m., Jun

[jira] [Commented] (KAFKA-1868) ConsoleConsumer shouldn't override dual.commit.enabled to false if not explicitly set

2015-01-15 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14279660#comment-14279660 ] Joel Koshy commented on KAFKA-1868: --- Appears to be a side-effect of KAFKA-924 > Console

[jira] [Commented] (KAFKA-1864) Revisit defaults for the internal offsets topic

2015-01-15 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14279677#comment-14279677 ] Jun Rao commented on KAFKA-1864: Also, another impact in this patch is that the offset top

Re: Review Request 29952: Patch for kafka-1864

2015-01-15 Thread Jun Rao
> On Jan. 16, 2015, 1:25 a.m., Joel Koshy wrote: > > core/src/main/scala/kafka/server/OffsetManager.scala, line 79 > > > > > > The only issue here is the problem raised in KAFKA-1867 - even though > > that should not h

[jira] [Updated] (KAFKA-1868) ConsoleConsumer shouldn't override dual.commit.enabled to false if not explicitly set

2015-01-15 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-1868: --- Resolution: Fixed Status: Resolved (was: Patch Available) Thanks for the review. Committed to both 0.

Re: [kafka-clients] Re: Heads up: KAFKA-1697 - remove code related to ack>1 on the broker

2015-01-15 Thread Guozhang Wang
+1 on Joe's suggestions, glad to see it happening! On Thu, Jan 15, 2015 at 5:19 PM, Gwen Shapira wrote: > The errors are part of the KIP process now, so I think the clients are > safe :) > > > https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals > > On Thu, Jan 15, 2015

Re: [kafka-clients] Re: Heads up: KAFKA-1697 - remove code related to ack>1 on the broker

2015-01-15 Thread Gwen Shapira
I created a KIP for this suggestion: https://cwiki.apache.org/confluence/display/KAFKA/KIP-1+-+Remove+support+of+request.required.acks Basically documenting what was already discussed here. Comments will be awesome! Gwen On Thu, Jan 15, 2015 at 5:19 PM, Gwen Shapira wrote: > The errors are pa

[jira] [Commented] (KAFKA-1476) Get a list of consumer groups

2015-01-15 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14279715#comment-14279715 ] Neha Narkhede commented on KAFKA-1476: -- Thanks for attaching the output, [~onurkarama

[jira] [Comment Edited] (KAFKA-1476) Get a list of consumer groups

2015-01-15 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14279715#comment-14279715 ] Neha Narkhede edited comment on KAFKA-1476 at 1/16/15 2:21 AM: -

Re: Compatibility + Unknown APIs

2015-01-15 Thread Guozhang Wang
The "hacky" method that Dana suggests does not sound too hacky to me actually. Since such scenario will only happen when 1) new clients talk to older server and 2) older clients talk to new server with some APIs deprecated, and "correlation_id" is always set to meaningful numbers before, old clien

Re: Review Request 29952: Patch for kafka-1864

2015-01-15 Thread Gwen Shapira
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29952/#review68395 --- core/src/main/scala/kafka/server/OffsetManager.scala

Re: Review Request 29952: Patch for kafka-1864

2015-01-15 Thread Jun Rao
> On Jan. 16, 2015, 2:54 a.m., Gwen Shapira wrote: > > core/src/main/scala/kafka/server/OffsetManager.scala, line 77 > > > > > > I'm wondering why you chose to change defaults here and not in > > KafkaConfig? > > U

Re: Review Request 29952: Patch for kafka-1864

2015-01-15 Thread Gwen Shapira
> On Jan. 16, 2015, 2:54 a.m., Gwen Shapira wrote: > > core/src/main/scala/kafka/server/OffsetManager.scala, line 77 > > > > > > I'm wondering why you chose to change defaults here and not in > > KafkaConfig? > > U

Re: kafka consumer

2015-01-15 Thread Guozhang Wang
Hi, 1. Not sure if I understand your question.. could you elaborate? 2. Yes, and then the data for that topic will be distributed at the granularity of partitions to your consumers. 3. The default value is set to 60 seconds I believe. You can read the config docs for its semantics here: http://ka

Re: Review Request 29952: Patch for kafka-1864

2015-01-15 Thread Joel Koshy
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29952/#review68406 --- Ship it! core/src/main/scala/kafka/server/OffsetManager.scala

Re: [DISCUSS] KIPs

2015-01-15 Thread Jay Kreps
Hey Joe, Yeah I guess the question is what is the definition of major? I agree we definitely don't want to generate a bunch of paperwork. We have enough problems just getting all the contributions reviewed and checked in in a timely fashion... So obviously bug fixes would not apply here. I think

Re: [kafka-clients] Re: Heads up: KAFKA-1697 - remove code related to ack>1 on the broker

2015-01-15 Thread Jay Kreps
This is a good case to discuss. Let's figure the general case of how we want to handle errors and get that documented in the protocol. The problem right now is that we give no guidance on this. I actually thought Gwen's suggestion made sense on the guidance we should have given which is that we wi

[jira] [Updated] (KAFKA-1674) auto.create.topics.enable docs are misleading

2015-01-15 Thread Manikumar Reddy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikumar Reddy updated KAFKA-1674: --- Fix Version/s: 0.8.2 This is simple documentation correction. we can push this to 0.8.2 > aut

[jira] [Commented] (KAFKA-1333) Add consumer co-ordinator module to the server

2015-01-15 Thread Andrii Biletskyi (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14279930#comment-14279930 ] Andrii Biletskyi commented on KAFKA-1333: - [~guozhang] many thanks for such inform

[jira] [Comment Edited] (KAFKA-1804) Kafka network thread lacks top exception handler

2015-01-15 Thread Alexey Ozeritskiy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14279396#comment-14279396 ] Alexey Ozeritskiy edited comment on KAFKA-1804 at 1/16/15 7:44 AM: -