Re: Review Request 36664: Patch for KAFKA-2353

2015-07-22 Thread Jiangjie Qin
> On July 22, 2015, 5:13 p.m., Guozhang Wang wrote: > > LGTM overall. Could you address Ismael's comments as well before check-in? Thanks, Guozhang. I updated patch to address Ismael's comments. - Jiangjie --- This is an automatically g

[jira] [Commented] (KAFKA-2350) Add KafkaConsumer pause capability

2015-07-22 Thread Jiangjie Qin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14638182#comment-14638182 ] Jiangjie Qin commented on KAFKA-2350: - [~yasuhiro.matsuda] [~jkreps] [~hachikuji] [~gw

[jira] [Commented] (KAFKA-2350) Add KafkaConsumer pause capability

2015-07-22 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14638139#comment-14638139 ] Jay Kreps commented on KAFKA-2350: -- [~hachikuji] yeah i vote for pause/resume (or unpause

Re: Review Request 33620: Patch for KAFKA-1690

2015-07-22 Thread Jun Rao
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33620/#review92711 --- clients/src/main/java/org/apache/kafka/common/network/ByteBufferSen

Re: Round Robin Among Consumers

2015-07-22 Thread J A
Why have partition at all, if I don't need to scale topic. Coupling topic scalability with consumer scalability just goes against messaging systems core principle of decoupling consumer and producers On Wednesday, July 22, 2015, Aditya Auradkar wrote: > Hi, > > Why not simply have as many partit

Re: Round Robin Among Consumers

2015-07-22 Thread J A
Ordering gurantee should be optional. If someone needs ordering, they can always fall back on exclusive consumer strategy On Wednesday, July 22, 2015, Ashish Singh wrote: > Hey, don't you think that would be against the basic ordering guarantees > Kafka provides? > > On Wed, Jul 22, 2015 at 2:14

[jira] [Commented] (KAFKA-2351) Brokers are having a problem shutting down correctly

2015-07-22 Thread Mayuresh Gharat (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14638076#comment-14638076 ] Mayuresh Gharat commented on KAFKA-2351: [~junrao], [~guozhang] can you take a loo

[jira] [Commented] (KAFKA-2350) Add KafkaConsumer pause capability

2015-07-22 Thread Jiangjie Qin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14638070#comment-14638070 ] Jiangjie Qin commented on KAFKA-2350: - I think an explicit configuration makes sense.

[jira] [Commented] (KAFKA-2350) Add KafkaConsumer pause capability

2015-07-22 Thread Jiangjie Qin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14638063#comment-14638063 ] Jiangjie Qin commented on KAFKA-2350: - Yes, that is what I meant. Isn't it the case t

Re: Review Request 36664: Patch for KAFKA-2353

2015-07-22 Thread Jiangjie Qin
> On July 21, 2015, 11:15 p.m., Gwen Shapira wrote: > > core/src/main/scala/kafka/network/SocketServer.scala, line 400 > > > > > > So in case of unexpected exception, we log an error and keep running? > > > >

Re: Review Request 36664: Patch for KAFKA-2353

2015-07-22 Thread Jiangjie Qin
> On July 22, 2015, 10:40 a.m., Ismael Juma wrote: > > core/src/main/scala/kafka/network/SocketServer.scala, line 401 > > > > > > Is it intentional to ignore `java.lang.Error` too? I think java.lang.Error is a subcl

Re: Review Request 36681: Patch for KAFKA-2275

2015-07-22 Thread Jason Gustafson
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36681/#review92700 --- Ship it! Just one question on the unit test. Otherwise, LGTM. cli

[jira] [Commented] (KAFKA-2355) Add an unit test to validate the deletion of a partition marked as deleted

2015-07-22 Thread Edward Ribeiro (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14638036#comment-14638036 ] Edward Ribeiro commented on KAFKA-2355: --- Updated reviewboard https://reviews.apache.

[jira] [Updated] (KAFKA-2355) Add an unit test to validate the deletion of a partition marked as deleted

2015-07-22 Thread Edward Ribeiro (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edward Ribeiro updated KAFKA-2355: -- Attachment: KAFKA-2355_2015-07-22_22:23:13.patch > Add an unit test to validate the deletion of

Re: Review Request 36670: Patch for KAFKA-2355

2015-07-22 Thread Edward Ribeiro
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36670/ --- (Updated July 23, 2015, 1:23 a.m.) Review request for kafka. Bugs: KAFKA-2355

[jira] [Commented] (KAFKA-1893) Allow regex subscriptions in the new consumer

2015-07-22 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14638028#comment-14638028 ] Jason Gustafson commented on KAFKA-1893: [~singhashish] Sure, no problem. > Allow

Re: Review Request 36681: Patch for KAFKA-2275

2015-07-22 Thread Edward Ribeiro
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36681/#review92696 --- clients/src/main/java/org/apache/kafka/clients/consumer/internals/F

[jira] [Updated] (KAFKA-2353) SocketServer.Processor should catch exception and close the socket properly in configureNewConnections.

2015-07-22 Thread Jiangjie Qin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiangjie Qin updated KAFKA-2353: Attachment: KAFKA-2353_2015-07-22_17:51:42.patch > SocketServer.Processor should catch exception and

Re: Review Request 36664: Patch for KAFKA-2353

2015-07-22 Thread Jiangjie Qin
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36664/ --- (Updated July 23, 2015, 12:51 a.m.) Review request for kafka. Bugs: KAFKA-235

[jira] [Commented] (KAFKA-2353) SocketServer.Processor should catch exception and close the socket properly in configureNewConnections.

2015-07-22 Thread Jiangjie Qin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14638008#comment-14638008 ] Jiangjie Qin commented on KAFKA-2353: - Updated reviewboard https://reviews.apache.org/

[jira] [Commented] (KAFKA-2355) Add an unit test to validate the deletion of a partition marked as deleted

2015-07-22 Thread Edward Ribeiro (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14637996#comment-14637996 ] Edward Ribeiro commented on KAFKA-2355: --- Hi [~ijuma] and [~granthenke], I have updat

[jira] [Updated] (KAFKA-2355) Add an unit test to validate the deletion of a partition marked as deleted

2015-07-22 Thread Edward Ribeiro (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edward Ribeiro updated KAFKA-2355: -- Attachment: KAFKA-2355_2015-07-22_21:37:51.patch > Add an unit test to validate the deletion of

[jira] [Commented] (KAFKA-2355) Add an unit test to validate the deletion of a partition marked as deleted

2015-07-22 Thread Edward Ribeiro (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14637984#comment-14637984 ] Edward Ribeiro commented on KAFKA-2355: --- Updated reviewboard https://reviews.apache.

Re: Review Request 36670: Patch for KAFKA-2355

2015-07-22 Thread Edward Ribeiro
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36670/ --- (Updated July 23, 2015, 12:38 a.m.) Review request for kafka. Bugs: KAFKA-235

Re: Review Request 36670: Patch for KAFKA-2355

2015-07-22 Thread Edward Ribeiro
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36670/#review92688 --- core/src/test/scala/unit/kafka/admin/DeleteTopicTest.scala (line 26

Re: Should 0.8.3 consumers correctly function with 0.8.2 brokers?

2015-07-22 Thread Stevo Slavić
Jiangjie, Seems I was misunderstood; KafakConsumer.poll after subscribing to topic via "KafkaConsumer.subscribe(String... topics)" does not work, "Unknown api code 11" error, even with both client and broker being latest 0.8.3/trunk. Will try to create a failing test and open bug report. Kind reg

[jira] [Commented] (KAFKA-1893) Allow regex subscriptions in the new consumer

2015-07-22 Thread Ashish K Singh (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14637894#comment-14637894 ] Ashish K Singh commented on KAFKA-1893: --- [~hachikuji] I think after working on KAFKA

Re: Review Request 36681: Patch for KAFKA-2275

2015-07-22 Thread Ashish Singh
> On July 22, 2015, 4:43 p.m., Jason Gustafson wrote: > > Hey Ashish, this looks pretty good to me. Just some minor comments. Thanks for the review! Addressed your concerns. > On July 22, 2015, 4:43 p.m., Jason Gustafson wrote: > > clients/src/main/java/org/apache/kafka/clients/consumer/intern

[jira] [Commented] (KAFKA-2350) Add KafkaConsumer pause capability

2015-07-22 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14637890#comment-14637890 ] Guozhang Wang commented on KAFKA-2350: -- [~becket_qin] OK I understand now. The "subsc

[jira] [Updated] (KAFKA-2275) Add a ListTopics() API to the new consumer

2015-07-22 Thread Ashish K Singh (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish K Singh updated KAFKA-2275: -- Attachment: KAFKA-2275_2015-07-22_16:09:34.patch > Add a ListTopics() API to the new consumer >

[jira] [Commented] (KAFKA-2275) Add a ListTopics() API to the new consumer

2015-07-22 Thread Ashish K Singh (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14637889#comment-14637889 ] Ashish K Singh commented on KAFKA-2275: --- Updated reviewboard https://reviews.apache.

Re: Review Request 36681: Patch for KAFKA-2275

2015-07-22 Thread Ashish Singh
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36681/ --- (Updated July 22, 2015, 11:09 p.m.) Review request for kafka. Bugs: KAFKA-227

[jira] [Commented] (KAFKA-2350) Add KafkaConsumer pause capability

2015-07-22 Thread Jiangjie Qin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14637792#comment-14637792 ] Jiangjie Qin commented on KAFKA-2350: - [~guozhang] In the last case you mentioned, is

[jira] [Commented] (KAFKA-2350) Add KafkaConsumer pause capability

2015-07-22 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14637753#comment-14637753 ] Jason Gustafson commented on KAFKA-2350: [~jkreps] This is a little unrelated to t

[jira] [Commented] (KAFKA-2004) Write Kafka messages directly to HDFS

2015-07-22 Thread Ashish K Singh (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14637745#comment-14637745 ] Ashish K Singh commented on KAFKA-2004: --- Spark streaming supports reading from Kafka

Re: Round Robin Among Consumers

2015-07-22 Thread Aditya Auradkar
Hi, Why not simply have as many partitions as the set of consumers you want to round robin across? Aditya On Wed, Jul 22, 2015 at 2:37 PM, Ashish Singh wrote: > Hey, don't you think that would be against the basic ordering guarantees > Kafka provides? > > On Wed, Jul 22, 2015 at 2:14 PM, J A

Jenkins build is back to normal : KafkaPreCommit #160

2015-07-22 Thread Apache Jenkins Server
See

Re: Round Robin Among Consumers

2015-07-22 Thread Ashish Singh
Hey, don't you think that would be against the basic ordering guarantees Kafka provides? On Wed, Jul 22, 2015 at 2:14 PM, J A wrote: > Hi, This is reference to stackoverflow question " > > http://stackoverflow.com/questions/31547216/kafka-log-deletion-and-load-balancing-across-consumers > " > Si

Round Robin Among Consumers

2015-07-22 Thread J A
Hi, This is reference to stackoverflow question " http://stackoverflow.com/questions/31547216/kafka-log-deletion-and-load-balancing-across-consumers " Since Kafka 0.8 already maintains a client offset, i would like to request a feature, where a single partition consumption can be round robin across

Re: Review Request 36664: Patch for KAFKA-2353

2015-07-22 Thread Gwen Shapira
> On July 21, 2015, 11:15 p.m., Gwen Shapira wrote: > > core/src/main/scala/kafka/network/SocketServer.scala, line 465 > > > > > > Turns out that catching Throwable is a really bad idea: > > https://www.sumologic.co

Re: Review Request 36681: Patch for KAFKA-2275

2015-07-22 Thread Guozhang Wang
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36681/#review92644 --- Ship it! LGTM. I think we can checkin after Jason's comments get ad

Re: [VOTE] Switch to GitHub pull requests for new contributions

2015-07-22 Thread Ismael Juma
Hi, On 22 Jul 2015 19:32, "Jiangjie Qin" wrote: > > +1 (non binding) > > Can we have a wiki for procedure and let people verify the steps? After > that we can update the Apache project page. Yes, I linked to the page in the original message: https://cwiki.apache.org/confluence/display/KAFKA/Con

[jira] [Commented] (KAFKA-2342) KafkaConsumer rebalance with in-flight fetch can cause invalid position

2015-07-22 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14637539#comment-14637539 ] Guozhang Wang commented on KAFKA-2342: -- [~sslavic] corrected, thanks. > KafkaConsume

[jira] [Updated] (KAFKA-2342) KafkaConsumer rebalance with in-flight fetch can cause invalid position

2015-07-22 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-2342: - Fix Version/s: (was: 0.9.0) 0.8.3 > KafkaConsumer rebalance with in-flight

[jira] [Commented] (KAFKA-2342) KafkaConsumer rebalance with in-flight fetch can cause invalid position

2015-07-22 Thread Stevo Slavic (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14637533#comment-14637533 ] Stevo Slavic commented on KAFKA-2342: - [~guozhang], fix version 0.9.0 or 0.8.3? > Kaf

[jira] [Commented] (KAFKA-2342) KafkaConsumer rebalance with in-flight fetch can cause invalid position

2015-07-22 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14637525#comment-14637525 ] ASF GitHub Bot commented on KAFKA-2342: --- Github user asfgit closed the pull request

[jira] [Resolved] (KAFKA-2342) KafkaConsumer rebalance with in-flight fetch can cause invalid position

2015-07-22 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang resolved KAFKA-2342. -- Resolution: Fixed Fix Version/s: 0.9.0 Issue resolved by pull request 88 [https://github.

[GitHub] kafka pull request: KAFKA-2342; KafkaConsumer rebalance with in-fl...

2015-07-22 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/88 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enable

Zookeeper use cases with Kafka

2015-07-22 Thread Prabhjot Bharaj
Hello Folks, I wish to contribute to Kafka internals. And, one of the things which can help me do that is understanding how kafka uses zookeeper. I have some of these basic doubts:- 1. Is zookeeper primarily used for locking ? If yes, in what cases and what kind of nodes does it use - sequential/

Re: Review Request 36664: Patch for KAFKA-2353

2015-07-22 Thread Jiangjie Qin
> On July 21, 2015, 11:15 p.m., Gwen Shapira wrote: > > core/src/main/scala/kafka/network/SocketServer.scala, line 465 > > > > > > Turns out that catching Throwable is a really bad idea: > > https://www.sumologic.co

[jira] [Commented] (KAFKA-2004) Write Kafka messages directly to HDFS

2015-07-22 Thread sutanu das (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14637419#comment-14637419 ] sutanu das commented on KAFKA-2004: --- We just want to stream each logfile as event from k

Re: Review Request 36664: Patch for KAFKA-2353

2015-07-22 Thread Gwen Shapira
OK, I'm convinced :) I'll commit after Becket addresses other review points and adds some comments on how our error handling works (to make sure we don't screw it up later). Gwen On Wed, Jul 22, 2015 at 11:34 AM, Todd Palino wrote: > Since I've been dealing with the fallout of this particular p

Re: Should 0.8.3 consumers correctly function with 0.8.2 brokers?

2015-07-22 Thread Jiangjie Qin
I don't think we have consumer coordinator in 0.8.2 brokers. So KafkaConsumer in 0.8.3 will only be able to subscribe to partitions explicitly. Subscribing to a topic won't work with 0.8.2 brokers. Jiangjie (Becket) Qin On Wed, Jul 22, 2015 at 4:26 AM, Stevo Slavić wrote: > I'm getting "Unknown

Re: Review Request 36664: Patch for KAFKA-2353

2015-07-22 Thread Todd Palino
Since I've been dealing with the fallout of this particular problem all week, I'll add a few thoughts... On Wed, Jul 22, 2015 at 10:51 AM, Gwen Shapira wrote: > > > > On July 21, 2015, 11:15 p.m., Gwen Shapira wrote: > > > core/src/main/scala/kafka/network/SocketServer.scala, line 465 > > > < >

Re: [VOTE] Switch to GitHub pull requests for new contributions

2015-07-22 Thread Jiangjie Qin
+1 (non binding) Can we have a wiki for procedure and let people verify the steps? After that we can update the Apache project page. On Tue, Jul 21, 2015 at 5:38 PM, Edward Ribeiro wrote: > +1 (non binding) > > On Tue, Jul 21, 2015 at 7:36 PM, Jay Kreps wrote: > > > +1 > > > > -Jay > > > > On

Re: [DISCUSS] KIP-27 - Conditional Publish

2015-07-22 Thread Gwen Shapira
Tangent: I think we should complete the move of Produce / Fetch RPC to the client libraries before we add more revisions to this protocol. On Wed, Jul 22, 2015 at 11:02 AM, Jiangjie Qin wrote: > I missed yesterday's KIP hangout. I'm currently working on another KIP for > enriched metadata of mess

[jira] [Commented] (KAFKA-2350) Add KafkaConsumer pause capability

2015-07-22 Thread Gwen Shapira (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14637338#comment-14637338 ] Gwen Shapira commented on KAFKA-2350: - Agree with [~yasuhiro.matsuda]. The notion of

[jira] [Commented] (KAFKA-2350) Add KafkaConsumer pause capability

2015-07-22 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14637317#comment-14637317 ] Jay Kreps commented on KAFKA-2350: -- [~hachikuji] Yeah I agree that would help this case.

Re: [DISCUSS] KIP-27 - Conditional Publish

2015-07-22 Thread Jiangjie Qin
I missed yesterday's KIP hangout. I'm currently working on another KIP for enriched metadata of messages. Guozhang has already created a wiki page before ( https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Enriched+Message+Metadata). We plan to fill the relative offset to the offset field in

Re: Review Request 36664: Patch for KAFKA-2353

2015-07-22 Thread Gwen Shapira
> On July 21, 2015, 11:15 p.m., Gwen Shapira wrote: > > core/src/main/scala/kafka/network/SocketServer.scala, line 465 > > > > > > Turns out that catching Throwable is a really bad idea: > > https://www.sumologic.co

Re: Review Request 36664: Patch for KAFKA-2353

2015-07-22 Thread Gwen Shapira
> On July 22, 2015, 10:40 a.m., Ismael Juma wrote: > > core/src/main/scala/kafka/network/SocketServer.scala, line 467 > > > > > > As far as I can see `ClosedChannelException`, `IllegalStateException` > > and `Illega

[jira] [Commented] (KAFKA-2350) Add KafkaConsumer pause capability

2015-07-22 Thread Yasuhiro Matsuda (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14637219#comment-14637219 ] Yasuhiro Matsuda commented on KAFKA-2350: - I think overloading subscribe/unsubscri

Re: Review Request 36664: Patch for KAFKA-2353

2015-07-22 Thread Guozhang Wang
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36664/#review92607 --- LGTM overall. Could you address Ismael's comments as well before che

[jira] [Commented] (KAFKA-2352) Possible memory leak in MirrorMaker and/or new Producer

2015-07-22 Thread Jiangjie Qin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14637208#comment-14637208 ] Jiangjie Qin commented on KAFKA-2352: - Can you verify whether the messages you passed

Re: Review Request 36664: Patch for KAFKA-2353

2015-07-22 Thread Guozhang Wang
> On July 21, 2015, 11:15 p.m., Gwen Shapira wrote: > > core/src/main/scala/kafka/network/SocketServer.scala, line 400 > > > > > > So in case of unexpected exception, we log an error and keep running? > > > >

[jira] [Comment Edited] (KAFKA-2350) Add KafkaConsumer pause capability

2015-07-22 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14637187#comment-14637187 ] Jason Gustafson edited comment on KAFKA-2350 at 7/22/15 5:01 PM: ---

[jira] [Commented] (KAFKA-2350) Add KafkaConsumer pause capability

2015-07-22 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14637187#comment-14637187 ] Jason Gustafson commented on KAFKA-2350: [~becket_qin] [~jkreps] Currently automat

[jira] [Commented] (KAFKA-2350) Add KafkaConsumer pause capability

2015-07-22 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14637179#comment-14637179 ] Guozhang Wang commented on KAFKA-2350: -- Currently there is already a function for ret

Re: Review Request 36681: Patch for KAFKA-2275

2015-07-22 Thread Jason Gustafson
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36681/#review92604 --- Hey Ashish, this looks pretty good to me. Just some minor comments.

[jira] [Commented] (KAFKA-2350) Add KafkaConsumer pause capability

2015-07-22 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14637126#comment-14637126 ] Jay Kreps commented on KAFKA-2350: -- [~becket_qin] Cool, we're on the same page, that was

Re: Review Request 33620: Patch for KAFKA-1690

2015-07-22 Thread Ismael Juma
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33620/#review92599 --- clients/src/main/java/org/apache/kafka/common/network/SSLTransportL

Re: Review Request 33620: Patch for KAFKA-1690

2015-07-22 Thread Ismael Juma
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33620/#review92405 --- clients/src/main/java/org/apache/kafka/common/network/ByteBufferSen

Re: Review Request 33620: Patch for KAFKA-1690

2015-07-22 Thread Sriharsha Chintalapani
> On July 22, 2015, 3 p.m., Jun Rao wrote: > > clients/src/main/java/org/apache/kafka/common/network/SSLTransportLayer.java, > > lines 231-234 > > > > > > Still some questions on this. > > > > 1. When hand

Re: Review Request 33620: Patch for KAFKA-1690

2015-07-22 Thread Sriharsha Chintalapani
> On July 22, 2015, 3 p.m., Jun Rao wrote: > > clients/src/main/java/org/apache/kafka/common/network/SSLTransportLayer.java, > > line 411 > > > > > > Is renegotiation actually supported? It seems that renegotiation

Re: Review Request 33620: Patch for KAFKA-1690

2015-07-22 Thread Sriharsha Chintalapani
> On July 22, 2015, 3 p.m., Jun Rao wrote: > > clients/src/main/java/org/apache/kafka/common/network/SSLTransportLayer.java, > > lines 361-374 > > > > > > Hmm, if status is ok and handshakeStatus is NEED_UNWRAP, co

Re: Review Request 33620: Patch for KAFKA-1690

2015-07-22 Thread Jun Rao
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33620/#review92314 --- Thanks for the patch. Looks good overall. A few comments below. The

[jira] [Updated] (KAFKA-2345) Attempt to delete a topic already marked for deletion throws ZkNodeExistsException

2015-07-22 Thread Stevo Slavic (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stevo Slavic updated KAFKA-2345: Affects Version/s: 0.8.2.0 > Attempt to delete a topic already marked for deletion throws > ZkNodeE

[GitHub] kafka pull request: ConsumerRecords are organized per topic partit...

2015-07-22 Thread sslavic
GitHub user sslavic opened a pull request: https://github.com/apache/kafka/pull/92 ConsumerRecords are organized per topic partition ConsumerRecords has records organized per topic partition, not per topic as ConsumerRecords javadoc suggested. You can merge this pull request into a

[jira] [Created] (KAFKA-2356) Support retrieving partitions of ConsumerRecords

2015-07-22 Thread Stevo Slavic (JIRA)
Stevo Slavic created KAFKA-2356: --- Summary: Support retrieving partitions of ConsumerRecords Key: KAFKA-2356 URL: https://issues.apache.org/jira/browse/KAFKA-2356 Project: Kafka Issue Type: Impr

Re: Review Request 36670: Patch for KAFKA-2355

2015-07-22 Thread Grant Henke
> On July 22, 2015, 10:29 a.m., Ismael Juma wrote: > > core/src/test/scala/unit/kafka/admin/DeleteTopicTest.scala, line 269 > > > > > > Should `verifyTopicDeletion` be inside `finally`? I don't think so > > because

Re: Should 0.8.3 consumers correctly function with 0.8.2 brokers?

2015-07-22 Thread Stevo Slavić
I'm getting "Unknown api code 11" even when both client and server are 0.8.3/trunk, when "KafkaConsumer.subscribe(String... topics)" is used. Bug? Kind regards, Stevo Slavic. On Fri, Apr 24, 2015 at 7:13 PM, Neha Narkhede wrote: > Yes, I was clearly confused :-) > > On Fri, Apr 24, 2015 at 9:3

Re: Kafka Unit Test Failures on a Mac

2015-07-22 Thread Ismael Juma
On Mon, Jul 20, 2015 at 10:45 PM, Grant Henke wrote: > In one run of the core tests I found the following: > >- 4584 regular files (REG) >- 376 .jar files > - Not much one can/should do here. Many are from gradle itself. > - 2392 kafka .log files > - why are these

Re: Review Request 36664: Patch for KAFKA-2353

2015-07-22 Thread Ismael Juma
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36664/#review92574 --- core/src/main/scala/kafka/network/SocketServer.scala (line 401)

Re: Review Request 36670: Patch for KAFKA-2355

2015-07-22 Thread Ismael Juma
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36670/#review92571 --- core/src/test/scala/unit/kafka/admin/DeleteTopicTest.scala (line 26

Re: Official Kafka Gitter Room?

2015-07-22 Thread Stevo Slavić
On Apache Mahout project we're using Slack as well - for release coordination. It was found that extra Slack channel does not really fit into Apache way - it was overused, there were too many design discussions going on there, to which community at large has no access to, was not and could not be i

Re: Official Kafka Gitter Room?

2015-07-22 Thread Ismael Juma
Hi Gwen, On Sun, Jul 19, 2015 at 2:26 AM, Gwen Shapira wrote: > So, as an experiment, I created: > https://apachekafka.slack.com > > I figured we'll give it a whirl for a week or two for dev discussions, > see how it goes and if we have activity we can add this to the website > and announce on t

[DISCUSS] Partitioning in Kafka

2015-07-22 Thread Gianmarco De Francisci Morales
Hello folks, I'd like to ask the community about its opinion on the partitioning functions in Kafka. With KAFKA-2091 integrated we are now able to have custom partitioners in the producer. The question now becomes *which* partitioners should ship