Re: [DISCUSS] Java 8 as a minimum requirement

2016-06-16 Thread Rajiv Kurian
+1 On Thu, Jun 16, 2016 at 1:45 PM, Ismael Juma wrote: > Hi all, > > I would like to start a discussion on making Java 8 a minimum requirement > for Kafka's next feature release (let's say Kafka 0.10.1.0 for now). This > is the first discussion on the topic so the idea is to understand how > peo

[jira] [Commented] (KAFKA-3159) Kafka consumer 0.9.0.0 client poll is very CPU intensive under certain conditions

2016-02-09 Thread Rajiv Kurian (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15140049#comment-15140049 ] Rajiv Kurian commented on KAFKA-3159: - I am running a patched broker with a cons

[jira] [Commented] (KAFKA-3159) Kafka consumer 0.9.0.0 client poll is very CPU intensive under certain conditions

2016-02-08 Thread Rajiv Kurian (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15138038#comment-15138038 ] Rajiv Kurian commented on KAFKA-3159: - I'll try it tomorrow for sure.

[jira] [Commented] (KAFKA-3159) Kafka consumer 0.9.0.0 client poll is very CPU intensive under certain conditions

2016-02-05 Thread Rajiv Kurian (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15135586#comment-15135586 ] Rajiv Kurian commented on KAFKA-3159: - Thanks Jason. I'll try to apply t

[jira] [Commented] (KAFKA-3159) Kafka consumer 0.9.0.0 client poll is very CPU intensive under certain conditions

2016-02-05 Thread Rajiv Kurian (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15134813#comment-15134813 ] Rajiv Kurian commented on KAFKA-3159: - Though I should mention that I've

[jira] [Commented] (KAFKA-3159) Kafka consumer 0.9.0.0 client poll is very CPU intensive under certain conditions

2016-02-05 Thread Rajiv Kurian (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15134776#comment-15134776 ] Rajiv Kurian commented on KAFKA-3159: - It does seem like it is related if not

[jira] [Comment Edited] (KAFKA-3159) Kafka consumer 0.9.0.0 client poll is very CPU intensive under certain conditions

2016-02-05 Thread Rajiv Kurian (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15134562#comment-15134562 ] Rajiv Kurian edited comment on KAFKA-3159 at 2/5/16 5:4

[jira] [Commented] (KAFKA-3159) Kafka consumer 0.9.0.0 client poll is very CPU intensive under certain conditions

2016-02-05 Thread Rajiv Kurian (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15134562#comment-15134562 ] Rajiv Kurian commented on KAFKA-3159: - I think I've found the isse. It turns

[jira] [Commented] (KAFKA-3200) MessageSet from broker seems invalid

2016-02-03 Thread Rajiv Kurian (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15131155#comment-15131155 ] Rajiv Kurian commented on KAFKA-3200: - Got it. Yeah it would be ideal to not

[jira] [Commented] (KAFKA-3200) MessageSet from broker seems invalid

2016-02-03 Thread Rajiv Kurian (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15131150#comment-15131150 ] Rajiv Kurian commented on KAFKA-3200: - Not attaching code any more since this s

[jira] [Commented] (KAFKA-3200) MessageSet from broker seems invalid

2016-02-03 Thread Rajiv Kurian (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15131148#comment-15131148 ] Rajiv Kurian commented on KAFKA-3200: - Yeah I have a similar workaround but

[jira] [Created] (KAFKA-3200) MessageSet from broker seems invalid

2016-02-03 Thread Rajiv Kurian (JIRA)
Rajiv Kurian created KAFKA-3200: --- Summary: MessageSet from broker seems invalid Key: KAFKA-3200 URL: https://issues.apache.org/jira/browse/KAFKA-3200 Project: Kafka Issue Type: Bug Affects

[jira] [Commented] (KAFKA-3159) Kafka consumer 0.9.0.0 client poll is very CPU intensive under certain conditions

2016-02-01 Thread Rajiv Kurian (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15126877#comment-15126877 ] Rajiv Kurian commented on KAFKA-3159: - [~hachikuji] I tried your patch.

[jira] [Updated] (KAFKA-3159) Kafka consumer 0.9.0.0 client poll is very CPU intensive under certain conditions

2016-02-01 Thread Rajiv Kurian (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajiv Kurian updated KAFKA-3159: Attachment: Memory-profile-patched-client.png Memory profile of the patched client. Notes: 1.A lot

[jira] [Updated] (KAFKA-3159) Kafka consumer 0.9.0.0 client poll is very CPU intensive under certain conditions

2016-02-01 Thread Rajiv Kurian (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajiv Kurian updated KAFKA-3159: Attachment: Screen Shot 2016-02-01 at 11.09.32 AM.png CPU break down of the patched client. Some

[jira] [Commented] (KAFKA-3159) Kafka consumer 0.9.0.0 client poll is very CPU intensive under certain conditions

2016-01-28 Thread Rajiv Kurian (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15122659#comment-15122659 ] Rajiv Kurian commented on KAFKA-3159: - Thanks Jason. I can try to do that early

[jira] [Commented] (KAFKA-3159) Kafka consumer 0.9.0.0 client poll is very CPU intensive under certain conditions

2016-01-28 Thread Rajiv Kurian (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15122614#comment-15122614 ] Rajiv Kurian commented on KAFKA-3159: - Actually I managed to dig through the logs

[jira] [Commented] (KAFKA-3159) Kafka consumer 0.9.0.0 client poll is very CPU intensive under certain conditions

2016-01-28 Thread Rajiv Kurian (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15122594#comment-15122594 ] Rajiv Kurian commented on KAFKA-3159: - I don't enable compression on

[jira] [Updated] (KAFKA-3159) Kafka consumer 0.9.0.0 client poll is very CPU intensive under certain conditions

2016-01-27 Thread Rajiv Kurian (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajiv Kurian updated KAFKA-3159: Description: We are using the new kafka consumer with the following config (as logged by kafka

[jira] [Updated] (KAFKA-3159) Kafka consumer 0.9.0.0 client poll is very CPU intensive under certain conditions

2016-01-27 Thread Rajiv Kurian (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajiv Kurian updated KAFKA-3159: Description: We are using the new kafka consumer with the following config (as logged by kafka

[jira] [Updated] (KAFKA-3159) Kafka consumer 0.9.0.0 client poll is very CPU intensive under certain conditions

2016-01-27 Thread Rajiv Kurian (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajiv Kurian updated KAFKA-3159: Description: We are using the new kafka consumer with the following config (as logged by kafka

[jira] [Updated] (KAFKA-3159) Kafka consumer 0.9.0.0 client poll is very CPU intensive under certain conditions

2016-01-27 Thread Rajiv Kurian (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajiv Kurian updated KAFKA-3159: Description: We are using the new kafka consumer with the following config (as logged by kafka

[jira] [Created] (KAFKA-3159) Kafka consumer 0.9.0.0 client poll is very CPU intensive under certain conditions

2016-01-27 Thread Rajiv Kurian (JIRA)
Rajiv Kurian created KAFKA-3159: --- Summary: Kafka consumer 0.9.0.0 client poll is very CPU intensive under certain conditions Key: KAFKA-3159 URL: https://issues.apache.org/jira/browse/KAFKA-3159

[jira] [Comment Edited] (KAFKA-2045) Memory Management on the consumer

2015-03-28 Thread Rajiv Kurian (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14385500#comment-14385500 ] Rajiv Kurian edited comment on KAFKA-2045 at 3/28/15 7:4

[jira] [Comment Edited] (KAFKA-2045) Memory Management on the consumer

2015-03-28 Thread Rajiv Kurian (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14385500#comment-14385500 ] Rajiv Kurian edited comment on KAFKA-2045 at 3/28/15 7:4

[jira] [Commented] (KAFKA-2045) Memory Management on the consumer

2015-03-28 Thread Rajiv Kurian (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14385500#comment-14385500 ] Rajiv Kurian commented on KAFKA-2045: - [~jkreps] Totally agree with you on

[jira] [Comment Edited] (KAFKA-2045) Memory Management on the consumer

2015-03-27 Thread Rajiv Kurian (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14384660#comment-14384660 ] Rajiv Kurian edited comment on KAFKA-2045 at 3/27/15 9:0

[jira] [Commented] (KAFKA-2045) Memory Management on the consumer

2015-03-27 Thread Rajiv Kurian (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14384660#comment-14384660 ] Rajiv Kurian commented on KAFKA-2045: - 1. "We can actually make serious pe

[jira] [Commented] (KAFKA-2045) Memory Management on the consumer

2015-03-27 Thread Rajiv Kurian (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14384544#comment-14384544 ] Rajiv Kurian commented on KAFKA-2045: - [~jkreps] the simple pool of ByteBuf

[jira] [Comment Edited] (KAFKA-2045) Memory Management on the consumer

2015-03-26 Thread Rajiv Kurian (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14383418#comment-14383418 ] Rajiv Kurian edited comment on KAFKA-2045 at 3/27/15 6:5

[jira] [Comment Edited] (KAFKA-2045) Memory Management on the consumer

2015-03-26 Thread Rajiv Kurian (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14383418#comment-14383418 ] Rajiv Kurian edited comment on KAFKA-2045 at 3/27/15 6:5

[jira] [Comment Edited] (KAFKA-2045) Memory Management on the consumer

2015-03-26 Thread Rajiv Kurian (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14383418#comment-14383418 ] Rajiv Kurian edited comment on KAFKA-2045 at 3/27/15 6:5

[jira] [Commented] (KAFKA-2045) Memory Management on the consumer

2015-03-26 Thread Rajiv Kurian (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14383418#comment-14383418 ] Rajiv Kurian commented on KAFKA-2045: - Copying from the email list and expanding

[jira] [Comment Edited] (KAFKA-2045) Memory Management on the consumer

2015-03-26 Thread Rajiv Kurian (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14383418#comment-14383418 ] Rajiv Kurian edited comment on KAFKA-2045 at 3/27/15 6:5

[jira] [Commented] (KAFKA-2045) Memory Management on the consumer

2015-03-26 Thread Rajiv Kurian (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14383147#comment-14383147 ] Rajiv Kurian commented on KAFKA-2045: - Bounding the ByteBuffers and static

Re: [DISCUSSION] adding the serializer api back to the new java producer

2014-12-17 Thread Rajiv Kurian
Has the mvn repo been updated too? On Wed, Dec 17, 2014 at 4:31 PM, Jun Rao wrote: > > Thanks everyone for the feedback and the discussion. The proposed changes > have been checked into both 0.8.2 and trunk. > > Jun > > On Tue, Dec 16, 2014 at 10:43 PM, Joel Koshy wrote: > > > > Jun, > > > > Tha

Re: [DISCUSSION] adding the serializer api back to the new java producer

2014-12-02 Thread Rajiv Kurian
he > underlying responses. > > Thanks, > > Jun > > On Tue, Dec 2, 2014 at 5:26 PM, Rajiv Kurian wrote: > > > I for one use the consumer (Simple Consumer) without any > deserialization. I > > just take the ByteBuffer wrap it a preallocated flyweight and use it >

Re: [DISCUSSION] adding the serializer api back to the new java producer

2014-12-02 Thread Rajiv Kurian
I for one use the consumer (Simple Consumer) without any deserialization. I just take the ByteBuffer wrap it a preallocated flyweight and use it without creating any objects. I'd ideally not have to wrap this logic in a deserializer interface. For every one who does do this, it seems like a very sm

Re: [DISCUSSION] adding the serializer api back to the new java producer

2014-12-02 Thread Rajiv Kurian
Why can't the organization package the Avro implementation with a kafka client and distribute that library though? The risk of different users supplying the kafka client with different serializer/deserializer implementations still exists. On Tue, Dec 2, 2014 at 12:11 PM, Jun Rao wrote: > Joel, R