[jira] [Created] (KAFKA-1316) Refactor Sender

Jay Kreps created KAFKA-1316: Summary: Refactor Sender Key: KAFKA-1316 URL: https://issues.apache.org/jira/browse/KAFKA-1316 Project: Kafka Issue Type: Sub-task Reporter: Jay Kreps

[jira] [Commented] (KAFKA-1315) log.dirs property in KafkaServer intolerant of trailing slash

[ https://issues.apache.org/jira/browse/KAFKA-1315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13941356#comment-13941356 ] Jay Kreps commented on KAFKA-1315: -- This seems pretty bad since many people always add a

Re: Review Request 18299: Fix KAFKA-1253

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18299/#review37837 --- clients/src/main/java/org/apache/kafka/common/record/ByteBufferOutp

Re: Review Request 18299: Fix KAFKA-1253

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18299/#review37846 --- clients/src/main/java/org/apache/kafka/common/record/Compressor.jav

Re: Review Request 18299: Fix KAFKA-1253

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18299/#review37838 --- clients/src/main/java/org/apache/kafka/common/record/Compressor.jav

[jira] [Updated] (KAFKA-1315) log.dirs property in KafkaServer intolerant of trailing slash

[ https://issues.apache.org/jira/browse/KAFKA-1315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brent Bradbury updated KAFKA-1315: -- Description: A trailing slash in log.dirs causes a java.util.NoSuchElementException on the pro

Re: Review Request 18299: Fix KAFKA-1253

> On March 19, 2014, 3:17 a.m., Jay Kreps wrote: > > clients/src/main/java/org/apache/kafka/common/record/Compressor.java, line > > 103 > > > > > > These should not be swallowed. > > Guozhang Wang wrote: > The o

[jira] [Updated] (KAFKA-1315) log.dirs property in KafkaServer intolerant of trailing slash

[ https://issues.apache.org/jira/browse/KAFKA-1315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brent Bradbury updated KAFKA-1315: -- Description: A trailing slash in log.dirs causes a java.util.NoSuchElementException on the pro

[jira] [Updated] (KAFKA-1315) log.dirs property in KafkaServer intolerant of trailing slash

[ https://issues.apache.org/jira/browse/KAFKA-1315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brent Bradbury updated KAFKA-1315: -- Description: A trailing slash in log.dirs causes a java.util.NoSuchElementException on the pro

[jira] [Updated] (KAFKA-1315) log.dirs property in KafkaServer intolerant of trailing slash

[ https://issues.apache.org/jira/browse/KAFKA-1315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brent Bradbury updated KAFKA-1315: -- Description: A trailing slash in log.dirs causes a java.util.NoSuchElementException on the pro

[jira] [Updated] (KAFKA-1315) log.dirs property in KafkaServer intolerant of trailing slash

[ https://issues.apache.org/jira/browse/KAFKA-1315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brent Bradbury updated KAFKA-1315: -- Description: A trailing slash in log.dirs causes a java.util.NoSuchElementException on the pro

[jira] [Created] (KAFKA-1315) log.dirs property in KafkaServer intolerant of trailing slash

Brent Bradbury created KAFKA-1315: - Summary: log.dirs property in KafkaServer intolerant of trailing slash Key: KAFKA-1315 URL: https://issues.apache.org/jira/browse/KAFKA-1315 Project: Kafka

[jira] [Updated] (KAFKA-1251) Add metrics to the producer

[ https://issues.apache.org/jira/browse/KAFKA-1251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-1251: - Attachment: KAFKA-1251_2014-03-19_17:30:32.patch > Add metrics to the producer >

[jira] [Commented] (KAFKA-1251) Add metrics to the producer

[ https://issues.apache.org/jira/browse/KAFKA-1251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13941241#comment-13941241 ] Jay Kreps commented on KAFKA-1251: -- Updated reviewboard https://reviews.apache.org/r/1938

Re: Review Request 19388: Patch for KAFKA-1251

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/19388/ --- (Updated March 20, 2014, 12:30 a.m.) Review request for kafka. Bugs: KAFKA-12

[jira] [Commented] (KAFKA-1303) metadata request in the new producer can be delayed

[ https://issues.apache.org/jira/browse/KAFKA-1303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13941240#comment-13941240 ] Jay Kreps commented on KAFKA-1303: -- Oops, I made a mistake in the calculations in the las

[jira] [Commented] (KAFKA-1303) metadata request in the new producer can be delayed

[ https://issues.apache.org/jira/browse/KAFKA-1303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13941237#comment-13941237 ] Jay Kreps commented on KAFKA-1303: -- Jun and I chatted briefly. The old approach is quite

[jira] [Commented] (KAFKA-1303) metadata request in the new producer can be delayed

[ https://issues.apache.org/jira/browse/KAFKA-1303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13941186#comment-13941186 ] Guozhang Wang commented on KAFKA-1303: -- I think the old producer approach would be a

[jira] [Commented] (KAFKA-1253) Implement compression in new producer

[ https://issues.apache.org/jira/browse/KAFKA-1253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13941177#comment-13941177 ] Guozhang Wang commented on KAFKA-1253: -- Updated reviewboard https://reviews.apache.or

[jira] [Updated] (KAFKA-1253) Implement compression in new producer

[ https://issues.apache.org/jira/browse/KAFKA-1253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-1253: - Attachment: KAFKA-1253_2014-03-19_16:31:39.patch > Implement compression in new producer > --

Re: Review Request 18299: Fix KAFKA-1253

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18299/ --- (Updated March 19, 2014, 11:32 p.m.) Review request for kafka. Bugs: KAFKA-12

Re: Review Request 18299: Fix KAFKA-1253

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18299/ --- (Updated March 19, 2014, 11:31 p.m.) Review request for kafka. Bugs: KAFKA-12

Re: Review Request 18299: Fix KAFKA-1253

> On March 19, 2014, 3:17 a.m., Jay Kreps wrote: > > clients/src/main/java/org/apache/kafka/common/record/CompressionFactory.java, > > line 46 > > > > > > I don't think this is needed. Inside an if statement you shoul

Re: Review Request 18299: Fix KAFKA-1253

> On March 19, 2014, 3:17 a.m., Jay Kreps wrote: > > clients/src/main/java/org/apache/kafka/clients/producer/internals/RecordBatch.java, > > line 58 > > > > > > What is the reason for this check? There is a corner ca

[jira] [Resolved] (KAFKA-1312) Update gitignore to ignore IDE generated files

[ https://issues.apache.org/jira/browse/KAFKA-1312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps resolved KAFKA-1312. -- Resolution: Fixed Fix Version/s: 0.8.2 Applied, thanks! > Update gitignore to ignore IDE genera

Re: Review Request 19421: Patch for KAFKA-1312

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/19421/#review37802 --- Ship it! Ship It! - Guozhang Wang On March 19, 2014, 8:21 p.m.,

[jira] [Commented] (KAFKA-1) The log4j appender still uses the SyncProducer API

[ https://issues.apache.org/jira/browse/KAFKA-1?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13940954#comment-13940954 ] Timothy Chen commented on KAFKA-1: -- Updated reviewboard against branch origin/trunk > The

[jira] [Updated] (KAFKA-1) The log4j appender still uses the SyncProducer API

[ https://issues.apache.org/jira/browse/KAFKA-1?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy Chen updated KAFKA-1: - Attachment: KAFKA-1_2014-03-19_13:38:39.patch > The log4j appender still uses the SyncProducer API > ---

[jira] [Commented] (KAFKA-1312) Update gitignore to ignore IDE generated files

[ https://issues.apache.org/jira/browse/KAFKA-1312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13940938#comment-13940938 ] Timothy Chen commented on KAFKA-1312: - Created reviewboard https://reviews.apache.org/

[jira] [Updated] (KAFKA-1312) Update gitignore to ignore IDE generated files

[ https://issues.apache.org/jira/browse/KAFKA-1312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy Chen updated KAFKA-1312: Attachment: KAFKA-1312.patch > Update gitignore to ignore IDE generated files > ---

Review Request 19421: Patch for KAFKA-1312

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/19421/ --- Review request for kafka. Bugs: KAFKA-1312 https://issues.apache.org/jira/b

Re: Review Request 19388: Patch for KAFKA-1251

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/19388/#review37791 --- Ship it! Ship It! - Timothy Chen On March 19, 2014, 5:29 p.m., J

[jira] [Created] (KAFKA-1314) recurring errors

alex kamil created KAFKA-1314: - Summary: recurring errors Key: KAFKA-1314 URL: https://issues.apache.org/jira/browse/KAFKA-1314 Project: Kafka Issue Type: Bug Affects Versions: 0.8.0

[jira] [Commented] (KAFKA-1306) Offset commit API, does it work?

[ https://issues.apache.org/jira/browse/KAFKA-1306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13940890#comment-13940890 ] korebantic2 commented on KAFKA-1306: Yup. As you said, it wasn't documented correctly.

[jira] [Comment Edited] (KAFKA-1306) Offset commit API, does it work?

[ https://issues.apache.org/jira/browse/KAFKA-1306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13940890#comment-13940890 ] korebantic2 edited comment on KAFKA-1306 at 3/19/14 7:33 PM: -

[jira] [Comment Edited] (KAFKA-1306) Offset commit API, does it work?

[ https://issues.apache.org/jira/browse/KAFKA-1306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13940890#comment-13940890 ] korebantic2 edited comment on KAFKA-1306 at 3/19/14 7:32 PM: -

[jira] [Comment Edited] (KAFKA-1306) Offset commit API, does it work?

[ https://issues.apache.org/jira/browse/KAFKA-1306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13940890#comment-13940890 ] korebantic2 edited comment on KAFKA-1306 at 3/19/14 7:32 PM: -

[jira] [Resolved] (KAFKA-1306) Offset commit API, does it work?

[ https://issues.apache.org/jira/browse/KAFKA-1306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] korebantic2 resolved KAFKA-1306. Resolution: Fixed > Offset commit API, does it work? > > >

[jira] [Updated] (KAFKA-1313) Support adding replicas to existing topic partitions

[ https://issues.apache.org/jira/browse/KAFKA-1313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neha Narkhede updated KAFKA-1313: - Priority: Critical (was: Minor) > Support adding replicas to existing topic partitions > ---

[jira] [Updated] (KAFKA-1313) Support adding replicas to existing topic partitions

[ https://issues.apache.org/jira/browse/KAFKA-1313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neha Narkhede updated KAFKA-1313: - Fix Version/s: 0.8.2 > Support adding replicas to existing topic partitions > ---

[jira] [Created] (KAFKA-1313) Support adding replicas to existing topic partitions

Marc Labbe created KAFKA-1313: - Summary: Support adding replicas to existing topic partitions Key: KAFKA-1313 URL: https://issues.apache.org/jira/browse/KAFKA-1313 Project: Kafka Issue Type: New

Re: Requesting advice about Producer & Consumer Design

Hi Jay, Thanks for taking time for the details ! Appreciate that. Just clarifying myself couple of things in the same line. 1) The new producer and consumer is being designed to take care of auto balancing between partitions. Right ? 2) with the current available producer and consumer, is my cu

[jira] [Commented] (KAFKA-1303) metadata request in the new producer can be delayed

[ https://issues.apache.org/jira/browse/KAFKA-1303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13940748#comment-13940748 ] Jun Rao commented on KAFKA-1303: The way the old producer works is the following. The prod

[jira] [Updated] (KAFKA-1251) Add metrics to the producer

[ https://issues.apache.org/jira/browse/KAFKA-1251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-1251: - Attachment: KAFKA-1251_2014-03-19_10:19:27.patch > Add metrics to the producer >

[jira] [Created] (KAFKA-1312) Update gitignore to ignore IDE generated files

Timothy Chen created KAFKA-1312: --- Summary: Update gitignore to ignore IDE generated files Key: KAFKA-1312 URL: https://issues.apache.org/jira/browse/KAFKA-1312 Project: Kafka Issue Type: Improv

Re: Review Request 19388: Patch for KAFKA-1251

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/19388/ --- (Updated March 19, 2014, 5:29 p.m.) Review request for kafka. Bugs: KAFKA-125

[jira] [Commented] (KAFKA-1251) Add metrics to the producer

[ https://issues.apache.org/jira/browse/KAFKA-1251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13940721#comment-13940721 ] Jay Kreps commented on KAFKA-1251: -- Updated reviewboard https://reviews.apache.org/r/1938

Re: Review Request 19388: Patch for KAFKA-1251

> On March 19, 2014, 6:49 a.m., Timothy Chen wrote: > > clients/src/main/java/org/apache/kafka/clients/tools/ProducerPerformance.java, > > line 48 > > > > > > Not sure if this is intentional. Hey that was going to be

[jira] [Updated] (KAFKA-1251) Add metrics to the producer

[ https://issues.apache.org/jira/browse/KAFKA-1251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-1251: - Attachment: KAFKA-1251_2014-03-19_10:29:05.patch > Add metrics to the producer >

[jira] [Commented] (KAFKA-1251) Add metrics to the producer

[ https://issues.apache.org/jira/browse/KAFKA-1251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13940708#comment-13940708 ] Jay Kreps commented on KAFKA-1251: -- Updated reviewboard https://reviews.apache.org/r/1938

Re: Review Request 19388: Patch for KAFKA-1251

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/19388/ --- (Updated March 19, 2014, 5:19 p.m.) Review request for kafka. Summary (update

Re: [DISCUSS] 0.8.1.1 vs 0.8.2

The problem with doing this point release from trunk is that it will pick up the offset management feature, which is pretty large and risky for the same reasons. I think it is worth taking blocker bug fixes only on 0.8.1 branch and in two weeks or so release 0.8.1.1. Thanks, Neha On Wed, Mar 19,

[jira] [Updated] (KAFKA-1305) Controller can hang on controlled shutdown with auto leader balance enabled

[ https://issues.apache.org/jira/browse/KAFKA-1305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neha Narkhede updated KAFKA-1305: - Priority: Blocker (was: Major) > Controller can hang on controlled shutdown with auto leader bal

[jira] [Commented] (KAFKA-1306) Offset commit API, does it work?

[ https://issues.apache.org/jira/browse/KAFKA-1306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13940636#comment-13940636 ] korebantic2 commented on KAFKA-1306: Man Jimi had it all wrong, it should have been if

Re: Requesting advice about Producer & Consumer Design

Hey Krishna, Let me clarify the current state of things a little: 1. Kafka offers a single producer interface as well as two consumer interfaces: the low-level "simple consumer" which just directly makes network requests, and the higher level interface which handles fault-tolerance, partition assi

[DISCUSS] 0.8.1.1 vs 0.8.2

I wanted to start a discussion thread around the next release. 1) Are we going to-do a release that has the fixes for the critical issues that came up with 0.8.1? 2) If we do a release with those fixes are we doing them off of trunk or back port them to 0.8.1 branch and tag that as 0.8.1.1 o

Requesting advice about Producer & Consumer Design

Hello Experts & Kafka Team, Its existing to learn and work on using Kafka. I have been going through lot of pages and Q&A. We are building an infra & topology using Kafka for events processing in our application. We need some advice about designing the Producer and Consumer. *Please find attach