[jira] [Updated] (KAFKA-1926) Replace kafka.utils.Utils with o.a.k.common.utils.Utils

2015-02-07 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-1926: - Description: There is currently a lot of duplication between the Utils class in common and the one in

[jira] [Created] (KAFKA-1926) Replace kafka.utils.Utils with o.a.k.common.utils.Utils

2015-02-07 Thread Jay Kreps (JIRA)
Jay Kreps created KAFKA-1926: Summary: Replace kafka.utils.Utils with o.a.k.common.utils.Utils Key: KAFKA-1926 URL: https://issues.apache.org/jira/browse/KAFKA-1926 Project: Kafka Issue Type

[jira] [Updated] (KAFKA-1865) Add a flush() call to the new producer API

2015-02-07 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-1865: - Summary: Add a flush() call to the new producer API (was: Investigate adding a flush() call to new

[jira] [Updated] (KAFKA-1865) Investigate adding a flush() call to new producer API

2015-02-07 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-1865: - Assignee: Jay Kreps Status: Patch Available (was: Open) > Investigate adding a flush() call to

[jira] [Commented] (KAFKA-1865) Investigate adding a flush() call to new producer API

2015-02-07 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14310927#comment-14310927 ] Jay Kreps commented on KAFKA-1865: -- Created reviewboard https://reviews.apache.o

[jira] [Updated] (KAFKA-1865) Investigate adding a flush() call to new producer API

2015-02-07 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-1865: - Attachment: KAFKA-1865.patch > Investigate adding a flush() call to new producer

[jira] [Commented] (KAFKA-1646) Improve consumer read performance for Windows

2015-02-07 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14310912#comment-14310912 ] Jay Kreps commented on KAFKA-1646: -- Hey guys if this forces full recovery the impac

[jira] [Commented] (KAFKA-1865) Investigate adding a flush() call to new producer API

2015-02-06 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14310590#comment-14310590 ] Jay Kreps commented on KAFKA-1865: -- A key aspect of this that isn't obviou

Re: [DISCUSS] KIPs

2015-02-06 Thread Jay Kreps
t; >> > Why not go with usual definition: "lazy" = "No strong > objections for > > > > >>few > > > > >> > days"? > > > > >> > This means contributors will not be blocked on issues where no > one > > &

Re: [DISCUSS] KIPs

2015-02-05 Thread Jay Kreps
f > there is a reasonable objection. > > On Thu, Feb 05, 2015 at 06:17:44PM -0500, Joe Stein wrote: > > +1 > > > > On Thu, Feb 5, 2015 at 6:16 PM, Neha Narkhede wrote: > > > > > Sounds good. > > > > > > On Thu, Feb 5, 2015 at 2:35 PM, Jay

Re: [DISCUSS] KIPs

2015-02-05 Thread Jay Kreps
. > > Any objection to this? > > On Sun, Jan 18, 2015 at 10:31:08AM -0800, Jay Kreps wrote: > > Great! Sounds like everyone is on the same page > > > >- I created a template page to make things easier. If you do > Tools->Copy > >on this page you

[jira] [Updated] (KAFKA-1919) Metadata request issued with no backoff in new producer if there are no topics

2015-02-04 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-1919: - Attachment: KAFKA-1919-v1.patch > Metadata request issued with no backoff in new producer if there are

[jira] [Commented] (KAFKA-1919) Metadata request issued with no backoff in new producer if there are no topics

2015-02-04 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14306357#comment-14306357 ] Jay Kreps commented on KAFKA-1919: -- Here is what I think is happening: in 0.8.1

[jira] [Commented] (KAFKA-1760) Implement new consumer client

2015-02-04 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14306332#comment-14306332 ] Jay Kreps commented on KAFKA-1760: -- Actually shameful to say but I'm not sure

[jira] [Updated] (KAFKA-1919) Metadata request issued with no backoff in new producer if there are no topics

2015-02-04 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-1919: - Affects Version/s: 0.8.2 > Metadata request issued with no backoff in new producer if there are no top

[jira] [Created] (KAFKA-1919) Metadata request issued with no backoff in new producer if there are no topics

2015-02-04 Thread Jay Kreps (JIRA)
Jay Kreps created KAFKA-1919: Summary: Metadata request issued with no backoff in new producer if there are no topics Key: KAFKA-1919 URL: https://issues.apache.org/jira/browse/KAFKA-1919 Project: Kafka

Re: kafka.utils.DoublyLinkedList

2015-02-04 Thread Jay Kreps
> - the scaladocs seem to indicate that the doubly linked lists are > deprecated as of 2.11 > > Joel > > On Wednesday, February 4, 2015, Jay Kreps wrote: > > > Hey Jiangjie, > > > > Both the scala and java doubly linked lists allow O(1) deletion--it isn't &g

Re: kafka.utils.DoublyLinkedList

2015-02-04 Thread Jay Kreps
ter on we also want to have unit > test for it, that¹s why it becomes a separate util class and we have unit > test in UtilsTest.scala. > > -Jiangjie (Becket) Qin > > On 2/3/15, 9:15 PM, "Jay Kreps" wrote: > > >How come we added this? Is it better than the linked

kafka.utils.DoublyLinkedList

2015-02-03 Thread Jay Kreps
How come we added this? Is it better than the linked list in java and scala? Also it was added with only one test which is actually put inside UtilsTest.scala which is meant to house tests for Utils.scala. -Jay

checkstyle

2015-02-03 Thread Jay Kreps
I just committed the checkstyle patch (KAFKA-1915). This will do the checkstyle evaluation when you run the unit tests and error on style inconsistencies. It can be hard to say which checks are more trouble than they are worth or are two pedantic. Let's give this a try and see if it is annoying or

[jira] [Updated] (KAFKA-1915) Integrate checkstyle for java code

2015-02-03 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-1915: - Attachment: KAFKA-1915_2015-02-03_09:17:34.patch > Integrate checkstyle for java c

[jira] [Commented] (KAFKA-1915) Integrate checkstyle for java code

2015-02-03 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14303614#comment-14303614 ] Jay Kreps commented on KAFKA-1915: -- Updated reviewboard https://reviews.apache.o

Re: Review Request 30547: Integrate checkstyle.

2015-02-03 Thread Jay Kreps
/java/kafka/examples/SimpleConsumerDemo.java c79192c5c195d4c3a7facf1be8f503478b6cc809 Diff: https://reviews.apache.org/r/30547/diff/ Testing --- Thanks, Jay Kreps

Re: Review Request 30547: Integrate checkstyle.

2015-02-03 Thread Jay Kreps
an existing package in common you would need to think about this dependency and explicitly whitelist it. This will probably lead to better code modularity but may be slightly annoying. - Jay --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30547/#r

[jira] [Updated] (KAFKA-1915) Integrate checkstyle for java code

2015-02-02 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-1915: - Attachment: KAFKA-1915.patch > Integrate checkstyle for java c

[jira] [Updated] (KAFKA-1915) Integrate checkstyle for java code

2015-02-02 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-1915: - Status: Patch Available (was: Open) > Integrate checkstyle for java c

[jira] [Commented] (KAFKA-1915) Integrate checkstyle for java code

2015-02-02 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14302796#comment-14302796 ] Jay Kreps commented on KAFKA-1915: -- Created reviewboard https://reviews.apache.o

Review Request 30547: Integrate checkstyle.

2015-02-02 Thread Jay Kreps
/ Testing --- Thanks, Jay Kreps

[jira] [Created] (KAFKA-1915) Integrate checkstyle for java code

2015-02-02 Thread Jay Kreps (JIRA)
Jay Kreps created KAFKA-1915: Summary: Integrate checkstyle for java code Key: KAFKA-1915 URL: https://issues.apache.org/jira/browse/KAFKA-1915 Project: Kafka Issue Type: Improvement

Re: [kafka-clients] Re: [VOTE] 0.8.2.0 Candidate 3

2015-02-02 Thread Jay Kreps
Yay! -Jay On Mon, Feb 2, 2015 at 2:23 PM, Neha Narkhede wrote: > Great! Thanks Jun for helping with the release and everyone involved for > your contributions. > > On Mon, Feb 2, 2015 at 1:32 PM, Joe Stein wrote: > > > Huzzah! > > > > Thanks Jun for preparing the release candidates and getting

Re: Review Request 30482: Add the coordinator to server

2015-02-01 Thread Jay Kreps
core/src/main/scala/kafka/server/KafkaApis.scala <https://reviews.apache.org/r/30482/#comment115739> ditto Not sure this stuff is actually here for review...may still be a work in progress. Overall this structure of code makes a ton of sense to me. Left some minor comment

Re: Review Request 30403: Patch for KAFKA-1906

2015-02-01 Thread Jay Kreps
gory. > > -Jaikiran > > > On Thursday 29 January 2015 08:00 PM, Jeff Holoman wrote: > > Maybe I'm in the minority here, but I actually don't think there should be > a default for this param and you should be required to explicitly set this. > > On Thu, Jan 29

[jira] [Created] (KAFKA-1912) Create a simple request re-routing facility

2015-02-01 Thread Jay Kreps (JIRA)
Jay Kreps created KAFKA-1912: Summary: Create a simple request re-routing facility Key: KAFKA-1912 URL: https://issues.apache.org/jira/browse/KAFKA-1912 Project: Kafka Issue Type: Improvement

[jira] [Commented] (KAFKA-1910) Refactor KafkaConsumer

2015-02-01 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14300251#comment-14300251 ] Jay Kreps commented on KAFKA-1910: -- I guess the real question is how to slice it up.

Re: [VOTE] 0.8.2.0 Candidate 3

2015-02-01 Thread Jay Kreps
You may already know this but the producer doesn't require a complete list of brokers in its config, it just requires the connection info for one active broker which it uses to discover the rest of the brokers. We allow you to specify multiple urls here for failover in cases where you aren't using

[jira] [Commented] (KAFKA-1905) KafkaProducer's performance could be halved when MaxInFlightRequest is set to 1

2015-01-29 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14298106#comment-14298106 ] Jay Kreps commented on KAFKA-1905: -- It's not every day you double performance

[jira] [Commented] (KAFKA-1760) Implement new consumer client

2015-01-29 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14296710#comment-14296710 ] Jay Kreps commented on KAFKA-1760: -- Updated reviewboard https://reviews.apache.o

[jira] [Updated] (KAFKA-1760) Implement new consumer client

2015-01-29 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-1760: - Attachment: KAFKA-1760_2015-01-29_03:20:20.patch > Implement new consumer cli

Re: Review Request 27799: New consumer

2015-01-29 Thread Jay Kreps
ac15d34425795d5be20c51b01fa1108bdcd66583 Diff: https://reviews.apache.org/r/27799/diff/ Testing --- Thanks, Jay Kreps

Re: Review Request 27799: New consumer

2015-01-29 Thread Jay Kreps
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27799/#review69936 ------- On Jan. 23, 2015, 9:15 p.m., Jay Kreps wrote: > > --

Re: Review Request 27799: New consumer

2015-01-29 Thread Jay Kreps
to html codes intentional? Yeah my IDE seems to do that. Kind of annoying. - Jay --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27799/#review69737 ------- On Jan. 23, 2015, 9:15 p.m., Jay K

Re: Review Request 27799: New consumer

2015-01-29 Thread Jay Kreps
e the public from the interface methods? > > Jay Kreps wrote: > Can you explain...? > > Aditya Auradkar wrote: > I gather all interface methods are implicitly public.. so that should be > unnecessary. > > Aditya Auradkar wrote: > http://docs.oracle.com/

Re: Review Request 27799: New consumer

2015-01-29 Thread Jay Kreps
ally. > > > > In general, I would suggest we add the @throws label to private > > functions also for easy maintainning the throwable exceptions. Yeah the behavior is actually correct, I think. The general philosophy is that if the error is recoverable we should log it and keep trying but if it is fatal we should thro

Re: Review Request 30403: Patch for KAFKA-1906

2015-01-29 Thread Jay Kreps
cases > default it to a path which resides under the Kafka install directory. > > If we use the --override option in our (startup) scripts to set > log.dir=${KAFKA_HOME}/data, we will end up forcing this value as the log.dir > even when the user has intentionally specifie

Re: Review Request 30403: Patch for KAFKA-1906

2015-01-29 Thread Jay Kreps
> On Jan. 29, 2015, 6:50 a.m., Gwen Shapira wrote: > > We added --override option to KafkaServer that allows overriding default > > configuration from commandline. > > I believe that just changing the shell script to include --override > > log.dir=${KAFKA_HOME}/data > > may be enough? > > > >

Re: [DISCUSSION] Boot dependency in the new producer

2015-01-26 Thread Jay Kreps
Hey Guozhang, That line shouldn't cause any connections to Kafka to be established, does it? All that is doing is creating the Cluster pojo using the supplied addresses. The use of InetSocketAddress may cause some dns stuff to happen, though... -Jay On Mon, Jan 26, 2015 at 10:50 AM, Guozhang Wan

Re: What to do when file.rename fails?

2015-01-26 Thread Jay Kreps
know about. There may be a larger issue where we have bugs that can lead to double deleting a log segment which currently is innocuous but if we made it shut down the server would be quite severe. -Jay On Mon, Jan 26, 2015 at 1:06 PM, Jay Kreps wrote: > Having a relative path and keeping data u

Re: What to do when file.rename fails?

2015-01-26 Thread Jay Kreps
A_INSTALL_FOLDER/data/kafka-logs/ folder. Is > that something we should do? > > -Jaikiran > > On Monday 26 January 2015 12:23 AM, Jay Kreps wrote: > >> Hmm, but I don't think tmp gets cleaned while the server is running... >> >> The reason for using tmp was beca

Re: What to do when file.rename fails?

2015-01-25 Thread Jay Kreps
incipal Consultant > Big Data Open Source Security LLC > http://www.stealth.ly > Twitter: @allthingshadoop > ********/ > On Jan 25, 2015 11:37 AM, "Jay Kreps" wrote: > > > I think you are right, good catch. It could be that this u

Re: What to do when file.rename fails?

2015-01-25 Thread Jay Kreps
pen file resources from this program > System.out.println("File channel closed. Check the lsof output and > press any key to terminate the program"); > System.in.read(); > // all done, exit > System.out.println("Program will terminate"

Re: [kafka-clients] Re: [VOTE] 0.8.2.0 Candidate 2 (with the correct links)

2015-01-24 Thread Jay Kreps
s just makes things worse. -Jay On Sat, Jan 24, 2015 at 2:51 PM, Joe Stein wrote: > Maybe. I think the StringSerialzer could look more like a typical type of > message. Instead of encoding being a property it would be more typically > just written in the bytes. > > On Sat, Jan 24, 201

What to do when file.rename fails?

2015-01-24 Thread Jay Kreps
Hey guys, Jaikiran posted a patch on KAFKA-1853 to improve the handling of failures during delete. https://issues.apache.org/jira/browse/KAFKA-1853 The core problem here is that we are doing File.rename() as part of the delete sequence which returns false if the rename failed. Or file delete sequ

Re: Review Request 29755: Patch for KAFKA-1853

2015-01-23 Thread Jay Kreps
tps://reviews.apache.org/r/29755/#comment114225> Seems like in the case of the exception we actually forcefully delete, but then after deleting schedule another one. Should the scheduler action be inside the try block so it only occurs if the rename succeeds? - Jay Kreps On Jan. 24, 2015, 5:

Re: [kafka-clients] Re: [VOTE] 0.8.2.0 Candidate 2 (with the correct links)

2015-01-23 Thread Jay Kreps
I don't think so--see if you buy my explanation. We previously defaulted to the byte array serializer and it was a source of unending frustration and confusion. Since it wasn't a required config people just went along plugging in whatever objects they had, and thinking that changing the parametric

[jira] [Commented] (KAFKA-1897) Enhance MockProducer for more sophisticated tests

2015-01-23 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14290436#comment-14290436 ] Jay Kreps commented on KAFKA-1897: -- Thanks for reporting this. It would be great to

Re: [KIP-DISCUSSION] Mirror Maker Enhancement

2015-01-23 Thread Jay Kreps
ded. Any reason this wouldn't work? -Jay On Wed, Jan 21, 2015 at 5:29 PM, Jiangjie Qin wrote: > Hi Jay, > > Thanks for comments. Please see inline responses. > > Jiangjie (Becket) Qin > > On 1/21/15, 1:33 PM, "Jay Kreps" wrote: > > >Hey guys, > &g

Re: [DISCUSS] KIP-5 - Broker Configuration Management

2015-01-23 Thread Jay Kreps
will do. > > On Fri, Jan 23, 2015 at 12:05 AM, Jay Kreps wrote: > > > Hey Joe, > > > > Can you fill in this KIP? The purpose of these KIPs is to give a full > > overview of the feature, how it will work, be implemented, the > > considerations involved, etc. There

Re: Review Request 27799: New consumer

2015-01-23 Thread Jay Kreps
/kafka/utils/TestUtils.scala ac15d34425795d5be20c51b01fa1108bdcd66583 Diff: https://reviews.apache.org/r/27799/diff/ Testing --- Thanks, Jay Kreps

Re: Review Request 27799: New consumer

2015-01-23 Thread Jay Kreps
Diff: https://reviews.apache.org/r/27799/diff/ Testing --- Thanks, Jay Kreps

Re: Review Request 27799: New consumer

2015-01-23 Thread Jay Kreps
/TestUtils.scala ac15d34425795d5be20c51b01fa1108bdcd66583 Diff: https://reviews.apache.org/r/27799/diff/ Testing --- Thanks, Jay Kreps

Re: Review Request 27799: New consumer

2015-01-23 Thread Jay Kreps
Anyhow I'll fix this case but I think we have this issue in about 1000 places and fixing it is quite ugly (see the fixed code I'm about to post) and obscures what is going on. Also fixing it with a timeout is far worse since then you have a timing assumption that will randomly fail.

[jira] [Commented] (KAFKA-1760) Implement new consumer client

2015-01-23 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14289987#comment-14289987 ] Jay Kreps commented on KAFKA-1760: -- Updated reviewboard https://reviews.apache.o

[jira] [Updated] (KAFKA-1760) Implement new consumer client

2015-01-23 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-1760: - Attachment: KAFKA-1760_2015-01-23_13:13:00.patch > Implement new consumer cli

Re: Review Request 27799: New consumer

2015-01-23 Thread Jay Kreps
tava wrote: > > core/src/main/scala/kafka/tools/ConsumerPerformance.scala, line 21 > > <https://reviews.apache.org/r/27799/diff/6/?file=831557#file831557line21> > > > > Lots of unused import cleanup here. Yeah I use Eclipse which doesn't show those for scala :-( - Jay ---

[jira] [Created] (KAFKA-1895) Investigate moving deserialization and decompression out of KafkaConsumer

2015-01-23 Thread Jay Kreps (JIRA)
Jay Kreps created KAFKA-1895: Summary: Investigate moving deserialization and decompression out of KafkaConsumer Key: KAFKA-1895 URL: https://issues.apache.org/jira/browse/KAFKA-1895 Project: Kafka

[jira] [Comment Edited] (KAFKA-1655) Allow high performance SimpleConsumer use cases to still work with new Kafka 0.9 consumer APIs

2015-01-23 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14289744#comment-14289744 ] Jay Kreps edited comment on KAFKA-1655 at 1/23/15 7:03 PM: --

[jira] [Commented] (KAFKA-1655) Allow high performance SimpleConsumer use cases to still work with new Kafka 0.9 consumer APIs

2015-01-23 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14289744#comment-14289744 ] Jay Kreps commented on KAFKA-1655: -- I believe this is handled in the new consumer

[jira] [Created] (KAFKA-1894) Avoid long or infinite blocking in the consumer

2015-01-23 Thread Jay Kreps (JIRA)
Jay Kreps created KAFKA-1894: Summary: Avoid long or infinite blocking in the consumer Key: KAFKA-1894 URL: https://issues.apache.org/jira/browse/KAFKA-1894 Project: Kafka Issue Type: Sub-task

[jira] [Created] (KAFKA-1893) Allow regex subscriptions in the new consumer

2015-01-23 Thread Jay Kreps (JIRA)
Jay Kreps created KAFKA-1893: Summary: Allow regex subscriptions in the new consumer Key: KAFKA-1893 URL: https://issues.apache.org/jira/browse/KAFKA-1893 Project: Kafka Issue Type: Sub-task

[jira] [Created] (KAFKA-1892) System tests for the new consumer and co-ordinator

2015-01-23 Thread Jay Kreps (JIRA)
Jay Kreps created KAFKA-1892: Summary: System tests for the new consumer and co-ordinator Key: KAFKA-1892 URL: https://issues.apache.org/jira/browse/KAFKA-1892 Project: Kafka Issue Type: Sub

[jira] [Commented] (KAFKA-1332) Add functionality to the offsetsBeforeTime() API

2015-01-23 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14289655#comment-14289655 ] Jay Kreps commented on KAFKA-1332: -- It will be good to rethink this API, but for n

[jira] [Resolved] (KAFKA-1331) Add ability to commit offsets to the new consumer

2015-01-23 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps resolved KAFKA-1331. -- Resolution: Duplicate Done as part of KAFKA-1760 > Add ability to commit offsets to the new consu

[jira] [Resolved] (KAFKA-1330) Implement subscribe(TopicPartition...partitions) and pool(timeout) in the consumer

2015-01-23 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps resolved KAFKA-1330. -- Resolution: Duplicate Done as part of KAFKA-1760 > Implement subscribe(TopicPartition...partiti

[jira] [Updated] (KAFKA-1329) Add metadata fetch and refresh functionality to the consumer

2015-01-23 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-1329: - Resolution: Fixed Status: Resolved (was: Patch Available) I incorporated this into KAFKA-1760

Re: [DISCUSS] KIP-6 - New reassignment partition logic for re-balancing

2015-01-22 Thread Jay Kreps
Ditto on this one. Can you give the algorithm we want to implement? Also I think in terms of scope this is just proposing to change the logic in ReassignPartitionsCommand? I think we've had the discussion various times on the mailing list that what people really want is just for Kafka to do it's b

Re: [DISCUSS] KIP-4 - Command line and centralized administrative operations

2015-01-22 Thread Jay Kreps
Hey Joe, This is great. A few comments on KIP-4 1. This is much needed functionality, but there are a lot of the so let's really think these protocols through. We really want to end up with a set of well thought-out, orthoganol apis. For this reason I think it is really important to think through

Re: [DISCUSS] KIP-5 - Broker Configuration Management

2015-01-22 Thread Jay Kreps
Hey Joe, Can you fill in this KIP? The purpose of these KIPs is to give a full overview of the feature, how it will work, be implemented, the considerations involved, etc. There is only like one sentence on this which isn't enough for anyone to know what you are thinking. Moving off of configs to

Re: Review Request 27799: New consumer

2015-01-22 Thread Jay Kreps
> On Jan. 22, 2015, 7:10 p.m., Guozhang Wang wrote: > > core/src/test/scala/unit/kafka/utils/TestUtils.scala, lines 733-743 > > <https://reviews.apache.org/r/27799/diff/5/?file=830071#file830071line733> > > > > Is this the same as createTopic in line 172?

[jira] [Updated] (KAFKA-1760) Implement new consumer client

2015-01-22 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-1760: - Attachment: KAFKA-1760_2015-01-22_20:21:56.patch > Implement new consumer cli

[jira] [Commented] (KAFKA-1760) Implement new consumer client

2015-01-22 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14288721#comment-14288721 ] Jay Kreps commented on KAFKA-1760: -- Updated reviewboard https://reviews.apache.o

Re: Review Request 27799: New consumer

2015-01-22 Thread Jay Kreps
://reviews.apache.org/r/27799/diff/ Testing --- Thanks, Jay Kreps

Re: Review Request 27799: New consumer

2015-01-22 Thread Jay Kreps
ly didn't add that method. The code style isn't even mine (e.g. I would never never put spaces after the paren in a for loop). Yet there it is. Maybe it came with the patch that added the request that I built off of...dunno. Anyhow it isn't used so I'll delete it. -

Re: [DISCUSS] KIPs

2015-01-22 Thread Jay Kreps
on to match the right protocol is irrelevant for the client. > > I really like the new design since it preserves a lot of the same > configurations and APIs. > > Thoughts? > > Gwen > > On Thu, Jan 22, 2015 at 9:19 AM, Jay Kreps wrote: > > I think I am still confused

Re: Review Request 27799: New consumer

2015-01-22 Thread Jay Kreps
ac15d34425795d5be20c51b01fa1108bdcd66583 Diff: https://reviews.apache.org/r/27799/diff/ Testing --- Thanks, Jay Kreps

[jira] [Commented] (KAFKA-1760) Implement new consumer client

2015-01-22 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14287869#comment-14287869 ] Jay Kreps commented on KAFKA-1760: -- Updated reviewboard https://reviews.apache.o

[jira] [Updated] (KAFKA-1760) Implement new consumer client

2015-01-22 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-1760: - Attachment: KAFKA-1760_2015-01-22_10:03:26.patch > Implement new consumer cli

Re: Review Request 27799: New consumer

2015-01-22 Thread Jay Kreps
://reviews.apache.org/r/27799/diff/ Testing --- Thanks, Jay Kreps

Re: Review Request 27799: New consumer

2015-01-22 Thread Jay Kreps
e the public from the interface methods? Can you explain...? - Jay --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27799/#review69204 -------

Re: Review Request 27799: New consumer

2015-01-22 Thread Jay Kreps
ted from that original patch. Fixed. - Jay --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27799/#review69117 ----------- On Jan. 21, 2015, 4:47 p.m., Jay Kreps wrote: > > --

Re: [DISCUSS] KIPs

2015-01-22 Thread Jay Kreps
the broker and producer/consumer configuration changes, > added some example values and added the new zookeeper json. > > Hope this makes things clearer. > > On Wed, Jan 21, 2015 at 2:19 PM, Jay Kreps wrote: > > Hey Gwen, > > > > Could we get the actual cha

Re: NIO and Threading implementation

2015-01-22 Thread Jay Kreps
ing in selector loops, I would love to have a conversation > with you around this. > > Thanks again .. > > > > > On Wed, Jan 21, 2015 at 2:15 PM, Jay Kreps wrote: > > > 1. a. I think startup is a public method on KafkaServer so for people > > embedding Kafka in some way th

Re: [DISCUSS] KIPs

2015-01-21 Thread Jay Kreps
on over a month > ago and implementation is close to complete (I hope!). Lets consider > this voted and accepted? > > Gwen > > On Sun, Jan 18, 2015 at 10:31 AM, Jay Kreps wrote: > > Great! Sounds like everyone is on the same page > > > >- I created a template page

Re: NIO and Threading implementation

2015-01-21 Thread Jay Kreps
1. a. I think startup is a public method on KafkaServer so for people embedding Kafka in some way this helps guarantee correctness. b. I think KafkaScheduler tries to be a bit too clever, there is a patch out there that just moves to global synchronization for the whole class which is easier to rea

Re: [KIP-DISCUSSION] Mirror Maker Enhancement

2015-01-21 Thread Jay Kreps
Hey guys, A couple questions/comments: 1. The callback and user-controlled commit offset functionality is already in the new consumer which we are working on in parallel. If we accelerated that work it might help concentrate efforts. I admit this might take slightly longer in calendar time but co

Re: Review Request 27799: New consumer

2015-01-21 Thread Jay Kreps
ac15d34425795d5be20c51b01fa1108bdcd66583 Diff: https://reviews.apache.org/r/27799/diff/ Testing --- Thanks, Jay Kreps

[jira] [Commented] (KAFKA-1760) Implement new consumer client

2015-01-21 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14285851#comment-14285851 ] Jay Kreps commented on KAFKA-1760: -- Updated reviewboard https://reviews.apache.o

[jira] [Updated] (KAFKA-1760) Implement new consumer client

2015-01-21 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-1760: - Attachment: KAFKA-1760_2015-01-21_08:42:20.patch > Implement new consumer cli

Re: Review Request 27799: New consumer

2015-01-21 Thread Jay Kreps
ac15d34425795d5be20c51b01fa1108bdcd66583 Diff: https://reviews.apache.org/r/27799/diff/ Testing --- Thanks, Jay Kreps

Re: Review Request 27799: Patch for KAFKA-1760

2015-01-21 Thread Jay Kreps
--------- On Jan. 19, 2015, 3:10 a.m., Jay Kreps wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/27799/ > -

Re: Review Request 27799: Patch for KAFKA-1760

2015-01-21 Thread Jay Kreps
views.apache.org/r/27799/diff/2/?file=816233#file816233line222> > > > > I think this case is covered by catching EOFException? No it turns out that buffer.position(val_larger_than_limit) will actually throw IllegalArgumentException - Jay ------

<    5   6   7   8   9   10   11   12   13   14   >