kafka mirroring vs. replication

2011-07-20 Thread Joel Koshy
Sending this to kafka-dev since jira updates go to kafka-commits. I created this jira (https://issues.apache.org/jira/browse/KAFKA-74) that describes a small enhancement for kafka mirrors. It also contains a brief discussion on a config change and clearing up possible confusion between mirroring a

Re: [VOTE] Release Kafka 0.7.0-incubating (candidate 5)

2011-11-02 Thread Joel Koshy
One more change to the quickstart - the consumer's message streams now need to be parameterized, and we can add a note on consumer decoders. Thanks, Joel On Wed, Nov 2, 2011 at 10:56 AM, Jun Rao wrote: > +1 from me. Tested quick start and embedded_consumer test in system_test. > > One thing is

Re: [VOTE] Release Kafka 0.7.0 incubating candidate 6

2011-11-11 Thread Joel Koshy
I also ran a mirroring test and an audit. +1 (Although, I'm past the 12pm deadline.) Thanks, Joel On Tue, Nov 8, 2011 at 1:23 PM, Neha Narkhede wrote: > Hi, > This is the sixth candidate for the first incubator release for Apache > Kafka, version 0.7.0-incubating. There were a few binary dis

Re: JIRA reports

2011-11-11 Thread Joel Koshy
> any fix for that. Another way to do it would be if there is a way just to > search for things that have patch files attached, but I couldn't figure We could approximate it by filtering on attachment count - give this filter a try: https://issues.apache.org/jira/secure/IssueNavigator.jspa?mode=h

Consumer API extension in KAFKA-249

2012-03-27 Thread Joel Koshy
The newest patch for KAFKA-249 adds a new method to the consumer connector (after feedback on the first patch). Since this is an addition to the consumer API, I wanted to send it around for comments/concerns. The new method basically allows the consumer connector to subscribe to a whitelist or bl

API change in consumer

2012-03-30 Thread Joel Koshy
Hi all, This is a follow-up to the email I had sent out a few days ago on the consumer API extension as part of KAFKA-249 - after the code review, a more major API change may be more suitable, so here is an overview. The new method in the consumer connector that supports wildcarding (in the v2 pa

Re: Invitation to join Apache Kafka as a committer

2012-05-24 Thread Joel Koshy
Works now. Thanks, Joel On Wed, May 23, 2012 at 8:08 PM, Kevan Miller wrote: > > On May 23, 2012, at 10:11 PM, Alan D. Cabrera wrote: > > > -kafka-private > > +kafka-dev > > +general > > > > Ahh, account was only created. According to root: > > > >> Only PMC chairs can grant karma. If needed,

Re: setters and getters

2012-05-25 Thread Joel Koshy
Personally, I like options 3 and 4. (Option 4 more than 3, but I'm not sure I follow it correctly - and I did not know that shorthand for overloading! So is this right:) class GetSetXYZ { private var underlying = 10 def xyz = underlying def xyz_=(x: Int) {underlying = x} } val o = new GetSe

Re: setters and getters

2012-05-25 Thread Joel Koshy
Another advantage of 4 over 3 is that we don't need the case match to deal with the Option. On Fri, May 25, 2012 at 2:13 PM, Joel Koshy wrote: > Personally, I like options 3 and 4. (Option 4 more than 3, but I'm not > sure I follow it correctly - and I did not know th

Re: setters and getters

2012-05-25 Thread Joel Koshy
g this out, but I guess > better late then never. :-) > > -Jay > > On Fri, May 25, 2012 at 2:13 PM, Joel Koshy wrote: > > > Personally, I like options 3 and 4. (Option 4 more than 3, but I'm not > sure > > I follow it correctly - and I did not know that short

Re: setters and getters

2012-05-30 Thread Joel Koshy
tisfied) {underlying = update} else {throw new Exception} } All of the above changes will be binary compatible since under the covers, reads/assignments are all through getter/setter methods. Joel On Fri, May 25, 2012 at 4:58 PM, Joel Koshy wrote: > Interesting - so according to that article

Re: setters and getters

2012-05-31 Thread Joel Koshy
Agreed - if there are no objections, I can add this to our coding conventions page. Joel On Wed, May 30, 2012 at 11:24 PM, Jay Kreps wrote: > Cool, so let's standardize on that approach then? > > -Jay > > On Wed, May 30, 2012 at 4:09 PM, Joel Koshy wrote: > > >

Re: [DISCUSS] Graduate from incubator?

2012-06-06 Thread Joel Koshy
I think an 0.7.1 release would be suitable in any case given that there are few additional features and API changes since 0.7.0 - I can drive that, although an 0.7.1 release need not necessarily hold up pursuing graduation if people prefer (1). Joel On Wed, Jun 6, 2012 at 12:51 PM, Jay Kreps wro

Re: [VOTE] Release Kafka 0.7.1-incubating (Candidate 2)

2012-06-18 Thread Joel Koshy
I would like to give this a look today - I can vote by tomorrow. Thanks, Joel On Mon, Jun 18, 2012 at 10:14 AM, Jun Rao wrote: > Any other voters? > > Thanks, > > Jun > > On Thu, Jun 14, 2012 at 11:42 AM, Joe Stein wrote: > > > Hello, > > > > This is the second candidate for the second incuba

Re: Preparing for July report

2012-06-28 Thread Joel Koshy
I can write up the report for Kafka. Thanks, Joel On Thu, Jun 28, 2012 at 4:12 PM, Jukka Zitting wrote: > Hi, > > On Thu, Jun 28, 2012 at 1:48 AM, Jukka Zitting > wrote: > > It's a week until the next podling report deadline on July 4th. > > Podlings, when preparing your reports, please pay sp

Re: Incubator PMC/Board report for Jul 2012 ([ppmc])

2012-07-02 Thread Joel Koshy
Looks like I don't have write access to the incubator wiki, and I have requested it. Meanwhile, here is the content I have. Would be helpful if people can review it to see if there is anything else they would like to add: Thanks, Joel Kafka (introduced to Apache incubator on Jul 4, 2011) Kafk

Re: Incubator PMC/Board report for Jul 2012 ([ppmc])

2012-07-02 Thread Joel Koshy
Hi - added this to the wiki. Alan, can you review/sign-off? Thanks, Joel On Mon, Jul 2, 2012 at 5:02 PM, Joel Koshy wrote: > Looks like I don't have write access to the incubator wiki, and I have > requested it. Meanwhile, here is the content I have. Would be helpful if > people

Re: Incubator PMC/Board report for Jul 2012 ([ppmc])

2012-07-03 Thread Joel Koshy
. It > seems obvious to me that this is no longer an issue. I think we should > stress that the project feels that this is no longer an issue with a > specific bullet item or paragraph. > > Otherwise it looks great! Thanks! > > > Regards, > Alan > > > On Jul 2, 2012

Re: nginx module feeding into kafka

2012-07-16 Thread Joel Koshy
Alan, Assuming this is for sending tracking events to a Kafka cluster. You may want to take a look at the C++ client as a reference: https://svn.apache.org/repos/asf/incubator/kafka/trunk/clients/cpp/ Thanks, Joel On Sat, Jul 14, 2012 at 7:58 AM, Alan D. Cabrera wrote: > I'd like to write an n

Re: KAFKA-203 - Improve Metrics (use codehale/metrics)

2012-07-18 Thread Joel Koshy
Not directly related to KAFKA-203, but I started trying out metrics for KAFKA-385. Looks promising - we can see how it plays out. Joel On Tue, Jul 17, 2012 at 7:49 PM, Jonathan Creasy wrote: > What's the current status of this? > > KAFKA-203 (https://issues.apache.org/jira/browse/KAFKA-203) > >

Re: Are we doing a 0.7.2?

2012-07-18 Thread Joel Koshy
0.7.1 patch == patch to apply on branch 0.7.1 or trunk? I thought we would be doing double-commits on trunk and 0.8 and not worry about release branches for now. Also, 0.7.2 would be cut off trunk. (Both 0.7.1 and 0.7 were cut off trunk.) Joel On Wed, Jul 18, 2012 at 10:19 AM, Jay Kreps wrote:

Re: Build failed in Jenkins: Kafka-0.8 #7

2012-07-20 Thread Joel Koshy
> Until we know the cause, looks like deleting the doc on the Producer's > constructor might be ok. Thoughts ? > +1 . This shouldn't hold up jenkins and it does look like a bug.

Re: Replication status

2012-07-20 Thread Joel Koshy
Thanks for the status update. > The following jiras can be worked on in parallel and we welcome > contribution from the community. > > KAFKA-385: RequestPurgatory enhancements > I started on the above, although got side-tracked with some other work. Will get back to it next week. Joel

Re: [DISCUSS] git as the primary data source

2012-07-26 Thread Joel Koshy
+1 for git in general - however, the apache git mirror has been working pretty well for me for local work/code review. So if the caveats you mention are significant enough then maybe we should stay on svn? Joel On Thu, Jul 26, 2012 at 11:31 AM, Taylor Gautier wrote: > I prefer git… > > On Thu, J

replication development dashboard

2012-08-13 Thread Joel Koshy
I'm thinking of putting together an 0.8 development dashboard using the wiki's jiraissues plugin. In order to classify issues, I will need to label issues as follows: core-0.8: core features (this is mostly done) bugs: known bugs tools: e.g., migration tool, producer performance improvements, etc.

Re: replication development dashboard

2012-08-14 Thread Joel Koshy
https://cwiki.apache.org/confluence/display/KAFKA/Kafka+replication+development On Mon, Aug 13, 2012 at 5:23 PM, Joel Koshy wrote: > I'm thinking of putting together an 0.8 development dashboard using the > wiki's jiraissues plugin. In order to classify issues, I will need to l

Re: PartitionData

2012-09-12 Thread Joel Koshy
That's a good point. I did notice that while working on KAFKA-391. The idea behind it must have been that both producer and fetch requests deal with actual partition data. However, the hw and error code don't make sense on the request side. I can include this change as part of KAFKA-391 if that mak

Re: PartitionData

2012-09-12 Thread Joel Koshy
pen a ticket > > On Wed, Sep 12, 2012 at 12:40 PM, Joel Koshy wrote: > > > That's a good point. I did notice that while working on KAFKA-391. The > idea > > behind it must have been that both producer and fetch requests deal with > > actual partition data. Howe

Re: ??? 0.7.2 before 0.8 ???

2012-09-13 Thread Joel Koshy
> > Since 0.7.1, we have fixed 3-4 issues in trunk. The most critical fix is > probably the one that prevents wrong topic name from corrupting log and ZK > data structure. As for 0.8, I think we can start letting people trying the > code from mid to end of Oct. Depending on the result of that, we c

Coding convention: tuples vs case classes

2012-09-13 Thread Joel Koshy
In KAFKA-391 I tried switching from (topic, partition) tuples to a TopicPartition case class in the producer/fetch request handling code. I think it helped quite a bit in terms of code clarity and access from java-api's. There are still lots of places where we have this: https://gist.github.com/37

Re: whitespace

2012-09-18 Thread Joel Koshy
I'm also obsessive compulsive and prefer B. Joel On Tue, Sep 18, 2012 at 3:01 PM, Prashanth Menon wrote: > I prefer A, but go with C only when iterating over certain collections > (maps/tuples etc) to take advantage of Scala's destructuring capabilities; > in these case the spaced brackets make

Re: Kafka 0.8

2012-09-19 Thread Joel Koshy
The wire format proposal is slightly out-of-date due to recent changes. I will update that - although I think if anyone needs to parse the requests directly the code (not the wiki) should be used as a reference. Thanks, Joel On Wed, Sep 19, 2012 at 11:26 AM, Neha Narkhede wrote: > The design of

Re: [VOTE] Release Kafka 0.7.2-incubating (Candidate 4)

2012-09-28 Thread Joel Koshy
Rat fails on the system_test/common/util.sh as it is missing the license. Joel On Friday, September 28, 2012, Jay Kreps wrote: > +1 > > -Jay > > On Fri, Sep 28, 2012 at 8:28 AM, Jun Rao > > wrote: > > > +1. Checked LICENSE and NOTICE. Quickstart commands work. > > > > Thanks, > > > > Jun > > > >

Re: [DISCUSS] Kafka graduation resolution

2012-09-28 Thread Joel Koshy
Updated the list. Thanks, Joel On Wed, Sep 26, 2012 at 3:50 PM, Alan D. Cabrera wrote: > > On Sep 26, 2012, at 3:18 PM, Jakob Homan wrote: > > > OK, everybody's good to go. I'll add the new three new committers in the > > text to be VOTED on and call the vote soon. Before that, though, there'

Re: [VOTE] Release Kafka 0.7.2-incubating (Candidate 4)

2012-09-28 Thread Joel Koshy
AL./system_test/mirror_maker/config/blacklisttest.consumer.properties > AL./system_test/mirror_maker/config/mirror_producer.properties > AL./system_test/mirror_maker/config/server_source_1_1.properties > AL./system_test/mirror_maker/config/server_source_1_2.properties >

Re: [VOTE] Release Kafka 0.7.2-incubating (Candidate 5)

2012-10-01 Thread Joel Koshy
+1 Thanks, Joel On Mon, Oct 1, 2012 at 9:05 AM, Jun Rao wrote: > +1 on RC5. > > Thanks, > > Jun > > On Fri, Sep 28, 2012 at 7:40 PM, Joe Stein wrote: > > > Hello, > > > > This is the fifth candidate for the third incubator release for Apache > > Kafka, version 0.7.2-incubating. > > > > This r

Re: Incubator PMC/Board report for Oct 2012 ([ppmc])

2012-10-03 Thread Joel Koshy
Apparently, this is due today. I put together a draft at http://wiki.apache.org/incubator/October2012 Alan, can you sign-off if it looks good to you? Thanks, Joel On Tue, Oct 2, 2012 at 12:11 PM, Marvin wrote: > > > Dear podling, > > This email was sent by an automated system on behalf of the

Re: CSV metrics

2012-10-30 Thread Joel Koshy
This will be handled in https://issues.apache.org/jira/browse/KAFKA-585 Thanks, Joel On Tue, Oct 30, 2012 at 9:24 AM, Jay Kreps wrote: > I also get this exception over and over again. It's weird because I can > manually create that file so it isn't a permissions issue. In any case the > defaul

[jira] [Created] (KAFKA-75) Corp replica does not shutdown on IO error

2011-07-26 Thread Joel Koshy (JIRA)
Corp replica does not shutdown on IO error -- Key: KAFKA-75 URL: https://issues.apache.org/jira/browse/KAFKA-75 Project: Kafka Issue Type: Improvement Reporter: Joel Koshy The embedded

[jira] [Commented] (KAFKA-75) Corp replica does not shutdown on IO error

2011-07-26 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-75?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13071264#comment-13071264 ] Joel Koshy commented on KAFKA-75: - If we use a producer to mirror the source cluster,

[jira] [Updated] (KAFKA-75) Kafka mirror does not shutdown on IO error

2011-08-03 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-75?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy updated KAFKA-75: Description: The embedded consumer in the kafka mirror implementation uses the low-level Log api to create

[jira] [Closed] (KAFKA-75) Kafka mirror does not shutdown on IO error

2011-08-03 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-75?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy closed KAFKA-75. --- Resolution: Fixed This issue has been subsumed by the fix for KAFKA-74. > Kafka mirror does not shutdown

[jira] [Updated] (KAFKA-74) Kafka mirror (corp replica): auto-discovery of topics

2011-08-03 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-74?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy updated KAFKA-74: Attachment: svn_diff_1153618_1312409247 Patch for KAFKA-74 and KAFKA-75. It includes an enhanced embedded

[jira] [Created] (KAFKA-83) Options in SyncProducerConfig and AsyncProducerConfig can leak

2011-08-04 Thread Joel Koshy (JIRA)
Components: config Affects Versions: 0.6, 0.7 Reporter: Joel Koshy Priority: Minor There is the high-level producer api, and then there is the sync producer and then there is the async producer. Some config options are shared across these which leads to a small degree of

[jira] [Updated] (KAFKA-83) Options in SyncProducerConfig and AsyncProducerConfig can leak

2011-08-04 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-83?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy updated KAFKA-83: Attachment: kafkaproducerconfig.pdf Config hierarchy. > Options in SyncProducerConfig

[jira] [Updated] (KAFKA-83) Options in SyncProducerConfig and AsyncProducerConfig can leak

2011-08-04 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-83?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy updated KAFKA-83: Attachment: kafka-83.patch Patch for KAFKA-83 > Options in SyncProducerConfig and AsyncProducerConfig

[jira] [Updated] (KAFKA-74) Kafka mirror (corp replica): auto-discovery of topics

2011-08-05 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-74?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy updated KAFKA-74: Attachment: svn_diff_1153618_1312569290 Here's the updated patch. The reason the older patch did not co

[jira] [Updated] (KAFKA-74) Kafka mirror (corp replica): auto-discovery of topics

2011-08-09 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-74?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy updated KAFKA-74: Attachment: patch_1155041_v3 Updated patch: - Based on offline discussion with Jun, a simpler API for the

[jira] [Updated] (KAFKA-74) Kafka mirror (corp replica): auto-discovery of topics

2011-08-09 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-74?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy updated KAFKA-74: Attachment: patch_1155041_v4 Thanks for pointing out the session expiration issue. Here is an updated patch

[jira] [Commented] (KAFKA-93) Change code header to follow standard ASF source header

2011-08-10 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-93?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13082510#comment-13082510 ] Joel Koshy commented on KAFKA-93: - Hi Henry, there are a few more files: grep

[jira] [Commented] (KAFKA-93) Change code header to follow standard ASF source header

2011-08-10 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-93?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13082514#comment-13082514 ] Joel Koshy commented on KAFKA-93: - Also, do test bash scripts (e.g., the system tests)

[jira] [Updated] (KAFKA-74) Kafka mirror (corp replica): auto-discovery of topics

2011-08-10 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-74?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy updated KAFKA-74: Attachment: patch_v5 This was tested on a linux box. The test script used GNU options in the ps command

[jira] [Created] (KAFKA-94) KafkaServer can throw a NullPointerException during startup if zookeeper is down

2011-08-11 Thread Joel Koshy (JIRA)
Issue Type: Bug Components: core Reporter: Joel Koshy Priority: Minor Starting up KafkaServer when zookeeper is down can lead to a NullPointerException as shown below. The LogManager throws a ZkTimeoutException from startup if zookeeper is down. This is

[jira] [Commented] (KAFKA-94) KafkaServer can throw a NullPointerException during startup if zookeeper is down

2011-08-11 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-94?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13083863#comment-13083863 ] Joel Koshy commented on KAFKA-94: - Since the logmanager is also uninitialized, we sh

[jira] [Updated] (KAFKA-94) KafkaServer can throw a NullPointerException during startup if zookeeper is down

2011-08-11 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-94?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy updated KAFKA-94: Attachment: kafka-94.patch.v1 > KafkaServer can throw a NullPointerException during startup if zookeeper

[jira] [Created] (KAFKA-103) Make whitelist/blacklist mirror configs more consistent

2011-08-15 Thread Joel Koshy (JIRA)
: 0.7 Reporter: Joel Koshy Priority: Minor The blacklist config for kafka mirrors is a comma separated list of topics. However, the whitelist config is a comma-separated list of "topics:numthreads" pairs, which allows for a multi-threaded consumer in the mirror.

[jira] [Created] (KAFKA-104) Invalid topics prevent broker start-up

2011-08-15 Thread Joel Koshy (JIRA)
Invalid topics prevent broker start-up -- Key: KAFKA-104 URL: https://issues.apache.org/jira/browse/KAFKA-104 Project: Kafka Issue Type: Bug Reporter: Joel Koshy Priority: Minor

[jira] [Updated] (KAFKA-103) Make whitelist/blacklist mirror configs more consistent

2011-08-15 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy updated KAFKA-103: - Attachment: kafka-103-patch.v1 Hmm.. looks like the mirror topics patch actually ignored the numthreads

[jira] [Updated] (KAFKA-103) Make whitelist/blacklist mirror configs more consistent

2011-08-15 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy updated KAFKA-103: - Attachment: (was: kafka-103-patch.v1) > Make whitelist/blacklist mirror configs more consist

[jira] [Updated] (KAFKA-103) Make whitelist/blacklist mirror configs more consistent

2011-08-15 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy updated KAFKA-103: - Attachment: kafka-103-patch.v1 Sorry about deleting that.. I forgot to update the property in system test

[jira] [Updated] (KAFKA-103) Make whitelist/blacklist mirror configs more consistent

2011-08-18 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy updated KAFKA-103: - Attachment: kafka-103-incremental-v2.patch Note that this is an incremental patch. That is a good point

[jira] [Commented] (KAFKA-103) Make whitelist/blacklist mirror configs more consistent

2011-08-19 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13088032#comment-13088032 ] Joel Koshy commented on KAFKA-103: -- At least per kafka-103-patch.v1 it is not tot

[jira] [Resolved] (KAFKA-94) KafkaServer can throw a NullPointerException during startup if zookeeper is down

2011-08-19 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-94?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy resolved KAFKA-94. - Resolution: Fixed Yes - this has been committed. > KafkaServer can throw a NullPointerException dur

[jira] [Created] (KAFKA-126) Log flush should complete upon broker shutdown

2011-08-31 Thread Joel Koshy (JIRA)
Log flush should complete upon broker shutdown -- Key: KAFKA-126 URL: https://issues.apache.org/jira/browse/KAFKA-126 Project: Kafka Issue Type: Bug Reporter: Joel Koshy Broker

[jira] [Created] (KAFKA-127) Tool to watch consumer offsets and lag

2011-08-31 Thread Joel Koshy (JIRA)
Tool to watch consumer offsets and lag -- Key: KAFKA-127 URL: https://issues.apache.org/jira/browse/KAFKA-127 Project: Kafka Issue Type: New Feature Reporter: Joel Koshy Priority

[jira] [Commented] (KAFKA-128) DumpLogSegments outputs wrong offsets

2011-08-31 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13095077#comment-13095077 ] Joel Koshy commented on KAFKA-128: -- The start offset is printed incorrectly before

[jira] [Commented] (KAFKA-124) Console consumer does not exit if consuming process dies

2011-08-31 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13095101#comment-13095101 ] Joel Koshy commented on KAFKA-124: -- +1 PrintStream javadoc suggests

[jira] [Updated] (KAFKA-127) Tool to watch consumer offsets and lag

2011-09-08 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy updated KAFKA-127: - Attachment: KAFKA-127_v1.patch > Tool to watch consumer offsets and

[jira] [Updated] (KAFKA-127) Tool to watch consumer offsets and lag

2011-09-08 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy updated KAFKA-127: - Status: Patch Available (was: Open) Running this tool spits out some zk/zkclient logging. Not sure if we

[jira] [Commented] (KAFKA-70) Introduce retention setting that depends on space

2011-09-09 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-70?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13101547#comment-13101547 ] Joel Koshy commented on KAFKA-70: - Prashanth, Thanks a lot for the p

[jira] [Updated] (KAFKA-127) Tool to watch consumer offsets and lag

2011-09-09 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy updated KAFKA-127: - Attachment: KAFKA-127_v2.patch After trying this out a bit more I decided to incorporate a small

[jira] [Updated] (KAFKA-127) Tool to watch consumer offsets and lag

2011-09-09 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy updated KAFKA-127: - Resolution: Fixed Status: Resolved (was: Patch Available) > Tool to watch consumer offsets

[jira] [Updated] (KAFKA-126) Log flush should complete upon broker shutdown

2011-09-15 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy updated KAFKA-126: - Attachment: KAFKA-126_v1.patch This is pretty much the simple fix that I suggested in the description

[jira] [Updated] (KAFKA-126) Log flush should complete upon broker shutdown

2011-09-15 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy updated KAFKA-126: - Status: Patch Available (was: Open) > Log flush should complete upon broker shutd

[jira] [Updated] (KAFKA-3) Consumer needs a pluggable decoder

2011-09-20 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy updated KAFKA-3: --- Attachment: KAFKA-3_v1.patch Here is a first stab at plugging in a decoder for the consumer API. The html

[jira] [Updated] (KAFKA-126) Log flush should complete upon broker shutdown

2011-09-20 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy updated KAFKA-126: - Attachment: KAFKA-126_v2.patch That makes sense. Here is the updated patch. > Log flush should compl

[jira] [Updated] (KAFKA-126) Log flush should complete upon broker shutdown

2011-09-20 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy updated KAFKA-126: - Attachment: KAFKA-126_v3.patch Oops - forgot to rebase. > Log flush should complete upon broker shutd

[jira] [Commented] (KAFKA-3) Consumer needs a pluggable decoder

2011-09-20 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13109273#comment-13109273 ] Joel Koshy commented on KAFKA-3: Thanks for the tip - that's an interesting id

[jira] [Updated] (KAFKA-332) Mirroring should use multiple producers; add producer retries to DefaultEventHandler

2012-04-24 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy updated KAFKA-332: - Attachment: KAFKA-332-v3.patch tryToHandle catches all now. > Mirroring should

[jira] [Commented] (KAFKA-337) upgrade ZKClient to allow conditional updates in ZK

2012-05-23 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13281992#comment-13281992 ] Joel Koshy commented on KAFKA-337: -- +1 For reference, here is the zkclient change

[jira] [Assigned] (KAFKA-353) tie producer-side ack with high watermark and progress of replicas

2012-05-30 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy reassigned KAFKA-353: Assignee: Joel Koshy > tie producer-side ack with high watermark and progress of repli

[jira] [Commented] (KAFKA-341) Create a new single host system test to validate all replicas on 0.8 branch

2012-06-06 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13290284#comment-13290284 ] Joel Koshy commented on KAFKA-341: -- I remembered I also had some review comments l

[jira] [Commented] (KAFKA-366) add jmx beans in broker to track # bytes in consumer

2012-06-18 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13396402#comment-13396402 ] Joel Koshy commented on KAFKA-366: -- +1 > add jmx beans in br

[jira] [Commented] (KAFKA-339) using MultiFetch in the follower

2012-06-19 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13397097#comment-13397097 ] Joel Koshy commented on KAFKA-339: -- Thanks for the patch. Some comments: AbstractFet

[jira] [Commented] (KAFKA-339) using MultiFetch in the follower

2012-06-20 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13397783#comment-13397783 ] Joel Koshy commented on KAFKA-339: -- +1 on v2 Minor comments: - AbstractFetcherMan

[jira] [Updated] (KAFKA-313) Add JSON output and looping options to ConsumerOffsetChecker

2012-06-20 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy updated KAFKA-313: - Fix Version/s: 0.8 This tool also needs to be updated for 0.8 for the broker failure and other system

[jira] [Commented] (KAFKA-370) Exception "java.util.NoSuchElementException: None.get" appears inconsistently in Mirror Maker log.

2012-06-22 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399451#comment-13399451 ] Joel Koshy commented on KAFKA-370: -- +1 > Ex

[jira] [Updated] (KAFKA-353) tie producer-side ack with high watermark and progress of replicas

2012-06-22 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy updated KAFKA-353: - Attachment: kafka-353_v1.patch Patch is not too big, but a bit tricky. To help with review, here is an

[jira] [Commented] (KAFKA-373) Broker Failure Test needs to be fixed to work with Mirror Maker in TRUNK

2012-06-22 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399781#comment-13399781 ] Joel Koshy commented on KAFKA-373: -- John, thanks for the patch. Some of these comm

[jira] [Commented] (KAFKA-353) tie producer-side ack with high watermark and progress of replicas

2012-06-25 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13400956#comment-13400956 ] Joel Koshy commented on KAFKA-353: -- Jun/Jay, thanks a lot for the very helpful rev

[jira] [Commented] (KAFKA-375) update the site page for the 0.7.1 release download and related changes

2012-06-26 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13401688#comment-13401688 ] Joel Koshy commented on KAFKA-375: -- Joe, thanks for pulling this together. There are

[jira] [Updated] (KAFKA-375) update the site page for the 0.7.1 release download and related changes

2012-06-26 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy updated KAFKA-375: - Attachment: KAFKA-375.consumerapi.patch > update the site page for the 0.7.1 release download

[jira] [Updated] (KAFKA-375) update the site page for the 0.7.1 release download and related changes

2012-06-26 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy updated KAFKA-375: - Attachment: KAFKA-375.consumerapi.patch.v2 Yes - added a few more comments on topic filters

[jira] [Updated] (KAFKA-353) tie producer-side ack with high watermark and progress of replicas

2012-06-27 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy updated KAFKA-353: - Attachment: kafka-353_v2.patch Here is v2 - changes from v1: - Rebased. - Addressed all the "

[jira] [Updated] (KAFKA-353) tie producer-side ack with high watermark and progress of replicas

2012-06-28 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy updated KAFKA-353: - Attachment: kafka-353_v3.patch Thanks for catch-21 :) Fixed that - (note that requiredAcks cannot be 0 in

[jira] [Updated] (KAFKA-353) tie producer-side ack with high watermark and progress of replicas

2012-06-28 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy updated KAFKA-353: - Attachment: kafka-353_v2_to_v3.patch Here is an incremental patch over v2 that makes the diff clearer

[jira] [Commented] (KAFKA-379) TopicCount.constructTopicCount isn't thread-safe

2012-06-28 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13403629#comment-13403629 ] Joel Koshy commented on KAFKA-379: -- Thanks for reporting this - and the SI refer

[jira] [Commented] (KAFKA-353) tie producer-side ack with high watermark and progress of replicas

2012-06-29 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13404165#comment-13404165 ] Joel Koshy commented on KAFKA-353: -- Added that comment on requiredAcks < 0. T

[jira] [Created] (KAFKA-384) Fix intermittent test failures and remove unnecessary sleeps

2012-06-29 Thread Joel Koshy (JIRA)
Joel Koshy created KAFKA-384: Summary: Fix intermittent test failures and remove unnecessary sleeps Key: KAFKA-384 URL: https://issues.apache.org/jira/browse/KAFKA-384 Project: Kafka Issue Type

[jira] [Created] (KAFKA-385) RequestPurgatory enhancements - expire/checkSatisfy issue; add jmx beans

2012-06-29 Thread Joel Koshy (JIRA)
Joel Koshy created KAFKA-385: Summary: RequestPurgatory enhancements - expire/checkSatisfy issue; add jmx beans Key: KAFKA-385 URL: https://issues.apache.org/jira/browse/KAFKA-385 Project: Kafka

  1   2   3   4   >