Re: Review Request 32650: Patch for KAFKA-2000

2015-04-21 Thread Joel Koshy
On April 20, 2015, 11:18 p.m., Joel Koshy wrote: core/src/main/scala/kafka/server/OffsetManager.scala, line 124 https://reviews.apache.org/r/32650/diff/1/?file=909897#file909897line124 I think there is an issue in relying on the metadata cache mainly due to start-up. E.g., when

Re: [KIP-DISCUSSION] KIP-13 Quotas

2015-04-21 Thread Joel Koshy
. For rejected alternatives, we've rejected per-partition distribution because we choose client based quotas where there is no notion of partitions. I've explained in a bit more detail in that section. Aditya From: Joel Koshy

Re: [KIP-DISCUSSION] KIP-13 Quotas

2015-04-21 Thread Joel Koshy
the design wiki page, it seems to me that each request will at least pay a penalty of checking quota enablement. Thanks. Tong Li OpenStack Kafka Community Development Building 501/B205 liton...@us.ibm.com From: Joel Koshy jjkosh...@gmail.com To: dev@kafka.apache.org Date: 04/21

[jira] [Updated] (KAFKA-1910) Refactor KafkaConsumer

2015-04-20 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy updated KAFKA-1910: -- Resolution: Fixed Status: Resolved (was: Patch Available) The follow-up has been committed

Re: Review Request 32650: Patch for KAFKA-2000

2015-04-20 Thread Joel Koshy
there is no _guarantee_ that you have the most current information in the cache (say, if the controller failed to send an UpdateMetadataRequest to the broker by the time the compactor task runs) - Joel Koshy On March 30, 2015, 9:47 p.m., Sriharsha Chintalapani wrote

[jira] [Updated] (KAFKA-2000) Delete consumer offsets from kafka once the topic is deleted

2015-04-20 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy updated KAFKA-2000: -- Status: In Progress (was: Patch Available) Delete consumer offsets from kafka once the topic

Re: [DISCUSS] KIP-11- Authorization design for kafka security

2015-04-20 Thread Joel Koshy
Hi Parth, Nice work on this KIP. I did another read through and had a few more comments (with edits after I went through the thread). Many of these comments were brought up by others as well, so it appears that the KIP would benefit from an update at this point to incorporate comments from the

Re: Review Request 33049: Patch for KAFKA-2084

2015-04-17 Thread Joel Koshy
on other metrics such as messages/sec or requests/sec core/src/main/scala/kafka/server/KafkaServer.scala https://reviews.apache.org/r/33049/#comment130590 We should probably expose various metric configs - reporters, window size, etc. in the server config. - Joel Koshy On April 15, 2015

[jira] [Commented] (KAFKA-2125) Infinite loop after controlled shutdown succeeds

2015-04-15 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14496685#comment-14496685 ] Joel Koshy commented on KAFKA-2125: --- {noformat} 2015-04-14 05:56:10,951 INFO [Thread-38

[jira] [Commented] (KAFKA-2125) Infinite loop after controlled shutdown succeeds

2015-04-15 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14496865#comment-14496865 ] Joel Koshy commented on KAFKA-2125: --- For some reason, the socket server's processors

[jira] [Commented] (KAFKA-2125) Infinite loop after controlled shutdown succeeds

2015-04-15 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14496951#comment-14496951 ] Joel Koshy commented on KAFKA-2125: --- Ah yes that would explain it. [~sriharsha] we were

Re: [KIP-DISCUSSION] KIP-13 Quotas

2015-04-08 Thread Joel Koshy
. Aditya From: Joel Koshy [jjkosh...@gmail.com] Sent: Friday, April 03, 2015 12:48 PM To: dev@kafka.apache.org Subject: Re: [KIP-DISCUSSION] KIP-13 Quotas Aditya, thanks for the updated KIP

Re: Review Request 31850: Patch for KAFKA-1660

2015-04-08 Thread Joel Koshy
/#comment128673 Similar comment as above. Once all accesses of closed are protected by the lock then we should perhaps remove the volatile qualifier. - Joel Koshy On April 8, 2015, 1:18 a.m., Jiangjie Qin wrote

Re: [VOTE] KIP-13 Quotas

2015-04-08 Thread Joel Koshy
Agreed on that - it is understood from the text of the KIP Clients that send current version (0) of those requests will not receive the quota status flag in the response = you have to send version (1) to get the new repsonse, but it should probably be clarified on the page. I'm +1 as well on the

Re: [KIP-DISCUSSION] KIP-13 Quotas

2015-04-07 Thread Joel Koshy
I'm hoping to avoid. Aditya From: Joel Koshy [jjkosh...@gmail.com] Sent: Friday, April 03, 2015 12:48 PM To: dev@kafka.apache.org Subject: Re: [KIP-DISCUSSION] KIP-13 Quotas Aditya, thanks for the updated KIP

Re: Review Request 32931: Revert the changes in NoOffset error code

2015-04-07 Thread Joel Koshy
- Joel Koshy On April 7, 2015, 6:31 p.m., Guozhang Wang wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32931/ --- (Updated

Re: Review Request 31850: Patch for KAFKA-1660

2015-04-06 Thread Joel Koshy
for calling close with a non-zero timeout in the callback? - Joel Koshy On March 27, 2015, 11:35 p.m., Jiangjie Qin wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31850

Re: Review Request 28769: Patch for KAFKA-1809

2015-04-03 Thread Joel Koshy
On April 3, 2015, 12:41 a.m., Joel Koshy wrote: clients/src/main/java/org/apache/kafka/common/protocol/SecurityProtocol.java, line 26 https://reviews.apache.org/r/28769/diff/22/?file=914062#file914062line26 Can we go with TRACE(Short.MAX_VALUE, TRACE) and start plaintext at 0

Re: [KIP-DISCUSSION] KIP-13 Quotas

2015-04-03 Thread Joel Koshy
level. Then they will need to deal with those metrics differently from the rest of the metrics. It would be better if we can hide this complexity from the users. Thanks, Jun On Thu, Mar 19, 2015 at 10:45 PM, Joel Koshy jjkosh...@gmail.com wrote: Actually thinking

[jira] [Commented] (KAFKA-1809) Refactor brokers to allow listening on multiple ports and IPs

2015-04-03 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14395266#comment-14395266 ] Joel Koshy commented on KAFKA-1809: --- Re: ApiVersion - ok, but I just thought it would

[jira] [Commented] (KAFKA-1546) Automate replica lag tuning

2015-04-03 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14395268#comment-14395268 ] Joel Koshy commented on KAFKA-1546: --- +1 on the latest doc updates. I can check

[jira] [Updated] (KAFKA-1546) Automate replica lag tuning

2015-04-03 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy updated KAFKA-1546: -- Resolution: Fixed Status: Resolved (was: Patch Available) Thanks for the patches. Committed

Re: Review Request 28769: Patch for KAFKA-1809

2015-04-02 Thread Joel Koshy
/testcase_0001/testcase_0001_properties.json https://reviews.apache.org/r/28769/#comment127774 were these changes intentional? - Joel Koshy On April 2, 2015, 10:12 p.m., Gwen Shapira wrote: --- This is an automatically generated e-mail

[jira] [Commented] (KAFKA-1910) Refactor KafkaConsumer

2015-03-31 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=1433#comment-1433 ] Joel Koshy commented on KAFKA-1910: --- BTW, just to be clear, what I would like to discuss

Re: KIP Hangout - March 31, 2015 - TL;DR;

2015-03-31 Thread Joel Koshy
We will be using KM for quota'ing on the new client-id-specific metrics. On Tue, Mar 31, 2015 at 08:44:44PM +, Jiangjie Qin wrote: Thanks a lot for the summary, Gwen! About the Quota, does that mean the first quota implementation will be based on YM? I¹m thinking can we pursue a quota

Re: Review Request 30809: Patch for KAFKA-1888

2015-03-31 Thread Joel Koshy
/30809/#comment127010 may be better to just use mktemp for temporary files system_test/broker_upgrade/bin/test-broker-upgrade.sh https://reviews.apache.org/r/30809/#comment127011 should these (and below) be in basedir? That said I don't see this created anywhere. - Joel Koshy On March

Re: KIP Hangout - March 31, 2015 - TL;DR;

2015-03-31 Thread Joel Koshy
for clarification Joel. Just wondering if we are going to depend on any KM specific features? Asking this because KM metric config has quota in side it. On 3/31/15, 1:53 PM, Joel Koshy jjkosh...@gmail.com wrote: We will be using KM for quota'ing on the new client-id-specific metrics. On Tue

[jira] [Commented] (KAFKA-1809) Refactor brokers to allow listening on multiple ports and IPs

2015-03-30 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14387348#comment-14387348 ] Joel Koshy commented on KAFKA-1809: --- Thanks for the ping - yes I would like to take

[jira] [Commented] (KAFKA-2068) Replace OffsetCommit Request/Response with org.apache.kafka.common.requests equivalent

2015-03-30 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14387360#comment-14387360 ] Joel Koshy commented on KAFKA-2068: --- Yes I think we can merge KAFKA-1841 changes from

[jira] [Resolved] (KAFKA-1634) Improve semantics of timestamp in OffsetCommitRequests and update documentation

2015-03-30 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy resolved KAFKA-1634. --- Resolution: Fixed Yes I think that makes more sense Improve semantics of timestamp

Re: [DISCUSSION] Keep docs updated per jira

2015-03-30 Thread Joel Koshy
Also for the wikis - those should probably correspond to the latest released version right? So for e.g., if we add or modify the protocol on trunk we can add it to the wiki but mark it with some highlight or similar just to make it clear that it is a change on trunk only. Thanks, Joel On Thu,

[jira] [Reopened] (KAFKA-1910) Refactor KafkaConsumer

2015-03-30 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy reopened KAFKA-1910: --- Reopening for the doc update and clarification on the change in error code. Refactor KafkaConsumer

[jira] [Commented] (KAFKA-1910) Refactor KafkaConsumer

2015-03-30 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14387652#comment-14387652 ] Joel Koshy commented on KAFKA-1910: --- [~guozhang] this patch modified some

[jira] [Commented] (KAFKA-1546) Automate replica lag tuning

2015-03-27 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14384242#comment-14384242 ] Joel Koshy commented on KAFKA-1546: --- Patch looks good. This is another ticket for which

Re: [VOTE] KIP-15 Add a close method with timeout to new producer

2015-03-27 Thread Joel Koshy
+1 On Fri, Mar 27, 2015 at 01:34:54PM -0700, Neha Narkhede wrote: +1 Thanks Becket! On Fri, Mar 27, 2015 at 1:19 PM, Jay Kreps jay.kr...@gmail.com wrote: +1 Thanks for your patience Jiangjie! -Jay On Fri, Mar 27, 2015 at 11:48 AM, Jiangjie Qin j...@linkedin.com.invalid

[jira] [Commented] (KAFKA-1546) Automate replica lag tuning

2015-03-27 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14384654#comment-14384654 ] Joel Koshy commented on KAFKA-1546: --- https://svn.apache.org/repos/asf/kafka/site/083

[jira] [Commented] (KAFKA-1546) Automate replica lag tuning

2015-03-27 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14384610#comment-14384610 ] Joel Koshy commented on KAFKA-1546: --- Actually, I'm referring to the documentation html

Re: Metrics package discussion

2015-03-27 Thread Joel Koshy
For the samples: it will be at least double that estimate I think since the long array contains (eight byte) references to the actual longs, each of which also have some object overhead. Re: testing: actually, it looks like YM metrics does allow you to drop in your own clock:

Re: KIP discussion Mar 24 at 11am PST

2015-03-27 Thread Joel Koshy
I think we decided that we should reconvene next Tuesday to collect everyone's thoughts on metrics and discuss other KIPs. Can everyone still do Tuesday or do people feel we need to push that further out by a few days? Thanks, Joel On Tue, Mar 24, 2015 at 12:28:04PM -0700, Jun Rao wrote: Just

[jira] [Commented] (KAFKA-1634) Improve semantics of timestamp in OffsetCommitRequests and update documentation

2015-03-26 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14383030#comment-14383030 ] Joel Koshy commented on KAFKA-1634: --- [~guozhang] we actually need to merge KAFKA-1841

Re: [VOTE] KIP-15 add a close with timeout to new producer

2015-03-26 Thread Joel Koshy
Talked to Jiangjie offline - actually looking at the code, we could just extend java.lang.Error. Alternately we could throw an IllegalArgumentException and though we catch Exception, we could catch that explicitly and rethrow to cause the sender to just exit. On Thu, Mar 26, 2015 at 01:29:41PM

Re: Review Request 27391: Rebase patch and incorporate Joel's comments

2015-03-26 Thread Joel Koshy
is provided we calculate retention from that explicit commit timestamp // - If v2 we use the default expiration timestamp ``` - Joel Koshy On March 26, 2015, 7:28 p.m., Guozhang Wang wrote: --- This is an automatically

Re: [VOTE] KIP-15 add a close with timeout to new producer

2015-03-26 Thread Joel Koshy
it unless they check the logs, which is the same as logging the error. Is there a problem with just logging an error and then blocking for the amount of time requested? -Jay On Thu, Mar 26, 2015 at 2:03 PM, Joel Koshy jjkosh...@gmail.com wrote: Talked to Jiangjie offline - actually

Re: [DISCUSSION] Keep docs updated per jira

2015-03-26 Thread Joel Koshy
+1 It is indeed too easy to forget and realize only much later that a jira needed a doc update. So getting into the habit of asking did you update the docs as part of review will definitely help. On Thu, Mar 26, 2015 at 06:36:43PM -0700, Gwen Shapira wrote: I strongly support the goal of

[jira] [Commented] (KAFKA-2046) Delete topic still doesn't work

2015-03-24 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378842#comment-14378842 ] Joel Koshy commented on KAFKA-2046: --- Hey Clark - Onur and I are looking

[jira] [Updated] (KAFKA-2046) Delete topic still doesn't work

2015-03-24 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy updated KAFKA-2046: -- Assignee: Onur Karaman (was: Sriharsha Chintalapani) Delete topic still doesn't work

Re: [VOTE] KIP-15 add a close method with timeout to KafkaProducer

2015-03-24 Thread Joel Koshy
Actually, since there are already votes on this and the KIP has changed a bit we should cancel this and start a new thread. On Tue, Mar 24, 2015 at 12:19 PM, Jiangjie Qin j...@linkedin.com.invalid wrote: Push up the thread for voting after discussion on the KIP hangout. On 3/19/15, 9:03 PM,

Re: [KIP-DISCUSSION] KIP-13 Quotas

2015-03-19 Thread Joel Koshy
in KAFKA-1930). I agree that this KIP doesn't need to block on the migration of the metrics package. Can you clarify the above? i.e., if we are going to quota on something then we would want to have migrated that metric over right? Or do you mean we don't need to complete the migration of all

Re: [KIP-DISCUSSION] KIP-13 Quotas

2015-03-19 Thread Joel Koshy
Actually thinking again - since these will be a few new metrics at the client id level (bytes in and bytes out to start with) maybe it is fine to have the two type of metrics coexist and we can migrate the existing metrics in parallel. On Thursday, March 19, 2015, Joel Koshy jjkosh...@gmail.com

Re: [KIP-DISCUSSION] KIP-13 Quotas

2015-03-19 Thread Joel Koshy
can be done asynchronously. -Jay On Thu, Mar 19, 2015 at 5:56 PM, Joel Koshy jjkosh...@gmail.com javascript:; wrote: in KAFKA-1930). I agree that this KIP doesn't need to block on the migration of the metrics package. Can you clarify the above? i.e., if we are going

Re: [DISCUSSION] KIP-15 close(timeout) for producer

2015-03-19 Thread Joel Koshy
close can probably just getCurrentThread and check if == senderThread. I'm actually not sure from this thread if there was clear agreement on whether it should change close(timeout)/close() to close(0) or if it should log an error and block up to the timeout. On Thursday, March 19, 2015, Jun Rao

Re: [DISCUSSION] KIP-15 close(timeout) for producer

2015-03-19 Thread Joel Koshy
(1) should work, but as Jun suggested earlier in the thread it is slightly misleading. The (intuitive) post-condition of close is that the producer has shutdown - i.e., its sender thread, closed its metrics, serializer/deserializer, etc. That is not necessarily a post-condition of close(0)

Re: [DISCUSS] KIP-4 - Command line and centralized administrative operations

2015-03-18 Thread Joel Koshy
requests, instead of forwarding them to the controller. This will simplify the implementation on the broker side. Thanks, Jun On Wed, Mar 18, 2015 at 11:58 AM, Joel Koshy jjkosh...@gmail.com wrote: (Thanks Andrii for the summary) For (1) yes we

Re: [DISCUSS] KIP-4 - Command line and centralized administrative operations

2015-03-18 Thread Joel Koshy
/reassignPartitions/preferredLeaderElection requests, instead of forwarding them to the controller. This will simplify the implementation on the broker side. Thanks, Jun On Wed, Mar 18, 2015 at 11:58 AM, Joel Koshy jjkosh...@gmail.com wrote: (Thanks Andrii for the summary) For (1

Re: [DISCUSS] KIP-4 - Command line and centralized administrative operations

2015-03-18 Thread Joel Koshy
. Thanks, Andrii Biletskyi On Tue, Mar 17, 2015 at 5:29 PM, Joel Koshy jjkosh...@gmail.com wrote: I may be missing some context but hopefully this will also be covered today: I thought the earlier proposal where there was an explicit ClusterMetadata request was clearer

Re: [DISCUSS] KIP-4 - Command line and centralized administrative operations

2015-03-17 Thread Joel Koshy
remove the ConsumerMetadata / ClusterMetadata requests. Guozhang On Tue, Mar 3, 2015 at 10:23 AM, Joel Koshy jjkosh...@gmail.com wrote: Thanks for sending that out Joe - I don't think I will be able

Re: [KIP-DISCUSSION] KIP-13 Quotas

2015-03-17 Thread Joel Koshy
Yes I think there are two policy issues but I don't think they are separate/mutually exclusive for the purposes of this discussion - the first being what should the broker do if quota is violated and second, what should it return to the client (error code or status code). (The separate discussion

Re: Review Request 31967: Patch for KAFKA-1546

2015-03-16 Thread Joel Koshy
we rename this to logEndOffsetBeforeRead? Also, can we just do with the Long (offset) instead of the entire LogOffsetMetadata? - Joel Koshy On March 16, 2015, 6:32 p.m., Aditya Auradkar wrote: --- This is an automatically

Re: [DISCUSS] KIP 16 - Replica lag tuning

2015-03-13 Thread Joel Koshy
Thanks for the write-up. Looks good to me. Minor comments: Under proposed changes: The proposal is to calculate replica lag as the amount of time not caught up to the leader. A replica is only in ISR if it is caught up. It may be worth clarifying this on the KIP a bit more - i.e., we currently

Re: [VOTE] KIP-16: Replica Lag Tuning

2015-03-13 Thread Joel Koshy
+1 on the KIP On Fri, Mar 13, 2015 at 04:54:04PM +, Aditya Auradkar wrote: Details in the KIP, Jira and RB. https://cwiki.apache.org/confluence/display/KAFKA/KIP+16+:+Automated+Replica+Lag+Tuning https://issues.apache.org/jira/browse/KAFKA-1546 https://reviews.apache.org/r/31967/

[jira] [Commented] (KAFKA-2019) RoundRobinAssignor clusters by consumer

2015-03-13 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14361424#comment-14361424 ] Joel Koshy commented on KAFKA-2019: --- When the round-robin assignor was done, I think we

[jira] [Commented] (KAFKA-1554) Corrupt index found on clean startup

2015-03-13 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14361449#comment-14361449 ] Joel Koshy commented on KAFKA-1554: --- That would be a work-around, but ideally we should

[jira] [Commented] (KAFKA-1987) Potential race condition in partition creation

2015-03-09 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14353502#comment-14353502 ] Joel Koshy commented on KAFKA-1987: --- I actually think it would be worthwhile to improve

Re: Review Request 31706: Patch for KAFKA-1997

2015-03-09 Thread Joel Koshy
/#comment123088 Failed to send message to ... Also, it is probably better to _not_ log the number of skipped messages since that is an accumulated count and is available via an mbean anyway. Patch needs a rebase. - Joel Koshy On March 6, 2015, 4:15 a.m., Jiangjie Qin wrote

Re: Review Request 31706: Patch for KAFKA-1997

2015-03-09 Thread Joel Koshy
On March 6, 2015, 12:14 a.m., Joel Koshy wrote: core/src/main/scala/kafka/consumer/ZookeeperConsumerConnector.scala, line 729 https://reviews.apache.org/r/31706/diff/4/?file=885022#file885022line729 partitionOwnerships - topicPartitionAssignment Jiangjie Qin wrote: I

Re: Review Request 31706: Patch for KAFKA-1997

2015-03-05 Thread Joel Koshy
://reviews.apache.org/r/31706/#comment122482 rename to maybeFlushAndCommitOffsets - Joel Koshy On March 4, 2015, 11:42 p.m., Jiangjie Qin wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r

Re: [KIP-DISCUSSION] KIP-7 Security - IP Filtering

2015-03-04 Thread Joel Koshy
but I do think it gives an additional (lightweight) control which would be useful. Security is about defense in depth, and this raises the bar a tad. Thanks Jeff On Tue, Mar 3, 2015 at 8:58 PM, Joel Koshy jjkosh...@gmail.com wrote: The proposal itself looks reasonable, but I have

Re: [VOTE] KIP-2 - Refactor brokers to allow listening on multiple ports and IPs

2015-03-04 Thread Joel Koshy
+1 binding On Tue, Mar 03, 2015 at 12:14:33PM -0800, Gwen Shapira wrote: Details are in the wiki: https://cwiki.apache.org/confluence/display/KAFKA/KIP-2+-+Refactor+brokers+to+allow+listening+on+multiple+ports+and+IPs

[jira] [Updated] (KAFKA-1852) OffsetCommitRequest can commit offset on unknown topic

2015-03-03 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy updated KAFKA-1852: -- Resolution: Fixed Status: Resolved (was: Patch Available) committed to trunk

[jira] [Updated] (KAFKA-1986) Producer request failure rate should not include InvalidMessageSizeException and OffsetOutOfRangeException

2015-03-03 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy updated KAFKA-1986: -- Resolution: Fixed Status: Resolved (was: Patch Available) Committed to trunk Producer

[jira] [Commented] (KAFKA-1852) OffsetCommitRequest can commit offset on unknown topic

2015-03-03 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14345612#comment-14345612 ] Joel Koshy commented on KAFKA-1852: --- (As noted in RB we need a separate jira to handle

Re: [DISCUSS] KIP-4 - Command line and centralized administrative operations

2015-03-03 Thread Joel Koshy
Thanks for sending that out Joe - I don't think I will be able to make it today, so if notes can be sent out afterward that would be great. On Mon, Mar 02, 2015 at 09:16:13AM -0800, Gwen Shapira wrote: Thanks for sending this out Joe. Looking forward to chatting with everyone :) On Mon, Mar

Re: Review Request 31306: Patch for KAFKA-1755

2015-03-03 Thread Joel Koshy
., deleted). - Joel --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31306/#review74986 --- On Feb. 26, 2015, 6:54 p.m., Joel Koshy wrote

Re: TL;DR; of the first KIP hangout

2015-03-03 Thread Joel Koshy
Thanks Gwen! On Tue, Mar 03, 2015 at 12:12:24PM -0800, Gwen Shapira wrote: Hi, I put together a (very) short summary of the discussion and decisions: KIPs: We reviewed the list of KIPs posted here: https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals * KIP-2:

[jira] [Assigned] (KAFKA-2001) OffsetCommitTest hang during setup

2015-03-03 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy reassigned KAFKA-2001: - Assignee: Joel Koshy I'll take a quick look at this OffsetCommitTest hang during setup

[jira] [Resolved] (KAFKA-2001) OffsetCommitTest hang during setup

2015-03-03 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy resolved KAFKA-2001. --- Resolution: Fixed This was due to a change I made as part of KAFKA-1755 I added compression.type

Re: Review Request 31306: Patch for KAFKA-1755

2015-03-03 Thread Joel Koshy
://reviews.apache.org/r/31306/#comment122056 fixed - Joel Koshy On Feb. 26, 2015, 6:54 p.m., Joel Koshy wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31306

Re: [KIP-DISCUSSION] KIP-7 Security - IP Filtering

2015-03-03 Thread Joel Koshy
The proposal itself looks reasonable, but I have a couple of questions as you made reference to operators of the system; and network team in your wiki. - Are spoofing attacks a concern even with this in place? If so, it would require some sort of internal ingress filtering which presumably

Re: Review Request 29912: Patch for KAFKA-1852

2015-02-27 Thread Joel Koshy
be a val core/src/main/scala/kafka/server/MetadataCache.scala https://reviews.apache.org/r/29912/#comment121176 Should probably do this inReadLock - Joel Koshy On Feb. 18, 2015, 9:13 p.m., Sriharsha Chintalapani wrote

Re: ConsumerTest

2015-02-27 Thread Joel Koshy
+1 On Fri, Feb 27, 2015 at 01:50:07PM -0800, Neha Narkhede wrote: Wow. That was quick :-) Any committers who would also like to give a +1? On Fri, Feb 27, 2015 at 1:44 PM, Ashish Singh asi...@cloudera.com wrote: +1 On Fri, Feb 27, 2015 at 1:42 PM, Gwen Shapira gshap...@cloudera.com

Re: Review Request 24704: Patch for KAFKA-1499

2015-02-27 Thread Joel Koshy
On Feb. 27, 2015, 10:53 p.m., Joel Koshy wrote: core/src/main/scala/kafka/message/CompressionCodec.scala, line 46 https://reviews.apache.org/r/24704/diff/14/?file=801514#file801514line46 Not sure what you mean - can you elaborate? Jun Rao wrote: Instead of contains

Re: Review Request 29912: Patch for KAFKA-1852

2015-02-27 Thread Joel Koshy
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29912/#review74614 --- Ship it! Ship It! - Joel Koshy On Feb. 27, 2015, 9:50 p.m

Re: Review Request 24704: Patch for KAFKA-1499

2015-02-27 Thread Joel Koshy
://reviews.apache.org/r/24704/#comment121241 Not sure what you mean - can you elaborate? - Joel Koshy On Dec. 26, 2014, 4:09 p.m., Manikumar Reddy O wrote: --- This is an automatically generated e-mail. To reply, visit: https

Re: Review Request 29647: Patch for KAFKA-1697

2015-02-27 Thread Joel Koshy
://reviews.apache.org/r/29647/#comment121245 good point - Joel Koshy On Feb. 13, 2015, 2:57 a.m., Gwen Shapira wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29647

[jira] [Updated] (KAFKA-1755) Improve error handling in log cleaner

2015-02-26 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy updated KAFKA-1755: -- Attachment: KAFKA-1755_2015-02-26_10:54:50.patch Improve error handling in log cleaner

Re: Review Request 31306: Patch for KAFKA-1755

2015-02-26 Thread Joel Koshy
/kafka/message/ByteBufferMessageSetTest.scala 73a26377eb63ab9989698e0491049434f032cba2 Diff: https://reviews.apache.org/r/31306/diff/ Testing --- Thanks, Joel Koshy

[jira] [Commented] (KAFKA-1755) Improve error handling in log cleaner

2015-02-26 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14338907#comment-14338907 ] Joel Koshy commented on KAFKA-1755: --- Updated reviewboard https://reviews.apache.org/r

Re: Review Request 31306: Patch for KAFKA-1755

2015-02-26 Thread Joel Koshy
slightly more complicated. - Joel Koshy On Feb. 26, 2015, 6:54 p.m., Joel Koshy wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31306

Re: Review Request 31306: Patch for KAFKA-1755

2015-02-26 Thread Joel Koshy
. To reply, visit: https://reviews.apache.org/r/31306/#review74201 --- On Feb. 23, 2015, 10:29 p.m., Joel Koshy wrote: --- This is an automatically generated e-mail. To reply, visit: https

[jira] [Commented] (KAFKA-1987) Potential race condition in partition creation

2015-02-26 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14339225#comment-14339225 ] Joel Koshy commented on KAFKA-1987: --- Looking at the code I think this is possible when

Re: [VOTE] KIP-3 Mirror Maker Enhancement

2015-02-26 Thread Joel Koshy
+1 (binding) On Tue, Feb 24, 2015 at 06:37:53PM -0800, Guozhang Wang wrote: +1 binding On Tue, Feb 24, 2015 at 5:24 PM, Gwen Shapira gshap...@cloudera.com wrote: +1 (non-binding) On Tue, Feb 24, 2015 at 5:14 PM, Jiangjie Qin j...@linkedin.com.invalid wrote:

Re: Review Request 31449: Patch for KAFKA-1986

2015-02-26 Thread Joel Koshy
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31449/#review74380 --- Ship it! Ship It! - Joel Koshy On Feb. 25, 2015, 11:54 p.m

[jira] [Commented] (KAFKA-1755) Improve error handling in log cleaner

2015-02-23 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=1460#comment-1460 ] Joel Koshy commented on KAFKA-1755: --- Created reviewboard https://reviews.apache.org/r

[jira] [Updated] (KAFKA-1755) Improve error handling in log cleaner

2015-02-23 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy updated KAFKA-1755: -- Status: Patch Available (was: Open) Improve error handling in log cleaner

[jira] [Updated] (KAFKA-1755) Improve error handling in log cleaner

2015-02-23 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy updated KAFKA-1755: -- Attachment: KAFKA-1755.patch Improve error handling in log cleaner

Review Request 31306: Patch for KAFKA-1755

2015-02-23 Thread Joel Koshy
core/src/test/scala/unit/kafka/message/ByteBufferMessageSetTest.scala 73a26377eb63ab9989698e0491049434f032cba2 Diff: https://reviews.apache.org/r/31306/diff/ Testing --- Thanks, Joel Koshy

[jira] [Commented] (KAFKA-1755) Improve error handling in log cleaner

2015-02-23 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=1487#comment-1487 ] Joel Koshy commented on KAFKA-1755: --- I thought a bit more about this and here is a patch

[jira] [Commented] (KAFKA-1755) Improve error handling in log cleaner

2015-02-23 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=1489#comment-1489 ] Joel Koshy commented on KAFKA-1755: --- Also, I have an incremental patch that prevents

Re: Review Request 31306: Patch for KAFKA-1755

2015-02-23 Thread Joel Koshy
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31306/#review73584 --- On Feb. 23, 2015, 2:43 p.m., Joel Koshy wrote

Re: Review Request 31306: Patch for KAFKA-1755

2015-02-23 Thread Joel Koshy
://reviews.apache.org/r/31306/#review73618 --- On Feb. 23, 2015, 2:43 p.m., Joel Koshy wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31306

<    1   2   3   4   5   6   7   8   9   10   >