Re: [ANNOUNCE] New committer: Jason Gustafson

2016-09-09 Thread Aditya Auradkar
Congrats Jason. On Fri, Sep 9, 2016 at 5:30 AM, Michael Noll wrote: > My compliments, Jason -- well deserved! :-) > > -Michael > > > > On Wed, Sep 7, 2016 at 6:49 PM, Grant Henke wrote: > > > Congratulations and thank you for all of your contributions to Apache > > Kafka Jason! > > > > On Wed,

Re: [DISCUSS] KIP-55: Secure quotas for authenticated users

2016-05-25 Thread Aditya Auradkar
Hey Rajini - If the quota.type is set to 'user', what happens to unauthenticated clients? They don't supply a principal, so are they essentially unthrottled? This may be a nit, but I prefer 'quota.type' options to be 'authenticated-user' and 'client-id' as opposed to 'client' and 'user'. For a ne

[jira] [Commented] (KAFKA-3492) support quota based on authenticated user name

2016-04-08 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15232520#comment-15232520 ] Aditya Auradkar commented on KAFKA-3492: [~rsivaram] - Feel free to take

[jira] [Commented] (KAFKA-3492) support quota based on authenticated user name

2016-04-01 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15222349#comment-15222349 ] Aditya Auradkar commented on KAFKA-3492: Very cool. Jun - are you plannin

[jira] [Commented] (KAFKA-3456) In-house KafkaMetric misreports metrics when periodically observed

2016-03-25 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15212514#comment-15212514 ] Aditya Auradkar commented on KAFKA-3456: Accidentally changed the thread titl

[jira] [Updated] (KAFKA-3456) bihtfbucbcceinvujekclljidcuf

2016-03-25 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aditya Auradkar updated KAFKA-3456: --- Summary: bihtfbucbcceinvujekclljidcuf (was: In-house KafkaMetric misreports metrics when

[jira] [Updated] (KAFKA-3456) In-house KafkaMetric misreports metrics when periodically observed

2016-03-25 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aditya Auradkar updated KAFKA-3456: --- Summary: In-house KafkaMetric misreports metrics when periodically observed (was

[GitHub] kafka pull request: KAFKA-2309; ISR shrink rate not updated on Lea...

2016-03-23 Thread auradkar
GitHub user auradkar reopened a pull request: https://github.com/apache/kafka/pull/185 KAFKA-2309; ISR shrink rate not updated on LeaderAndIsr request with shrunk ISR Currently, a LeaderAndIsrRequest does not mark the isrShrinkRate if the received ISR is smaller than the existing

[GitHub] kafka pull request: KAFKA-2309; ISR shrink rate not updated on Lea...

2016-03-23 Thread auradkar
Github user auradkar closed the pull request at: https://github.com/apache/kafka/pull/185 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] kafka pull request: KAFKA-3427 - Broker should return correct vers...

2016-03-23 Thread auradkar
GitHub user auradkar opened a pull request: https://github.com/apache/kafka/pull/1128 KAFKA-3427 - Broker should return correct version of FetchResponse on exception Merging the fix from: https://issues.apache.org/jira/browse/KAFKA-3427 The original version of the code

[jira] [Commented] (KAFKA-3427) broker can return incorrect version of fetch response when the broker hits an unknown exception

2016-03-20 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15203370#comment-15203370 ] Aditya Auradkar commented on KAFKA-3427: [~junrao] - Certainly, I can patch

[jira] [Commented] (KAFKA-3310) fetch requests can trigger repeated NPE when quota is enabled

2016-03-01 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15174366#comment-15174366 ] Aditya Auradkar commented on KAFKA-3310: [~junrao] - can you take a

[GitHub] kafka pull request: KAFKA-3310: Fix for NPEs observed when throttl...

2016-03-01 Thread auradkar
GitHub user auradkar opened a pull request: https://github.com/apache/kafka/pull/989 KAFKA-3310: Fix for NPEs observed when throttling clients. The fix basically ensures that the throttleTimeSensor is non-null before handing off to record the metric value. We also record the

[jira] [Commented] (KAFKA-1215) Rack-Aware replica assignment option

2016-03-01 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15174090#comment-15174090 ] Aditya Auradkar commented on KAFKA-1215: Thanks Allan. I'll review it

[jira] [Commented] (KAFKA-3310) fetch requests can trigger repeated NPE when quota is enabled

2016-02-29 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15173223#comment-15173223 ] Aditya Auradkar commented on KAFKA-3310: [~junrao] - Just making sure,

[jira] [Commented] (KAFKA-3310) fetch requests can trigger repeated NPE when quota is enabled

2016-02-29 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15173204#comment-15173204 ] Aditya Auradkar commented on KAFKA-3310: [~junrao] - Let me investigate this

[jira] [Commented] (KAFKA-1215) Rack-Aware replica assignment option

2016-02-23 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15159491#comment-15159491 ] Aditya Auradkar commented on KAFKA-1215: [~allenxwang] - Is this patch ready

Re: [DISCUSS] KIP-46: Self Healing

2016-02-16 Thread Aditya Auradkar
the capability to move large > amounts of data to the controller, after we are very sure that the default > policy is well tested and the replica throttling works. If so, then it is > just a matter of placing the trigger in the controller vs in the tool. But > I'm skeptical

Re: [DISCUSS] KIP-46: Self Healing

2016-02-09 Thread Aditya Auradkar
ment in a running > cluster. Once we stabilize these changes and feel confident that they work, > we can push the policy into the controller and have it automatically be > triggered based on different events. > > Thanks, > Neha > > On Tue, Feb 2, 2016 at 6:13 PM, Aditya Aurad

[DISCUSS] KIP-46: Self Healing

2016-02-02 Thread Aditya Auradkar
Hey everyone, I just created a kip to discuss automated replica reassignment when we lose a broker in the cluster. https://cwiki.apache.org/confluence/display/KAFKA/KIP-46%3A+Self+Healing+Kafka Any feedback is welcome. Thanks, Aditya

[jira] [Commented] (KAFKA-3088) 0.9.0.0 broker crash on receipt of produce request with empty client ID

2016-01-21 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15111924#comment-15111924 ] Aditya Auradkar commented on KAFKA-3088: I personally prefer preserving the

Re: [VOTE] KIP-36 - Rack aware replica assignment

2016-01-19 Thread Aditya Auradkar
+1 (non-binding) On Tue, Jan 19, 2016 at 8:28 AM, Grant Henke wrote: > +1 (non-binding) > > On Tue, Jan 19, 2016 at 7:46 AM, Ismael Juma wrote: > > > +1 (non-binding) > > > > On Tue, Jan 19, 2016 at 6:12 AM, Guozhang Wang > wrote: > > > > > +1 (binding) > > > > > > On Mon, Jan 18, 2016 at 2:43

Re: [VOTE] KIP-32 Add CreateTime and LogAppendTime to Kafka message.

2016-01-08 Thread Aditya Auradkar
y make sure the version of Record(Message) and > > > > > > MemoryRecords(MessageSet) matches. > > > > > > > > > > > > Currently new clients uses classes in o.a.k.common.record, and > the > > > > broker > > > > > > and old clients uses c

Re: [VOTE] KIP-32 Add CreateTime and LogAppendTime to Kafka message.

2016-01-05 Thread Aditya Auradkar
onfigurable now. > > > > 61. In this design, it's possible for a log segment to have an empty time > > index. In the worse case, we may have to scan more than the active > segment > > to recover the latest timestamp. > > > > Thanks, > > > > J

Re: [DISCUSS] KIP-36 - Rack aware replica assignment

2016-01-05 Thread Aditya Auradkar
gt; >> > wrote: > >> > >> > >> > >>> Few suggestions on improving the KIP > >> > >>> > >> > >>> *If some brokers have rack, and some do not, the algorithm will > >> thrown > >> > an > >

Re: [VOTE] KIP-32 Add CreateTime and LogAppendTime to Kafka message.

2016-01-04 Thread Aditya Auradkar
Hey Becket/Anna - I have a few comments about the KIP. 1. (Minor) Can we rename the KIP? It's currently "Add CreateTime and LogAppendTime etc..". This is actually the title of the now rejected Option 1. 2. (Minor) Can we rename the proposed option? It isn't really "option 4" anymore. 3. I'm not c

[jira] [Commented] (KAFKA-2966) 0.9.0 docs missing upgrade notes regarding replica lag

2015-12-09 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15049357#comment-15049357 ] Aditya Auradkar commented on KAFKA-2966: I'll work on it since I m

[jira] [Assigned] (KAFKA-2966) 0.9.0 docs missing upgrade notes regarding replica lag

2015-12-09 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aditya Auradkar reassigned KAFKA-2966: -- Assignee: Aditya Auradkar > 0.9.0 docs missing upgrade notes regarding replica

Re: [ANNOUNCE] New Kafka Committer Ewen Cheslack-Postava

2015-12-08 Thread Aditya Auradkar
Congrats Ewen! On Tue, Dec 8, 2015 at 11:51 AM, Guozhang Wang wrote: > Congrats Ewen! Welcome onboard. > > Guozhang > > On Tue, Dec 8, 2015 at 11:42 AM, Liquan Pei wrote: > > > Congrats, Ewen! > > > > On Tue, Dec 8, 2015 at 11:37 AM, Neha Narkhede > wrote: > > > > > I am pleased to announce th

[jira] [Commented] (KAFKA-2310) Add config to prevent broker becoming controller

2015-12-02 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15036359#comment-15036359 ] Aditya Auradkar commented on KAFKA-2310: [~abiletskyi] - I see you submitt

Re: [DISCUSS] KIP-36 - Rack aware replica assignment

2015-11-02 Thread Aditya Auradkar
tools are TopicCommand and > ReassignPartitionsCommand. > > Thanks, > Allen > > > > > > On Mon, Oct 26, 2015 at 12:55 PM, Aditya Auradkar > wrote: > > > Hi Allen, > > > > For TopicMetadataResponse to understand version, you can bump up the > &

[jira] [Commented] (KAFKA-2502) Quotas documentation for 0.8.3

2015-10-29 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14980712#comment-14980712 ] Aditya Auradkar commented on KAFKA-2502: [~gwenshap] - Published a patch. Pl

[GitHub] kafka pull request: KAFKA-2502 - Documentation for quotas

2015-10-29 Thread auradkar
GitHub user auradkar opened a pull request: https://github.com/apache/kafka/pull/381 KAFKA-2502 - Documentation for quotas Followed the approach specified here: https://issues.apache.org/jira/browse/KAFKA-2502 I also made a minor fix to ConfigCommand to expose the right options

[jira] [Commented] (KAFKA-2502) Quotas documentation for 0.8.3

2015-10-28 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14979155#comment-14979155 ] Aditya Auradkar commented on KAFKA-2502: [~gwenshap][~ijuma] - Sorry for

[jira] [Created] (KAFKA-2699) Add test to validate times in RequestMetrics

2015-10-27 Thread Aditya Auradkar (JIRA)
Aditya Auradkar created KAFKA-2699: -- Summary: Add test to validate times in RequestMetrics Key: KAFKA-2699 URL: https://issues.apache.org/jira/browse/KAFKA-2699 Project: Kafka Issue Type

[GitHub] kafka pull request: KAFKA-2663, KAFKA-2664 - [Minor] Bugfixes

2015-10-27 Thread auradkar
GitHub user auradkar opened a pull request: https://github.com/apache/kafka/pull/369 KAFKA-2663, KAFKA-2664 - [Minor] Bugfixes This has 2 fixes: KAFKA-2664 - This patch changes the underlying map implementation of Metrics.java to a ConcurrentHashMap. Using a CopyOnWriteMap

Re: [DISCUSS] KIP-36 - Rack aware replica assignment

2015-10-26 Thread Aditya Auradkar
> We allow dynamic topic creation with incomplete broker-rack mapping and > fail fast in command line. Another option is to let user determine the > behavior for command line. For example, by default fail fast in command > line but allow incomplete broker-rack mapping if

Re: [VOTE] KIP-38: ZooKeeper authentication

2015-10-21 Thread Aditya Auradkar
+1 (non-binding). On Wed, Oct 21, 2015 at 3:57 PM, Ismael Juma wrote: > +1 (non-binding) > > On Wed, Oct 21, 2015 at 4:17 PM, Flavio Junqueira wrote: > > > Thanks everyone for the feedback so far. At this point, I'd like to start > > a vote for KIP-38. > > > > Summary: Add support for ZooKeeper

Re: [DISCUSS] KIP-39 Pinning controller to a broker

2015-10-20 Thread Aditya Auradkar
Hi Abhishek - Perhaps it would help if you explained the motivation behind your proposal. I know there was a bunch of discussion on KAFKA-1778, can you summarize? Currently, I'd agree with Neha and Jay that there isn't really a strong reason to pin the controller to a given broker or restricted to

[jira] [Assigned] (KAFKA-2664) Adding a new metric with several pre-existing metrics is very expensive

2015-10-20 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aditya Auradkar reassigned KAFKA-2664: -- Assignee: Aditya Auradkar (was: Onur Karaman) > Adding a new metric with several

Re: Slow request log in Kafka

2015-10-20 Thread Aditya Auradkar
pture detailed timing of a random sample of the requests and log it > > (i.e sample metrics rather than avgs). Note that clients that send more > > requests and longer requests are more likely to get sampled. I've found > > this super useful in the past. > > > > Gw

Re: [DISCUSS] KIP-36 - Rack aware replica assignment

2015-10-20 Thread Aditya Auradkar
t; >> > > > > >> > > > On Thu, Oct 8, 2015 at 9:23 AM, Allen Wang > >> > wrote: > >> > > > > >> > > > > I attended Tuesday's KIP hangout but this KIP was not discussed > >> due > >> > to &

[jira] [Commented] (KAFKA-2502) Quotas documentation for 0.8.3

2015-10-19 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14963909#comment-14963909 ] Aditya Auradkar commented on KAFKA-2502: Thanks Gwen and Ismael. I'll ha

[jira] [Commented] (KAFKA-2502) Quotas documentation for 0.8.3

2015-10-19 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14963605#comment-14963605 ] Aditya Auradkar commented on KAFKA-2502: [~ijuma] - I assume I need to su

[jira] [Commented] (KAFKA-2419) Allow certain Sensors to be garbage collected after inactivity

2015-10-15 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14960132#comment-14960132 ] Aditya Auradkar commented on KAFKA-2419: [~junrao] - Submitted a pull req

[jira] [Reopened] (KAFKA-2419) Allow certain Sensors to be garbage collected after inactivity

2015-10-15 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aditya Auradkar reopened KAFKA-2419: > Allow certain Sensors to be garbage collected after inactiv

[GitHub] kafka pull request: KAFKA-2419 - Fix to prevent background thread ...

2015-10-15 Thread auradkar
GitHub user auradkar opened a pull request: https://github.com/apache/kafka/pull/323 KAFKA-2419 - Fix to prevent background thread from getting created when not required See here for more discussion: https://issues.apache.org/jira/browse/KAFKA-2419 Basically, the fix involves

[jira] [Commented] (KAFKA-2419) Allow certain Sensors to be garbage collected after inactivity

2015-10-15 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14959588#comment-14959588 ] Aditya Auradkar commented on KAFKA-2419: I agree it would be nice to not ne

[jira] [Commented] (KAFKA-2419) Allow certain Sensors to be garbage collected after inactivity

2015-10-15 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14959484#comment-14959484 ] Aditya Auradkar commented on KAFKA-2419: [~junrao] - That is indeed the cur

[jira] [Commented] (KAFKA-2419) Allow certain Sensors to be garbage collected after inactivity

2015-10-15 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14959409#comment-14959409 ] Aditya Auradkar commented on KAFKA-2419: [~junrao] - I concur that option

[jira] [Comment Edited] (KAFKA-2419) Allow certain Sensors to be garbage collected after inactivity

2015-10-15 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14959409#comment-14959409 ] Aditya Auradkar edited comment on KAFKA-2419 at 10/15/15 6:5

[jira] [Commented] (KAFKA-2419) Allow certain Sensors to be garbage collected after inactivity

2015-10-15 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14958962#comment-14958962 ] Aditya Auradkar commented on KAFKA-2419: [~ijuma] - We create only a si

Slow request log in Kafka

2015-10-14 Thread Aditya Auradkar
Hey everyone, We were recently discussing a small logging improvement for Kafka. Basically, add a request log for queries that took longer than a certain configurable time to execute. This can be quite useful for debugging purposes, in fact it would have proven handy while investigating a recent i

[jira] [Commented] (KAFKA-2527) System Test for Quotas in Ducktape

2015-10-13 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14955720#comment-14955720 ] Aditya Auradkar commented on KAFKA-2527: [~gwenshap] - Thanks! > System T

[jira] [Commented] (KAFKA-2536) topics tool should allow users to alter topic configuration

2015-10-13 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14955375#comment-14955375 ] Aditya Auradkar commented on KAFKA-2536: [~gwenshap] - Thanks for reporting

[GitHub] kafka pull request: KAFKA-2209 - Change quotas dynamically using D...

2015-10-12 Thread auradkar
GitHub user auradkar opened a pull request: https://github.com/apache/kafka/pull/298 KAFKA-2209 - Change quotas dynamically using DynamicConfigManager Changes in this patch are: 1. ClientIdConfigHandler now passes through the config changes to the quota manager. 2. Removed

[jira] [Resolved] (KAFKA-2606) Remove kafka.utils.Time in favour of o.a.kafka.common.utils.Time

2015-10-02 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aditya Auradkar resolved KAFKA-2606. Resolution: Duplicate Duplicate of: https://issues.apache.org/jira/browse/KAFKA-2247

[jira] [Updated] (KAFKA-2247) Merge kafka.utils.Time and kafka.common.utils.Time

2015-10-02 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aditya Auradkar updated KAFKA-2247: --- Description: We currently have 2 different versions of Time in clients and core. These need

[jira] [Commented] (KAFKA-2606) Remove kafka.utils.Time in favour of o.a.kafka.common.utils.Time

2015-10-02 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14941984#comment-14941984 ] Aditya Auradkar commented on KAFKA-2606: [~ijuma] This is a duplicate of h

Re: [DISCUSS] KIP-36 - Rack aware replica assignment

2015-09-30 Thread Aditya Auradkar
ion about the physical location > > of servers. I don't relish the idea of having to maintain data in > multiple > > places. > > > > -Todd > > > > On Mon, Sep 28, 2015 at 4:48 PM, Aditya Auradkar < > > aaurad...@linkedin.com.invalid> wrote:

Re: [DISCUSS] KIP-36 - Rack aware replica assignment

2015-09-28 Thread Aditya Auradkar
Thanks for starting this KIP Allen. I agree with Gwen that having a RackLocator class that is pluggable seems to be too complex. The KIP refers to potentially non-ZK storage for the rack info which I don't think is necessary. Perhaps we can persist this info in zk under /brokers/ids/ similar to o

[jira] [Commented] (KAFKA-1215) Rack-Aware replica assignment option

2015-09-25 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14908460#comment-14908460 ] Aditya Auradkar commented on KAFKA-1215: [~allenxwang] - bump. > Rac

Re: Release plan for kafka security

2015-09-25 Thread Aditya Auradkar
Basically, 0.8.3 has been renamed to 0.9.0. The plan is to include security in the 0.9 release which should happen once all the blocker bugs have been resolved and testing is complete (committers can provide more accurate timelines). On Fri, Sep 25, 2015 at 10:35 AM, Whitney, Adam wrote: > Hello

Re: [VOTE] KIP-31 - Move to relative offsets in compressed message sets.

2015-09-23 Thread Aditya Auradkar
+1 On Wed, Sep 23, 2015 at 8:03 PM, Neha Narkhede wrote: > +1 > > On Wed, Sep 23, 2015 at 6:21 PM, Todd Palino wrote: > > > +1000 > > > > ! > > > > -Todd > > > > On Wednesday, September 23, 2015, Jiangjie Qin > > > wrote: > > > > > Hi, > > > > > > Thanks a lot for the reviews and feedback on K

[GitHub] kafka pull request: KAFKA-2419; Garbage collect unused sensors

2015-09-22 Thread auradkar
GitHub user auradkar opened a pull request: https://github.com/apache/kafka/pull/233 KAFKA-2419; Garbage collect unused sensors As discussed in KAFKA-2419 - I've added a time based sensor retention config to Sensor. Sensors that have not been "recorded" for

[jira] [Commented] (KAFKA-2567) throttle-time shouldn't be NaN

2015-09-22 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14903101#comment-14903101 ] Aditya Auradkar commented on KAFKA-2567: [~junrao] - I'l fix this i

[jira] [Assigned] (KAFKA-2567) throttle-time shouldn't be NaN

2015-09-22 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aditya Auradkar reassigned KAFKA-2567: -- Assignee: Aditya Auradkar > throttle-time shouldn'

[jira] [Updated] (KAFKA-2567) throttle-time shouldn't be NaN

2015-09-22 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aditya Auradkar updated KAFKA-2567: --- Labels: quotas (was: ) > throttle-time shouldn'

Re: [ANNOUNCE] New Committer Sriharsha Chintalapani

2015-09-21 Thread Aditya Auradkar
Congrats Sriharsha! Great work.. especially on SSL. On Mon, Sep 21, 2015 at 10:31 PM, Prabhjot Bharaj wrote: > Congratulations. It's inspiring for newbies like me > > Regards, > Prabhjot > On Sep 22, 2015 10:30 AM, "Ashish Singh" wrote: > > > Congrats Harsha! > > > > On Monday, September 21, 20

[jira] [Commented] (KAFKA-1599) Change preferred replica election admin command to handle large clusters

2015-09-21 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14901142#comment-14901142 ] Aditya Auradkar commented on KAFKA-1599: [~anigam] - Perhaps you can writ

[jira] [Commented] (KAFKA-1215) Rack-Aware replica assignment option

2015-09-15 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14745803#comment-14745803 ] Aditya Auradkar commented on KAFKA-1215: [~allenxwang] One of the committers

[GitHub] kafka pull request: KAFKA-2443 Expose windowSize on Rate

2015-09-14 Thread auradkar
GitHub user auradkar opened a pull request: https://github.com/apache/kafka/pull/213 KAFKA-2443 Expose windowSize on Rate This is a followup ticket from KAFKA-2084 to improve the windowSize calculation in Quotas. I've made the following changes: 1. Added a windo

[jira] [Updated] (KAFKA-2443) Expose windowSize on Rate

2015-09-14 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aditya Auradkar updated KAFKA-2443: --- Summary: Expose windowSize on Rate (was: Expose windowSize on Measurable) > Exp

[jira] [Updated] (KAFKA-2443) Expose windowSize on Rate

2015-09-14 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aditya Auradkar updated KAFKA-2443: --- Description: Currently, we dont have a means to measure the size of the metric window since

[jira] [Commented] (KAFKA-2528) Quota Performance Evaluation

2015-09-10 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14739899#comment-14739899 ] Aditya Auradkar commented on KAFKA-2528: One possible explanation for

[jira] [Commented] (KAFKA-2528) Quota Performance Evaluation

2015-09-10 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14739438#comment-14739438 ] Aditya Auradkar commented on KAFKA-2528: I'm not quite sure why the ac

[jira] [Commented] (KAFKA-2419) Allow certain Sensors to be garbage collected after inactivity

2015-09-10 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14739324#comment-14739324 ] Aditya Auradkar commented on KAFKA-2419: Thanks [~ijuma] I'll take a l

[jira] [Commented] (KAFKA-2500) Make logEndOffset available in the 0.8.3 Consumer

2015-09-10 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14739003#comment-14739003 ] Aditya Auradkar commented on KAFKA-2500: [~hachikuji] - Are you plannin

[jira] [Commented] (KAFKA-1215) Rack-Aware replica assignment option

2015-09-09 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14737918#comment-14737918 ] Aditya Auradkar commented on KAFKA-1215: [~allenxwang] Hi Allen. Thanks for

[jira] [Commented] (KAFKA-2419) Allow certain Sensors to be garbage collected after inactivity

2015-09-09 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14737756#comment-14737756 ] Aditya Auradkar commented on KAFKA-2419: [~junrao][~jjkoshy] We have a coupl

[jira] [Commented] (KAFKA-2419) Allow certain Sensors to be garbage collected after inactivity

2015-09-09 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14737497#comment-14737497 ] Aditya Auradkar commented on KAFKA-2419: [~junrao] I'm working on this n

Re: Maybe 0.8.3 should really be 0.9.0?

2015-09-08 Thread Aditya Auradkar
+1 on 0.9 On Tue, Sep 8, 2015 at 12:29 PM, Edward Ribeiro wrote: > +1 on 0.9.0 > > On Tue, Sep 8, 2015 at 4:07 PM, Ashish Singh wrote: > > > +1 on 0.9.0 > > > > On Tue, Sep 8, 2015 at 12:00 PM, Gwen Shapira wrote: > > > > > I propose a simple rename: s/0.8.3/0.9.0/ > > > > > > No change of sco

Re: Maybe 0.8.3 should really be 0.9.0?

2015-09-08 Thread Aditya Auradkar
Hi Gwen, I certainly think 0.9.0 is better than 0.8.3. As regards semantic versioning, do we have a plan for a 1.0 release? IIUC, compatibility rules don't really apply for pre-1.0 stuff. I'd argue that Kafka already qualifies for 1.x. Aditya On Tue, Sep 8, 2015 at 10:26 AM, Gwen Shapira wrote:

[jira] [Updated] (KAFKA-2209) Change client quotas dynamically using DynamicConfigManager

2015-09-02 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aditya Auradkar updated KAFKA-2209: --- Labels: quotas (was: ) > Change client quotas dynamically using DynamicConfigMana

[jira] [Updated] (KAFKA-2443) Expose windowSize on Measurable

2015-09-02 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aditya Auradkar updated KAFKA-2443: --- Labels: quotas (was: ) > Expose windowSize on Measura

[jira] [Updated] (KAFKA-2502) Quotas documentation for 0.8.3

2015-09-02 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aditya Auradkar updated KAFKA-2502: --- Labels: quotas (was: ) > Quotas documentation for 0.

[jira] [Created] (KAFKA-2502) Quotas documentation for 0.8.3

2015-09-02 Thread Aditya Auradkar (JIRA)
Aditya Auradkar created KAFKA-2502: -- Summary: Quotas documentation for 0.8.3 Key: KAFKA-2502 URL: https://issues.apache.org/jira/browse/KAFKA-2502 Project: Kafka Issue Type: Task

[jira] [Resolved] (KAFKA-2136) Client side protocol changes to return quota delays

2015-09-02 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aditya Auradkar resolved KAFKA-2136. Resolution: Fixed > Client side protocol changes to return quota del

Open source Mysql -> Kafka connector

2015-09-01 Thread Aditya Auradkar
I thought this would be of interest: https://developer.zendesk.com/blog/introducing-maxwell-a-mysql-to-kafka-binlog-processor A copycat connector that parses MySQL binlogs would be rather useful I think. Streaming connectors using jdbc are tricky to implement because they rely on an indexed timest

[GitHub] kafka pull request: Fixing KAFKA-2309

2015-09-01 Thread auradkar
GitHub user auradkar opened a pull request: https://github.com/apache/kafka/pull/185 Fixing KAFKA-2309 Currently, a LeaderAndIsrRequest does not mark the isrShrinkRate if the received ISR is smaller than the existing ISR. This can happen if one of the replicas is shut down. You

Re: [IMPORTANT] Questions related to AdminUtils thread safety

2015-09-01 Thread Aditya Auradkar
Correct. Aditya On Tue, Sep 1, 2015 at 9:47 AM, Jaikiran Pai wrote: > On Tuesday 01 September 2015 09:51 AM, Aditya Auradkar wrote: > >> createTopic should also be fine unless you try to create the same topic >> concurrently. >> > > Just to be clear - that would

Re: no Kafka KIP meeting tomorrow

2015-09-01 Thread Aditya Auradkar
Typically to discuss anything listed here or other important changes: https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals On Tue, Sep 1, 2015 at 1:28 AM, jinxing wrote: > Can I know what is KIP meeting? > > > > > > > At 2015-09-01 12:30:18, "Jun Rao" wrote: > >Since th

Re: [IMPORTANT] Questions related to AdminUtils thread safety

2015-08-31 Thread Aditya Auradkar
topicExists simply reads ZK, so yes. createTopic should also be fine unless you try to create the same topic concurrently. AdminUtils itself does not maintain any state, just some static util functions. On Mon, Aug 31, 2015 at 3:00 PM, Sivananda Reddys Thummala Abbigari < sthumm...@salesforce.com>

[jira] [Updated] (KAFKA-2136) Client side protocol changes to return quota delays

2015-08-25 Thread Aditya A Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aditya A Auradkar updated KAFKA-2136: - Attachment: KAFKA-2136_2015-08-25_11:29:52.patch > Client side protocol changes to ret

Re: Review Request 33378: Patch for KAFKA-2136

2015-08-25 Thread Aditya Auradkar
kafka/server/ThrottledResponseExpirationTest.scala c4b5803917e700965677d53624f1460c1a52bf52 Diff: https://reviews.apache.org/r/33378/diff/ Testing --- New tests added Thanks, Aditya Auradkar

Re: Review Request 33378: Patch for KAFKA-2136

2015-08-25 Thread Aditya Auradkar
it/kafka/server/ThrottledResponseExpirationTest.scala c4b5803917e700965677d53624f1460c1a52bf52 Diff: https://reviews.apache.org/r/33378/diff/ Testing --- New tests added Thanks, Aditya Auradkar

[jira] [Commented] (KAFKA-2136) Client side protocol changes to return quota delays

2015-08-25 Thread Aditya A Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14711745#comment-14711745 ] Aditya A Auradkar commented on KAFKA-2136: -- Updated reviewboard h

Re: Review Request 33378: Patch for KAFKA-2136

2015-08-25 Thread Aditya Auradkar
ly need a versionId as well (as is done in the Scala > > response) - i.e., when we move the broker over to use these protocol > > schemas. > > Aditya Auradkar wrote: > Makes sense. Do you want me to tackle this in this patch or should it be > tackled in the patch that

Re: Review Request 33378: Patch for KAFKA-2136

2015-08-24 Thread Aditya Auradkar
igrates the broker to use these schemas? - Aditya --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33378/#review96100 ---

[jira] [Commented] (KAFKA-2136) Client side protocol changes to return quota delays

2015-08-24 Thread Aditya A Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14709677#comment-14709677 ] Aditya A Auradkar commented on KAFKA-2136: -- Updated reviewboard h

  1   2   3   4   5   6   7   >