point. Could you add that to
> the
> > rejected section of the KIP. +1 on this KIP.
> >
> > Jun
> >
> > On Fri, Mar 30, 2018 at 10:06 AM, Allen Wang
> wrote:
> >
> > > Hi Jun,
> > >
> > > I think this KIP helps to gain insight i
ou can roughly estimate the ratio of
> consumers still needing down conversion. Does that cover the main thing
> that you want from this KIP?
>
> Jun
>
>
>
> On Wed, Mar 28, 2018 at 9:55 AM, Allen Wang wrote:
>
> > Hi All,
> >
> > I would like to start
Hi All,
I would like to start voting for KIP-272: Add API version tag to broker's
RequestsPerSec metric.
https://cwiki.apache.org/confluence/display/KAFKA/KIP-272%3A+Add+API+version+tag+to+broker%27s+RequestsPerSec+metric
Thanks,
Allen
Hello,
I would like to bring this to vote in the next day or two. Let me know if
you have further comments.
Thanks,
Allen
On Thu, Mar 22, 2018 at 11:37 AM, Xavier Léauté wrote:
> >
> > This kind of change will be problematic to us as the total RequestsPerSec
> > will be double counted in our
; >
> > -James
> >
> > Sent from my iPhone
> >
> >> On Mar 21, 2018, at 10:17 PM, Jeff Widman wrote:
> >>
> >> I agree with Allen.
> >>
> >> Go with the intuitive name, even if it means not deprecating. The
> impact of
> >> br
;>
> >> On Tue, Mar 20, 2018 at 2:49 PM, Ted Yu wrote:
> >>
> >>> Thanks for the response.
> >>>
> >>> Assuming number of client versions is limited in a cluster, memory
> >>> consumption is not a concern.
> >>>
> >>>
the metric to locate
> the metric *
>
> *Do you have estimate how much memory is needed for maintaining the hash
> map ?*
>
> *Thanks*
>
> On Mon, Mar 19, 2018 at 3:19 PM, Allen Wang wrote:
>
> > Hi all,
> >
> > I have created KIP-272: Add API version tag t
Hi all,
I have created KIP-272: Add API version tag to broker's RequestsPerSec
metric.
Here is the link to the KIP:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-272%3A+Add+API+version+tag+to+broker%27s+RequestsPerSec+metric
Looking forward to the discussion.
Thanks,
Allen
Hi Dong,
On Tue, Feb 27, 2018 at 10:07 PM, Dong Lin wrote:
> Hey Allen,
>
> Thanks for the comments.
>
> On Mon, Feb 26, 2018 at 9:27 PM, Allen Wang
> wrote:
>
> > Hi Dong,
> >
> > Please see my comments inline.
> >
> > Thanks,
> >
Hi Dong,
Please see my comments inline.
Thanks,
Allen
On Sun, Feb 25, 2018 at 3:33 PM, Dong Lin wrote:
> Hey Allen,
>
> Thanks for your comment. I will comment inline.
>
> On Thu, Feb 22, 2018 at 3:05 PM, Allen Wang
> wrote:
>
> > Overall this is a very usefu
Overall this is a very useful feature. With this we can finally scale keyed
messages.
+1 on the ability to remove partitions. This will greatly increase Kafka's
scalability in cloud.
For example, when there is traffic increase, we can add brokers and assign
new partitions to the new brokers. When
Allen Wang created KAFKA-6514:
-
Summary: Add API version as a tag for the RequestsPerSec metric
Key: KAFKA-6514
URL: https://issues.apache.org/jira/browse/KAFKA-6514
Project: Kafka
Issue Type
Allen Wang created KAFKA-6329:
-
Summary: Load trust store as a resource
Key: KAFKA-6329
URL: https://issues.apache.org/jira/browse/KAFKA-6329
Project: Kafka
Issue Type: Improvement
Allen Wang created KAFKA-5813:
-
Summary: Unexpected unclean leader election due to
leader/controller's unusual event handling order
Key: KAFKA-5813
URL: https://issues.apache.org/jira/browse/KAFKA
Allen Wang created KAFKA-5724:
-
Summary: AbstractPartitionAssignor does not take into
consideration that partition number may start from non-zero
Key: KAFKA-5724
URL: https://issues.apache.org/jira/browse/KAFKA-5724
[
https://issues.apache.org/jira/browse/KAFKA-2273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15935450#comment-15935450
]
Allen Wang commented on KAFKA-2273:
---
Does the current implementation have
[
https://issues.apache.org/jira/browse/KAFKA-2273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15806146#comment-15806146
]
Allen Wang commented on KAFKA-2273:
---
What is the plan for this JIRA/KIP? It is se
[
https://issues.apache.org/jira/browse/KAFKA-1215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15202362#comment-15202362
]
Allen Wang commented on KAFKA-1215:
---
[~junrao] Updated.
> Rack-Aware
[
https://issues.apache.org/jira/browse/KAFKA-1215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15169592#comment-15169592
]
Allen Wang commented on KAFKA-1215:
---
[~aauradkar] Yes it is ready for review.
&g
ApiUtils.writeShortString and ApiUtils.readShortString APIs support null.
However, I would like to know if there is any particular reason not to
allow null for STRING.
Any opinions?
Thanks,
Allen
On Wed, Jan 20, 2016 at 1:50 PM, Allen Wang wrote:
> Hi Arun,
>
> This is about making replica a
Hi Jun,
What about https://issues.apache.org/jira/browse/KAFKA-3100?
Thanks,
Allen
On Fri, Feb 5, 2016 at 1:19 PM, Ismael Juma wrote:
> Hi Becket,
>
> On Fri, Feb 5, 2016 at 9:15 PM, Becket Qin wrote:
>
> > I am taking KAFKA-3177 off the list because the correct fix might involve
> > some re
Cool. Thank you all for comments and participation.
On Mon, Jan 25, 2016 at 9:43 AM, Neha Narkhede wrote:
> Yes. Thanks for driving this.
>
> On Mon, Jan 25, 2016 at 9:42 AM, Allen Wang wrote:
>
> > There are 4 binding +1 and 4 non-binding +1. Shall we consider the
>
wrote:
> > > > >
> > > > > > +1 (non-binding)
> > > > > >
> > > > > > On Mon, Jan 18, 2016 at 2:32 PM, Jun Rao
> wrote:
> > > > > >
> > > > > > > Allen,
> > > > > &
ing to support only rack aware assignments?
>
> May be nice to make the implementation pluggable (with rack aware being
> one) so that other kind of assignment algorithms can be plugged in future.
>
> - Arun
>
>
>
> On 1/15/16, 12:22 AM, "Allen Wang" wrote:
&
Thanks Ismael. KIP is updated to use 0.9.0.0 and add link to the JIRA.
On Thu, Jan 14, 2016 at 8:46 AM, Ismael Juma wrote:
> On Thu, Jan 14, 2016 at 1:24 AM, Allen Wang wrote:
>
> > Updated KIP regarding how broker JSON version will be handled and new
> > procedure of upg
Updated KIP regarding how broker JSON version will be handled and new
procedure of upgrade.
On Wed, Jan 13, 2016 at 10:58 AM, Allen Wang wrote:
> Hi Jun,
>
> I feel it is a bit complicated and unconventional to have a major release
> have dependency on a minor release. It would als
gt; broker to 0.9.1 and beyond. This fix can be done in a separate jira.
>
> Thanks,
>
> Jun
>
> On Tue, Jan 12, 2016 at 5:35 PM, Allen Wang wrote:
>
> > Agreed. So it seems that for 0.9.1, the only option is to keep the JSON
> > version unchanged. But as part o
sue. We overlooked this issue
> in 0.9.0.0. The easiest fix is probably not to check the version in
> ZkUtils.getBrokerInfo().
> This way, as long as we are only adding new fields in broker json, we can
> preserve the compatibility.
>
> Thanks,
>
> Jun
>
> On Tue, Jan
ack info.
>
> If one follows the upgrade process, after the 2nd round of rolling bounces,
> every broker is capable of parsing version 3 of broker info in ZK. This is
> when the rack-aware feature will be used.
>
>
> Thanks,
>
> Jun
>
> On Tue, Jan 12, 2016 at 12:19
ll be updated to return the Broker with
rack so the rack information will be available once the server/client is
upgraded to the latest version.
On Wed, Jan 6, 2016 at 6:28 PM, Allen Wang wrote:
> Updated KIP according to Jun's comment and included changes to TMR.
>
> On Tue, Jan
dateMetadataRequest should be 2, instead of 1.
> Could you include the full wire protocol of version 2 of
> UpdateMetadataRequest and highlight the changed part?
>
> Thanks,
>
> Jun
>
> On Tue, Jan 5, 2016 at 3:11 PM, Allen Wang wrote:
>
> > Jun and I had a chance to
be a separate patch).. it isn't a
> large change.
>
> Aditya
>
> On Sat, Dec 26, 2015 at 3:01 PM, Allen Wang wrote:
>
> > Added the rolling upgrade instruction in the KIP, similar to those in
> 0.9.0
> > release notes.
> >
> > On Wed, Dec 16,
I would like to call for a vote for KIP-36 - Rack aware replica assignment.
The latest proposal is at
https://cwiki.apache.org/confluence/display/KAFKA/KIP-36
+Rack+aware+replica+assignment
Thanks,
Allen
Added the rolling upgrade instruction in the KIP, similar to those in 0.9.0
release notes.
On Wed, Dec 16, 2015 at 11:32 AM, Allen Wang wrote:
> Hi Jun,
>
> The reason that TopicMetadataResponse is not included in the KIP is that
> it currently is not version aware . So we need
e tasks, as well
> as read affinity in the future.
>
> Jun
>
>
>
> On Thu, Dec 10, 2015 at 9:38 AM, Allen Wang wrote:
>
> > If there are no more comments I would like to call for a vote.
> >
> >
> > On Sun, Nov 15, 2015 at 10:08 PM, Allen Wang
>
If there are no more comments I would like to call for a vote.
On Sun, Nov 15, 2015 at 10:08 PM, Allen Wang wrote:
> KIP is updated with more details and how to handle the situation where
> rack information is incomplete.
>
> In the situation where rack information is incomplete,
[
https://issues.apache.org/jira/browse/KAFKA-2308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15034439#comment-15034439
]
Allen Wang commented on KAFKA-2308:
---
[~gwenshap] [~guozhang] Is the fix in prod
replica distribution.
Also for command line tools it gives user a choice if for any reason they
want to ignore rack information and fallback to the original algorithm.
On Tue, Nov 10, 2015 at 9:04 AM, Allen Wang wrote:
> I am busy with some time pressing issues for the last few days. I wil
r seen by the user
> - imbalance on either the leaders or data or both.
>
> On Mon, Nov 2, 2015 at 6:39 PM, Aditya Auradkar
> wrote:
>
> > I think this sounds reasonable. Anyone else have comments?
> >
> > Aditya
> >
> > On Tue, Oct 27, 2015 at 5:23 PM, All
r 1 - I'd be fine with adding an option to the command line tools that
> check rack assignment. For e.g. "--strict-assignment" or something similar.
>
> Aditya
>
> On Thu, Oct 22, 2015 at 6:44 PM, Allen Wang wrote:
>
> > For 2 and 3, I have updated
tions#KIP-4-Commandlineandcentralizedadministrativeoperations-CreateTopicRequest
>
> Aditya
>
> On Fri, Oct 16, 2015 at 3:45 PM, Allen Wang wrote:
>
> > KIP is updated include rack as an optional property for broker. Please
> take
> > a look and let me know if more detail
thread or next hangout.
On Thu, Oct 15, 2015 at 10:42 AM, Allen Wang wrote:
> That's a good question. I can think of three actions if the rack
> information is incomplete:
>
> 1. Treat the node without rack as if it is on its unique rack
> 2. Disregard all rack information and
u think?
>
> Gwen
>
> On Thu, Oct 15, 2015 at 10:13 AM, Allen Wang wrote:
>
> > For scenario 1:
> >
> > - Add the rack information to broker property file or dynamically set it
> in
> > the wrapper code to bootstrap Kafka server. You would do that for all
the workflow for the following scenarios:
>
> 1. I currently have 6 brokers and want to add rack information for each
> 2. I'm adding a new broker and I want to specify which rack it belongs on
> while adding it.
>
> Thanks!
>
> On Tue, Oct 13, 2015 at 2:21 PM, Allen Wan
the meta data to help the consumer choose which in sync replica to
consume from as part of the future consumer enhancement.
I will update the KIP.
Thanks,
Allen
On Thu, Oct 8, 2015 at 9:23 AM, Allen Wang wrote:
> I attended Tuesday's KIP hangout but this KIP was not discussed due t
ck in
broker property as meta data may also help consumers in the future. So I am
open to adding stack property to broker.
Hopefully we can discuss this in the next KIP hangout.
On Wed, Sep 30, 2015 at 2:46 PM, Allen Wang wrote:
> Can you send me the information on the next KIP hangout?
>
cture/architectureSnitchRackInf_c.html
>
> Voldemort does a static node -> zone assignment based on configuration.
>
> Aditya
>
> On Wed, Sep 30, 2015 at 10:05 AM, Allen Wang wrote:
>
> > I would like to see if we can do both:
> >
> > - Make RackLocator plu
replica on a different rack
> and
> > > the rest on same rack (if possible) sounds better to me.
> > >
> > > 2) Rack-locator class seems overly complex compared to adding a
> > rack.number
> > > property to the broker properties file. Why do we want t
ly complex compared to adding a rack.number
> property to the broker properties file. Why do we want that?
>
> Gwen
>
>
>
> On Mon, Sep 28, 2015 at 12:15 PM, Allen Wang wrote:
>
> > Hello Kafka Developers,
> >
> > I just created KIP-36 for rack aware repli
Hello Kafka Developers,
I just created KIP-36 for rack aware replica assignment.
https://cwiki.apache.org/confluence/display/KAFKA/KIP-36+Rack+aware+replica+assignment
The goal is to utilize the isolation provided by the racks in data center
and distribute replicas to racks to provide fault tole
[
https://issues.apache.org/jira/browse/KAFKA-1215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14909911#comment-14909911
]
Allen Wang commented on KAFKA-1215:
---
Started the work on KIP:
https://cwiki.apache
[
https://issues.apache.org/jira/browse/KAFKA-1215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14746264#comment-14746264
]
Allen Wang commented on KAFKA-1215:
---
[~aauradkar] [~junrao] [~jkreps] My ap
Hello,
I would like to create a KIP but after I signed up for Apache Wiki I don't
seem to have write permission. Anything I need to do?
Thanks,
Allen
[
https://issues.apache.org/jira/browse/KAFKA-1215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14744465#comment-14744465
]
Allen Wang commented on KAFKA-1215:
---
[~aauradkar] Sure I can create a KIP. How
[
https://issues.apache.org/jira/browse/KAFKA-2106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14710364#comment-14710364
]
Allen Wang commented on KAFKA-2106:
---
I am curious in the following code why there
[
https://issues.apache.org/jira/browse/KAFKA-1792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14709887#comment-14709887
]
Allen Wang commented on KAFKA-1792:
---
I agree with [~88manpreet] that after
[
https://issues.apache.org/jira/browse/KAFKA-1215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14696026#comment-14696026
]
Allen Wang commented on KAFKA-1215:
---
[~junrao] Can you review the GitHub pull req
[
https://issues.apache.org/jira/browse/KAFKA-1215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14606043#comment-14606043
]
Allen Wang commented on KAFKA-1215:
---
[~junrao] AWS region (for example us-east-1)
[
https://issues.apache.org/jira/browse/KAFKA-1215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14598615#comment-14598615
]
Allen Wang commented on KAFKA-1215:
---
We have a working solution now for rack a
[
https://issues.apache.org/jira/browse/KAFKA-2096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14486355#comment-14486355
]
Allen Wang commented on KAFKA-2096:
---
To verify the fix, the socket connection
[
https://issues.apache.org/jira/browse/KAFKA-2096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allen Wang updated KAFKA-2096:
--
Reviewer: Jun Rao
Status: Patch Available (was: Open)
> Enable keepalive socket option
[
https://issues.apache.org/jira/browse/KAFKA-2096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allen Wang updated KAFKA-2096:
--
Attachment: patch.diff
Patch for the fix
> Enable keepalive socket option for broker to prevent soc
[
https://issues.apache.org/jira/browse/KAFKA-2096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14483849#comment-14483849
]
Allen Wang commented on KAFKA-2096:
---
[~junrao], yes I would like submit a patch.
[
https://issues.apache.org/jira/browse/KAFKA-2096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allen Wang updated KAFKA-2096:
--
Summary: Enable keepalive socket option for broker to prevent socket leak
(was: Enable keepalive
Allen Wang created KAFKA-2096:
-
Summary: Enable keepalive socket option for broker
Key: KAFKA-2096
URL: https://issues.apache.org/jira/browse/KAFKA-2096
Project: Kafka
Issue Type: Improvement
[
https://issues.apache.org/jira/browse/KAFKA-1215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14372058#comment-14372058
]
Allen Wang commented on KAFKA-1215:
---
What's the status of this JIRA?
I
:
> Probably you can try restarting the controller and have same version for
> the controller and the brokers.
> BTW, was there any specific reason you are running 2 different versions for
> the controller and other brokers?
>
> Thanks,
>
> Mayuresh
>
> On Tue, Mar 1
roller watcher
> fires for the respective zookeeper path. Is your watcher still alive?
>
> Thanks,
>
> Mayuresh
>
> On Tue, Mar 17, 2015 at 1:29 PM, Allen Wang
> wrote:
>
> > Looking a bit more into controller log, it seems that when the partition
> > assignmen
:51)
Does it explain why the brokers are not aware of the new assignments? Is
there anyway to recover from this communication problem, like restarting
the controller?
Thanks,
Allen
On Tue, Mar 17, 2015 at 10:34 AM, Allen Wang wrote:
> Hello,
>
> I developed a tool to add partitions a
Hello,
I developed a tool to add partitions and assign new partitions to a set of
brokers in one operation by utilizing the API
AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK().
It worked well in most cases. However, in one case, I found that the
brokers are not aware of new partitions
Allen Wang created KAFKA-1829:
-
Summary: Support adding new partitions on brokers without
reassignment of existing partitions
Key: KAFKA-1829
URL: https://issues.apache.org/jira/browse/KAFKA-1829
Project
71 matches
Mail list logo
