Hey Colin,
Sure, that's understandable, we should keep the project stable first and
foremost. I'll work out the details of the client side reassignment changes
so perhaps we'll have better, clearer picture about this and we'll decide
when we feel the reassignment API can take more changes in :).
Hi Viktor,
Thanks for the thoughtful reply.
I'm actually not a PMC on the Apache Kafka project, although I hope to become
one in the future :) In any case, the tasks that the PMC tackles tend to be
more related to things like project policies, committers, legal stuff, brand
stuff, and so on.
Hi Colin,
Thanks for the honest answer. As a bottom line I think a better
reassignment logic should be included with Kafka somehow instead of relying
on external tools. It makes the system more mature on its own and
standardizes what others implemented many times.
Actually I also agree that
Hi Stanislav & Viktor,
Kafka has historically delegated partition reassignment to external tools. For
example, the choice of which partitions to reassign is determined by an
external tool. The choice of what throttle to use is also set externally.
So I think that putting this into the
Hey all,
I spoke offline with Viktor regarding this KIP and intentions on starting a
vote soon, so I made another pass on it. I have some comments:
>From our past discussion:
> I reiterated on the current algorithm and indeed it would change the order
of replicas in ZK but wouldn't do a leader
Hey Folks,
I think I'll open a vote early next week about this if there are no more
feedback.
Thanks,
Viktor
On Fri, Aug 9, 2019 at 5:25 PM Viktor Somogyi-Vass
wrote:
> Hey Stanislav,
>
> I reiterated on the current algorithm and indeed it would change the order
> of replicas in ZK but
Hey Stanislav,
I reiterated on the current algorithm and indeed it would change the order
of replicas in ZK but wouldn't do a leader election, so one would need to
run the preferred replica election tool. However still in the light of this
I'm not sure I'd keep the existing behavior as users
Hi Stan,
I meant the following (maybe rare) scenario - we have replicas [1, 2, 3] on
a lot of partitions and the user runs a massive rebalance to change them
all to [3, 2, 1]. In the old behavior, I think that this would not do
anything but simply change the replica set in ZK.
Then, the user
Hey Viktor,
I think it is intuitive if there are on a global level...If we applied
> them on every batch then we
> couldn't really guarantee any limits as the user would be able to get
> around it with submitting lots of reassignments.
Agreed. Could we mention this explicitly in the KIP?
Also
Hey Stanislav,
Thanks for the thorough look at the KIP! :)
> Let me first explicitly confirm my understanding of the configs and the
> algorithm:
> * reassignment.parallel.replica.count - the maximum total number of
> replicas that we can move at once, *per partition*
> *
Hey there Viktor,
Thanks for working on this KIP! I agree with the notion that reliability,
stability and predictability of a reassignment should be a core feature of
Kafka.
Let me first explicitly confirm my understanding of the configs and the
algorithm:
* reassignment.parallel.replica.count -
Hi All,
I've renamed my KIP as its name was a bit confusing so we'll continue it in
this thread.
The previous thread for record:
https://lists.apache.org/thread.html/0e97e30271f80540d4da1947bba94832639767e511a87bb2ba530fe7@%3Cdev.kafka.apache.org%3E
A short summary of the KIP:
In case of a vast
12 matches
Mail list logo
