On Wed, Jun 19, 2019, at 03:36, Stanislav Kozlovski wrote: > Hey there Colin, > > Thanks for the work on this KIP. It is a much-needed improvement and I'm > excited to see it. Sorry for coming in so late to the discussion, I have > one question to better understand the change and a small suggestion. > > I see we allow reassignment cancellation at the partition level - what is > the motivation behind that? I think that having the back-end structures > support it is a good choice since it allows us more flexibility in the > future but what are the reasons for allowing a user to cancel at a > partition level? I think allowing it might let users shoot themselves in > the foot easier and make tools harder to implement (needing to guard > against it).
Hi Stanislav, Thanks for taking a look. I'm not sure I follow the comment about cancellation. Why would we need to guard against someone cancelling a reassignment of a single partition? Why would cancelling a single partition's reassignment be "shooting yourself in the foot"? > In all regards, what do you think about an ease of use improvement where we > allow a user to cancel all reassignments for a topic without specifying its > partitions? Essentially, we could cancel all reassignments for a topic if > the Partitions field in AlterPartitionAssignmentsRequest is null. I'm not sure why we would want to cancel all reassignments for a particular topic. In general, I would expect reassignments to be cancelled if the situation on a broker changed, and it was overloaded instead of underloaded. Or, some reassignments might be cancelled if they created more overhead for users of the system than anticipated. In both of these cases, topics are not really relevant. After all, the partitions for a particular topic are probably spread across the whole system. Topics are a useful administrative concept, but not really that relevant to the world of partition reassignment (or maybe I'm missing something?) best, Colin > > Best, > Stanislav > > On Mon, May 6, 2019 at 5:42 PM Colin McCabe <cmcc...@apache.org> wrote: > > > On Mon, May 6, 2019, at 07:39, Ismael Juma wrote: > > > Hi Colin, > > > > > > A quick comment. > > > > > > On Sat, May 4, 2019 at 11:18 PM Colin McCabe <cmcc...@apache.org> wrote: > > > > > > > The big advantage of doing batching on the controller is that the > > > > controller has more information about what is going on in the > > cluster. So > > > > it can schedule reassignments in a more optimal way. For instance, it > > can > > > > schedule reassignments so that the load is distributed evenly across > > > > nodes. This advantage is lost if we have to adhere to a rigid ordering > > > > that is set up in advance. We don't know exactly when anything will > > > > complete in any case. Just because one partition reassignment was > > started > > > > before another doesn't mean it will finish before another. > > > > > > > > > This is not quite true, right? The Controller doesn't know about > > partition > > > sizes, throughput per partition and other such information that external > > > tools like Cruise Control track. > > > > Hi Ismael, > > > > That's a good point, and one I should have included. > > > > I guess when I think about "do batching in the controller" versus "do > > batching in an external system" I tend to think about the information the > > controller could theoretically collect, rather than what it actually does > > :) But certainly, adding this information to the controller would be a > > significant change, and maybe one we don't want to do if the external > > systems work well enough. > > > > Thinking about this a little bit more, I can see three advantages to > > controller-side batching. Firstly, doing batching in the controller saves > > memory because we don't use a separate JVM, and don't duplicate the > > in-memory map of all the partitions. Secondly, the information we're > > acting on would also be more up-to-date. (I'm not sure how important this > > would be.) Finally, it's one less thing to deploy. I don't know if those > > are really enough to motivate switching now, but in a greenfield system I > > would probably choose controller-side rebalancing. > > > > In any case, this KIP is orthogonal to controller-side rebalancing versus > > external rebalancing. That's why the KIP states that we will continue to > > perform all the given partition rebalances immediately. I was just > > responding to the idea that maybe we should have an "ordering" of > > rebalancing partitions. I don't think we want that, for controller-side > > rebalancing or externally batched rebalancing. > > > > best, > > Colin > > > > > -- > Best, > Stanislav >
