One more thing, it looks like there is already a KIP-178: KIP-178: Size-based log directory selection strategy
Ismael On Wed, Jul 19, 2017 at 7:05 AM, Tom Bentley <t.j.bent...@gmail.com> wrote: > OK, I will work on adding support for this to the KIP, with the intention > of a two part implementation. > > On 19 July 2017 at 14:59, Ismael Juma <ism...@juma.me.uk> wrote: > > > Hi Tom, > > > > It's fine for the tool not to have this functionality from the start. > > However, since we're adding new Kafka protocol APIs, we need to consider > > some of these details to ensure we're building towards the end state, if > > that makes sense. Protocol APIs are used by multiple clients, so there is > > value in thinking ahead a bit when it comes to the design. The > > implementation can often be done in stages. > > > > Does that make sense? > > > > Ismael > > > > On Wed, Jul 19, 2017 at 6:23 AM, Tom Bentley <t.j.bent...@gmail.com> > > wrote: > > > > > Hi Ismael, > > > > > > Answers in-line: > > > > > > 1. Have you considered how progress would be reported? Partition > > > > reassignment can take a long time and it would be good to have a > > > mechanism > > > > for progress reporting. > > > > > > > > > > The ReassignPartitionsCommand doesn't currently have a mechanism to > track > > > progress. All you can do at the moment is initiate a reassignment (with > > > --execute), and later check whether the assignment is in the state you > > > asked for (with --verify). I agree it would be nice to be able to track > > > progress. > > > > > > This will be the first 'big' bit of work I've done on Kafka, so I would > > > prefer to limit the scope of this KIP where possible. That said, I > > suppose > > > it could be done by having receiving controllers publish their progress > > to > > > ZooKeeper, and adding Protocol and AdminClient API for getting this > > > information. If you're keen on this I can certainly modify the KIP to > add > > > this. > > > > > > Alternatively I could write a second KIP to add this ability. What > other > > > long running tasks are there for which we'd like the ability to report > > > progress? If there are others it might be possible to come up with a > > common > > > mechanism. > > > > > > > > > > 2. Removals can only happen in major releases. In your example, the > > > removal > > > > could only happen in 2.0.0. > > > > > > > > > > OK, I'll update the KIP. > > > > > >
