One more thought: What about older Producer/Consumers? They don't understand the new protocol. How can we guarantee backward compatibility?
Or would this "only" imply, that there is no ordering guarantee for older clients? -Matthias On 2/22/18 6:24 PM, Matthias J. Sax wrote: > Dong, > > thanks a lot for the KIP! > > Can you elaborate how this would work for compacted topics? If it does > not work for compacted topics, I think Streams API cannot allow to scale > input topics. > > This question seems to be particularly interesting for deleting > partitions: assume that a key is never (or for a very long time) > updated, a partition cannot be deleted. > > > -Matthias > > > On 2/22/18 5:19 PM, Jay Kreps wrote: >> Hey Dong, >> >> Two questions: >> 1. How will this work with Streams and Connect? >> 2. How does this compare to a solution where we physically split partitions >> using a linear hashing approach (the partition number is equivalent to the >> hash bucket in a hash table)? https://en.wikipedia.org/wiki/Linear_hashing >> >> -Jay >> >> On Sat, Feb 10, 2018 at 3:35 PM, Dong Lin <lindon...@gmail.com> wrote: >> >>> Hi all, >>> >>> I have created KIP-253: Support in-order message delivery with partition >>> expansion. See >>> https://cwiki.apache.org/confluence/display/KAFKA/KIP- >>> 253%3A+Support+in-order+message+delivery+with+partition+expansion >>> . >>> >>> This KIP provides a way to allow messages of the same key from the same >>> producer to be consumed in the same order they are produced even if we >>> expand partition of the topic. >>> >>> Thanks, >>> Dong >>> >> >
signature.asc
Description: OpenPGP digital signature
