Hi Alex, Responses below.
On Fri, Jun 4, 2021 at 9:27 AM Alexandre Brasil <alexandre.bra...@gmail.com> wrote: > Hi Mohan, > > I like the idea of adding those methods to the API, but I'd like to make a > suggestion: > > Although the most used scenario for min() / max() might possibly be for > numeric values, I think they could also be > useful on other objects like Dates, LocalDates or Strings. Why limit the > API to Numbers only? > > There was no specific reason. Just addressing the common scenario. But I don't see why this can't be supported given your suggestion below. Extending on the above, couldn't we change the API to provide a > Comparator<V> instead of the Function<V, VR> > for those methods, and make them return a KTable<K, V> instead? Not only > would this approach not limit the > usage of those methods to Numbers, but they'd also preserve the origin from > the min/max value [1]. The extraction of > a single (numeric?) value could be achieved by a subsequent .mapValues() > operator, and this strategy would also > allow us to reuse the stream's current value serde on min / max, making the > Materialized an optional parameter. > > I like your idea though it is odd that min/max returns KTable<K, V> instead of the KTable<K, VR> (like in count), but mapValues should do the trick. One extra complication of this approach is that now we'd have to handle > repeated min/max values from different > origins (two semantically different objects for which the comparator > returns 0), but we could solve that by adding > a parameter to specify whether to use the older or newer value (or assuming > one of these options as default for a > simpler API?). > > I am not sure whether this complexity is warranted. Why can't we just stick to the way a regular Comparator works ? Can you give me a real world example ? > > I know it's an implementation issue, but I'm curious on how you'd solve > handling the <VR extends Number> on > the sum(). Since the multiple implementations of this interface don't have > a common constructor nor an interface > method to add two Numbers, would it be possible to implement sum() and > retain the original VR type on the > returned KTable? > Not knowing the schema of the value (V) has its own set of problems. As I have alluded to in the proposal, this is a little bit messy. We already have "reduce" which can be used to implement sum (mapValues().reduce()). Thinking about it more, do you think "sum" would be useful ? One hacky way to implement this is to inspect the type of the return when the "func" is called the first time OR infer from the materialized or have an explicit initializer. Thanks Mohan > [1]: An example scenario for this would be to find the min / max Bidding > for a product where, at the end of the > auction, I need not only the min / max value of said Bidding, but also the > bidder's contact information. > > Best, > Alexandre > > On Wed, Jun 2, 2021 at 8:54 PM Mohan Parthasarathy <mposde...@gmail.com> > wrote: > > > Hi, > > > > I have created a proposal for adding some additional aggregation APIs > like > > count. > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-747+Add+support+for+basic+aggregation+APIs > > > > I have noted down some of the issues that need discussion. Thanks to > > Matthias for helping me with the scope of the proposal. > > > > Thanks > > Mohan > > >
