On Tue, 8 Aug 2017 at 20:11 Guozhang Wang <wangg...@gmail.com> wrote:
> Damian, > > Thanks for the proposal, I had a few comments on the APIs: > > 1. Printed#withFile seems not needed, as users should always spec if it is > to sysOut or to File at the beginning. In addition as a second thought, I > think serdes are not useful for prints anyways since we assume `toString` > is provided except for byte arrays, in which we will special handle it. > > +1 > Another comment about Printed in general is it differs with other options > that it is a required option than optional one, since it includes toSysOut > / toFile specs; what are the pros and cons for including these two in the > option and hence make it a required option than leaving them at the API > layer and make Printed as optional for mapper / label only? > > It isn't required as we will still have the no-arg print() which will just go to sysout as it does now. > > 2.1 KStream#through / to > > We should have an overloaded function without Produced? > Yes - we already have those so they are not part of the KIP, i.e, through(topic) > > 2.2 KStream#groupBy / groupByKey > > We should have an overloaded function without Serialized? > Yes, as above > > 2.3 KGroupedStream#count / reduce / aggregate > > We should have an overloaded function without Materialized? > As above > > 2.4 KStream#join > > We should have an overloaded function without Joined? > as above > > > 2.5 Each of KTable's operators: > > We should have an overloaded function without Produced / Serialized / > Materialized? > > as above > > > 3.1 Produced: the static functions have overlaps, which seems not > necessary. I'd suggest jut having the following three static with another > three similar member functions: > > public static <K, V> Produced<K, V> withKeySerde(final Serde<K> keySerde) > > public static <K, V> Produced<K, V> withValueSerde(final Serde<V> > valueSerde) > > public static <K, V> Produced<K, V> withStreamPartitioner(final > StreamPartitioner<K, V> partitioner) > > The key idea is that by using the same function name string for static > constructor and member functions, users do not need to remember what are > the differences but can call these functions with any ordering they want, > and later calls on the same spec will win over early calls. > > That would be great if java supported it, but it doesn't. You can't have static an member functions with the same signature. > > 3.2 Serialized: similarly > > public static <K, V> Serialized<K, V> withKeySerde(final Serde<K> keySerde) > > public static <K, V> Serialized<K, V> withValueSerde(final Serde<V> > valueSerde) > > public Serialized<K, V> withKeySerde(final Serde<K> keySerde) > > public Serialized<K, V> withValueSerde(final Serde valueSerde) > as above > > Also it has a final Serde<V> otherValueSerde in one of its static > constructor, it that intentional? > Nope: thanks. > > 3.3. Joined: similarly, keep the static constructor signatures the same as > its corresponding member fields. > > As above > 3.4 Materialized: it is a bit special, and I think we can keep its static > constructors with only two `as` as they are today.K > > 4. Is there any modifications on StateStoreSupplier? Is it replaced by > BytesStoreSupplier? Seems some more descriptions are lacking here. Also in > > No modifications to StateStoreSupplier. It is superseceded by BytesStoreSupplier. > public static <K, V, S extends StateStore> Materialized<K, V, S> > as(final StateStoreSupplier<S> > supplier) > > Is the parameter in type of BytesStoreSupplier? > Yep - thanks > > > > > Guozhang > > > On Thu, Jul 27, 2017 at 5:26 AM, Damian Guy <damian....@gmail.com> wrote: > > > Updated link: > > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > > 182%3A+Reduce+Streams+DSL+overloads+and+allow+easier+ > > use+of+custom+storage+engines > > > > Thanks, > > Damian > > > > On Thu, 27 Jul 2017 at 13:09 Damian Guy <damian....@gmail.com> wrote: > > > > > Hi, > > > > > > I've put together a KIP to make some changes to the KafkaStreams DSL > that > > > will hopefully allow us to: > > > 1) reduce the explosion of overloads > > > 2) add new features without having to continue adding more overloads > > > 3) provide simpler ways for people to use custom storage engines and > wrap > > > them with logging, caching etc if desired > > > 4) enable per-operator caching rather than global caching without > having > > > to resort to supplying a StateStoreSupplier when you just want to turn > > > caching off. > > > > > > The KIP is here: > > > https://cwiki.apache.org/confluence/pages/viewpage. > > action?pageId=73631309 > > > > > > Thanks, > > > Damian > > > > > > > > > -- > -- Guozhang >
