>> Another comment about Printed in general is it differs with other options >> that it is a required option than optional one, since it includes toSysOut >> / toFile specs; what are the pros and cons for including these two in the >> option and hence make it a required option than leaving them at the API >> layer and make Printed as optional for mapper / label only? >> >> >It isn't required as we will still have the no-arg print() which will just >go to sysout as it does now.
Got it. So just to clarify are we going to deprecate writeAsText or not? Guozhang On Wed, Aug 9, 2017 at 11:38 AM, Guozhang Wang <wangg...@gmail.com> wrote: > >> The key idea is that by using the same function name string for static > >> constructor and member functions, users do not need to remember what > are > >> the differences but can call these functions with any ordering they > want, > >> and later calls on the same spec will win over early calls. > >> > >> > >That would be great if java supported it, but it doesn't. You can't have > >static an member functions with the same signature. > > Got it, thanks. > > Does it still make sense to have one static constructors for each spec, > with one constructor having only one parameter to make it more usable, i.e. > as a user I do not need to give all parameters if I only want to override > one of them? Maybe we can just name the constructors as `with` but I'm not > sure if Java distinguish: > > public static <K, V> Produced<K, V> with(final Serde<K> keySerde) > public static <K, V> Produced<K, V> with(final Serde<V> valueSerde) > > as two function signatures. > > > Guozhang > > > On Wed, Aug 9, 2017 at 6:20 AM, Damian Guy <damian....@gmail.com> wrote: > >> On Tue, 8 Aug 2017 at 20:11 Guozhang Wang <wangg...@gmail.com> wrote: >> >> > Damian, >> > >> > Thanks for the proposal, I had a few comments on the APIs: >> > >> > 1. Printed#withFile seems not needed, as users should always spec if it >> is >> > to sysOut or to File at the beginning. In addition as a second thought, >> I >> > think serdes are not useful for prints anyways since we assume >> `toString` >> > is provided except for byte arrays, in which we will special handle it. >> > >> > >> +1 >> >> >> > Another comment about Printed in general is it differs with other >> options >> > that it is a required option than optional one, since it includes >> toSysOut >> > / toFile specs; what are the pros and cons for including these two in >> the >> > option and hence make it a required option than leaving them at the API >> > layer and make Printed as optional for mapper / label only? >> > >> > >> It isn't required as we will still have the no-arg print() which will just >> go to sysout as it does now. >> >> >> > >> > 2.1 KStream#through / to >> > >> > We should have an overloaded function without Produced? >> > >> >> Yes - we already have those so they are not part of the KIP, i.e, >> through(topic) >> >> >> > >> > 2.2 KStream#groupBy / groupByKey >> > >> > We should have an overloaded function without Serialized? >> > >> >> Yes, as above >> >> > >> > 2.3 KGroupedStream#count / reduce / aggregate >> > >> > We should have an overloaded function without Materialized? >> > >> >> As above >> >> > >> > 2.4 KStream#join >> > >> > We should have an overloaded function without Joined? >> > >> >> as above >> >> > >> > >> > 2.5 Each of KTable's operators: >> > >> > We should have an overloaded function without Produced / Serialized / >> > Materialized? >> > >> > >> as above >> >> >> > >> > >> > 3.1 Produced: the static functions have overlaps, which seems not >> > necessary. I'd suggest jut having the following three static with >> another >> > three similar member functions: >> > >> > public static <K, V> Produced<K, V> withKeySerde(final Serde<K> >> keySerde) >> > >> > public static <K, V> Produced<K, V> withValueSerde(final Serde<V> >> > valueSerde) >> > >> > public static <K, V> Produced<K, V> withStreamPartitioner(final >> > StreamPartitioner<K, V> partitioner) >> > >> > The key idea is that by using the same function name string for static >> > constructor and member functions, users do not need to remember what are >> > the differences but can call these functions with any ordering they >> want, >> > and later calls on the same spec will win over early calls. >> > >> > >> That would be great if java supported it, but it doesn't. You can't have >> static an member functions with the same signature. >> >> >> > >> > 3.2 Serialized: similarly >> > >> > public static <K, V> Serialized<K, V> withKeySerde(final Serde<K> >> keySerde) >> > >> > public static <K, V> Serialized<K, V> withValueSerde(final Serde<V> >> > valueSerde) >> > >> > public Serialized<K, V> withKeySerde(final Serde<K> keySerde) >> > >> > public Serialized<K, V> withValueSerde(final Serde valueSerde) >> > >> >> as above >> >> >> > >> > Also it has a final Serde<V> otherValueSerde in one of its static >> > constructor, it that intentional? >> > >> >> Nope: thanks. >> >> > >> > 3.3. Joined: similarly, keep the static constructor signatures the same >> as >> > its corresponding member fields. >> > >> > >> As above >> >> >> > 3.4 Materialized: it is a bit special, and I think we can keep its >> static >> > constructors with only two `as` as they are today.K >> > >> > >> 4. Is there any modifications on StateStoreSupplier? Is it replaced by >> > BytesStoreSupplier? Seems some more descriptions are lacking here. Also >> in >> > >> > >> No modifications to StateStoreSupplier. It is superseceded by >> BytesStoreSupplier. >> >> >> >> > public static <K, V, S extends StateStore> Materialized<K, V, S> >> > as(final StateStoreSupplier<S> >> > supplier) >> > >> > Is the parameter in type of BytesStoreSupplier? >> > >> >> Yep - thanks >> >> >> > >> > >> > >> > >> > Guozhang >> > >> > >> > On Thu, Jul 27, 2017 at 5:26 AM, Damian Guy <damian....@gmail.com> >> wrote: >> > >> > > Updated link: >> > > https://cwiki.apache.org/confluence/display/KAFKA/KIP- >> > > 182%3A+Reduce+Streams+DSL+overloads+and+allow+easier+ >> > > use+of+custom+storage+engines >> > > >> > > Thanks, >> > > Damian >> > > >> > > On Thu, 27 Jul 2017 at 13:09 Damian Guy <damian....@gmail.com> wrote: >> > > >> > > > Hi, >> > > > >> > > > I've put together a KIP to make some changes to the KafkaStreams DSL >> > that >> > > > will hopefully allow us to: >> > > > 1) reduce the explosion of overloads >> > > > 2) add new features without having to continue adding more overloads >> > > > 3) provide simpler ways for people to use custom storage engines and >> > wrap >> > > > them with logging, caching etc if desired >> > > > 4) enable per-operator caching rather than global caching without >> > having >> > > > to resort to supplying a StateStoreSupplier when you just want to >> turn >> > > > caching off. >> > > > >> > > > The KIP is here: >> > > > https://cwiki.apache.org/confluence/pages/viewpage. >> > > action?pageId=73631309 >> > > > >> > > > Thanks, >> > > > Damian >> > > > >> > > >> > >> > >> > >> > -- >> > -- Guozhang >> > >> > > > > -- > -- Guozhang > -- -- Guozhang
