Hi Matthias, Thanks for the KIP. Should TopologyBuilder#build() and KStreamBuilder#build accept a StreamsConfig as an argument?
Should we add a KStreamBuilder#topologyBuilder() for cases where people want to mix and match DSL & PAPI? Or do you think we already provide enough support for that via the DSL methods, i.e, KStream#process(...), KStream.transform(...) etc? I wonder if we could leave TopologyBuilder#addInternalTopic() ? It seems like it could be potentially useful? Perhaps the method could be renamed. Thanks, Damian On Mon, 6 Feb 2017 at 15:39 Mathieu Fenniak <mathieu.fenn...@replicon.com> wrote: > Hi Matthias, > > I use a few of the methods that you're pointing out that will be deprecated > and don't have an apparent alternative, so I wanted to just let you know > what they are and what my use-cases are for them. > > First of all, I use a combination of DSL and PAPI in the same application > very happily. It looks like this API would not permit that at all... it > looks like the intent here is to block that kind of usage, but as the DSL > is not extensible (short of building on "internal" API classes and using > reflection), this creates some real limitations in the type of applications > that can be built. > > TopologyBuilder.addInternalTopic -- as we've discussed before, I have some > reusable join components that build multiple processors w/ internal > repartition topics. We use addInternalTopic to create application-id > prefixed repartition topics. This could be worked around fairly easily. > > TopologyBuilder.nodeGroups & TopologyBuilder.build -- I have an option in > my KS app to run once & output a graphviz document with my entire topology > for debugging and analysis purposes; I use these methods to > create ProcessorTopology instances to inspect the topology and create this > output. I don't really see any alternative approach with this new API. > > KStreamBuilder.newName -- Similar to addInternalTopic, I use this to create > processor names in reusable components. Lacking this method would be > fairly easy to work around. > > Mathieu > > > On Fri, Feb 3, 2017 at 4:33 PM, Matthias J. Sax <matth...@confluent.io> > wrote: > > > Hi All, > > > > I did prepare a KIP to do some cleanup some of Kafka's Streaming API. > > > > Please have a look here: > > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > > 120%3A+Cleanup+Kafka+Streams+builder+API > > > > Looking forward to your feedback! > > > > > > -Matthias > > > > >
