Two things: 1. This is a minor thing but the proposed new name for KStreamBuilder is StreamsTopologyBuilder. I actually think we should not put topology in the name as topology is not a concept you need to understand at the kstreams layer right now. I'd think of three categories of concepts: (1) concepts you need to understand to get going even for a simple example, (2) concepts you need to understand to operate and debug a real production app, (3) concepts we truly abstract and you don't need to ever understand. I think in the kstream layer topologies are currently category (2), and this is where they belong. By introducing the name in even the simplest example it means the user has to go read about toplogies to really understand even this simple snippet. What if instead we called it KStreamsBuilder? 2. For the processor api, I think this api is mostly not for end users. However this are a couple cases where it might make sense to expose it. I think users coming from Samza, or JMS's MessageListener ( https://docs.oracle.com/javaee/7/api/javax/jms/MessageListener.html) understand a simple callback interface for message processing. In fact, people often ask why Kafka's consumer doesn't provide such an interface. I'd argue we do, it's KafkaStreams. The only issue is that the processor API documentation is a bit scary for a person implementing this type of api. My observation is that people using this style of API don't do a lot of cross-message operations, then just do single message operations and use a database for anything that spans messages. They also don't factor their code into many MessageListeners and compose them, they just have one listener that has the complete handling logic. Say I am a user who wants to implement a single Processor in this style. Do we have an easy way to do that today (either with the .transform/.process methods in kstreams or with the topology apis)? Is there anything we can do in the way of trivial helper code to make this better? Also, how can we explain that pattern to people? I think currently we have pretty in-depth docs on our apis but I suspect a person trying to figure out how to implement a simple callback might get a bit lost trying to figure out how to wire it up. A simple five line example in the docs would probably help a lot. Not sure if this is best addressed in this KIP or is a side comment.
Cheers, -Jay On Fri, Feb 3, 2017 at 3:33 PM, Matthias J. Sax <matth...@confluent.io> wrote: > Hi All, > > I did prepare a KIP to do some cleanup some of Kafka's Streaming API. > > Please have a look here: > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > 120%3A+Cleanup+Kafka+Streams+builder+API > > Looking forward to your feedback! > > > -Matthias > >
