Thank you Matthias for your answers. The mailing list that you linked shows a solution using the Processor API.
Actually, the set of subtypes is not known in advance this is why I need to compute output topics from messages. So the branch method is of any help in my context. I think, this feature should be supported by the DSL as the Processor API solution is not completely safe. 2016-10-18 10:01 GMT+02:00 Damian Guy <damian....@gmail.com>: > Hi Florian, > > Do you know the set of subtypes in advance? I.e, could you use: > > KStream[] branches = stream.branch(predicates); > > to split the stream based on the subtypes? > > Thanks, > Damian > > > On Tue, 18 Oct 2016 at 00:43 Matthias J. Sax <matth...@confluent.io> > wrote: > > > -----BEGIN PGP SIGNED MESSAGE----- > > Hash: SHA512 > > > > Hi, > > > > using DSL you cannot do this. However, if you use Processor API you can. > > > > There are similar question on the mailing list already. For example: > > http://search-hadoop.com/m/uyzND1lghNN1tzbf41&subj=kafka+stream+to+new+t > > opic+based+on+message+key > > > > As we got this request multiple times already, it might be worth > > adding it IMHO. Not sure what the opinion of other is? We should make > > sure that the feature gets accepted before you put a lot of effort in > > it. :) > > > > > > - -Matthias > > > > On 10/17/16 2:10 PM, Florian Hussonnois wrote: > > > Hi All, > > > > > > Currently, it seems not possible with KStream to produce messages > > > to topics which are not known until runtime. > > > > > > For a new project I am evaluating the Kafka Connect / Kafka > > > Streams architecture but without that feature I cannot retain the > > > KStreams API. > > > > > > Our use case is pretty basic. We have xml messages in input of our > > > topology. Each message is splitted into subtypes and formatted in > > > Avro before being sent to a dedicated topic. > > > > > > So the output topics depend of the subtype of each message. > > > > > > I think it would be nice to add methods into the KStream interface > > > to provide such feature. > > > > > > If you think that feature would be usefull I can create a jira and > > > contribute to it. Also, do I need to create a new KIP as this > > > requires changes on a public API ? > > > > > > Thanks, > > > > > -----BEGIN PGP SIGNATURE----- > > Comment: GPGTools - https://gpgtools.org > > > > iQIcBAEBCgAGBQJYBWIhAAoJECnhiMLycopPfTQQAI69Uii5xd8KvaEo/Aeqs0Xw > > AzdPHekdVoHANzo1h45W1x3/lnyeMU/n2v09Agsz46cxb+Xbz9NOKGqT3v9Ye0Ic > > Eyl5yib1B6sWr41rGuYmwDH8zBoC8dPfGZiWhfXL4Sypey3RWzQlVAUWg8Ob4tqF > > rFeubMjWp7yopKRe/7n//JHF029hVK/ePk1vdEsI+2lBI4N7q9ONT/1wKkeCAtdd > > CCkI2WP/WbHzCcUVmOL41KoqgQFnmrH7BtLH67jumzEIR16H+ZenGZmS1uzde56E > > 9mEsl4wmAvfB5GJu6y7JnS7FnQotw7pV7ZneQrA2q8eCZHZqs2fkXf+6ZJNYRir+ > > rysqt8wJG69ZN9bSNO1Q6/fNbRiSjYi0I7JnzkErP6scfDKlf3bWzlw6Ejc0+iUr > > Cd0x2m/RlCepVleMT0UZNDlJd0Ml9Q77npP1lyntHVYHjVvtZLdlB5BQYdTMAx3N > > KCLZ+WkY2CBKcwh/KuMr9kW2eWSxH89JSwEule+1bN4vSKyBA6vtrwDoshf6N23g > > dEhTiY5NsgkvAe1JEK6d7PLN2Tq1Tq4OJNoP8PZlqW+YSFl41klQUblo8yT1jSlF > > iCyQS4rgNRabjBs1iZnZNoZ5eodoJMcUyWPhHUYHne+MXuSr1cNNGeNcbS5W0UyE > > dPCe2IiY4zErzxW/Mjmw > > =4DpY > > -----END PGP SIGNATURE----- > > > -- Florian HUSSONNOIS
