[
https://issues.apache.org/jira/browse/KAFKA-4217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16472553#comment-16472553
]
Matthias J. Sax commented on KAFKA-4217:
----------------------------------------
It's just a question of the API – returning "zero or more records" was always
possible. However, you can return records either via `context.forward()` if via
the `return` statement – this part can be confusion because `return` only
allows you to return zero (ie, `null`) or one record (if you want to return
multiple records you need to use `context.forward`). In `flatMap` there is only
the `return` statement available that accepts an `Iterable` – thus, via
`return` you can emit zero or more record.
Also note, that in `transfromValues` you only allow to use `return` but we
don't allow to use `context.forward()` (because `context.forward` would allow
to modify the key). Thus, it might make sense to change the return type of
both, `Transformer#transform()` and `ValueTransformer#transform` to take an
`Iterable` and disallow the usage of `context.forward`. On the other hand,
`context.forward` allows to set metadata like record timestamp and thus, it
might be good to preserve `context.forward()`... Overall, it seem we need to
redesign the API to address the different concerns...
> KStream.transform equivalent of flatMap
> ---------------------------------------
>
> Key: KAFKA-4217
> URL: https://issues.apache.org/jira/browse/KAFKA-4217
> Project: Kafka
> Issue Type: Improvement
> Components: streams
> Affects Versions: 0.10.0.1
> Reporter: Elias Levy
> Priority: Major
> Labels: api, needs-kip, newbie
>
> {{KStream.transform}} gives you access to state stores while allowing you to
> return zero or one transformed {{KeyValue}}. Alas, it is unclear what method
> you should use if you want to access state stores and return zero or multiple
> {{KeyValue}}. Presumably you can use {{transform}}, always return {{null}},
> and use {{ProcessorContext.forward}} to emit {{KeyValues}}.
> It may be good to introduce a {{transform}}-like {{flatMap}} equivalent, or
> allow store access from other {{KStream}} methods, such as {{flatMap}} itself.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
