[
https://issues.apache.org/jira/browse/SAMZA-353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14109696#comment-14109696
]
Chinmay Soman commented on SAMZA-353:
-------------------------------------
Regarding broadcast stream (using unchecked SystemStreamPartitionGrouper):
=========================
If the use case for a broadcast stream is to provide means of communication
between the different tasks, does the MessageChooser have to do anything
special ? Like prioritize the broadcast stream over the others ? (Or maybe I
haven't understood MessageChooser in detail).
bq. Impose an ordering on offsets, and always start consuming from the lowest
offset that's required for all TaskNames within a single container for a given
SystemStreamPartition. The container can then filter any input messages that a
given StreamTask has already processed in cases where another task in the
container might be farther behind.
I'm a little confused here. If we're maintaining just one offset (the smallest)
- how can the container 'filter' messages for a given StreamTask that are
already processed ? How does this avoid reading duplicate input messages ?
> Support assigning the same SSP to multiple tasknames
> ----------------------------------------------------
>
> Key: SAMZA-353
> URL: https://issues.apache.org/jira/browse/SAMZA-353
> Project: Samza
> Issue Type: Bug
> Components: container
> Affects Versions: 0.8.0
> Reporter: Jakob Homan
> Attachments: DESIGN-SAMZA-353-0.md, DESIGN-SAMZA-353-0.pdf
>
>
> Post SAMZA-123, it is possible to add the same SSP to multiple tasknames,
> although currently we check for this and error out if this is done. We
> should think through the implications of having the same SSP appear in
> multiple tasknames and support this if it makes sense.
> This could be used as a broadcast stream that's either added by Samza itself
> to each taskname, or individual groupers could do this as makes sense. Right
> now the container maintains a map of SSP to TaskInstance and delivers the ssp
> to that task instance. With this change, we'd need to change the map to SSP
> to Set[TaskInstance] and deliver the message to each TI in the set.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
