[
https://issues.apache.org/jira/browse/SAMZA-353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14105999#comment-14105999
]
Yan Fang commented on SAMZA-353:
--------------------------------
For me, the use cases of "global state" and "global state store" make more
sense. So my following comment maybe a little out of this ticket's scope, more
use-case specific discussion.
If my understanding is correct, if using the suggested implementation, all
tasks will be able read the broadcast stream but can not modify the stream. If
that is the case, we may have another option: let AM read the global state and
then broadcast to all the containers. The "global state" could come from a
stream/key-value store/external db/user-defined variables from API. The
container will receive the global state when it starts. If we use a stream to
store the state, we also can let containers fully consume the state stream
every time it starts. The assumption here is that, the "global state" should
not be very big.
P.S. Spark has the concept of [share
variables|https://spark.apache.org/docs/latest/programming-guide.html#shared-variables],
which solve some of the state problem but still have many limitations. This
may give us some hints to make some trade-off.
> Support assigning the same SSP to multiple tasknames
> ----------------------------------------------------
>
> Key: SAMZA-353
> URL: https://issues.apache.org/jira/browse/SAMZA-353
> Project: Samza
> Issue Type: Bug
> Components: container
> Affects Versions: 0.8.0
> Reporter: Jakob Homan
>
> Post SAMZA-123, it is possible to add the same SSP to multiple tasknames,
> although currently we check for this and error out if this is done. We
> should think through the implications of having the same SSP appear in
> multiple tasknames and support this if it makes sense.
> This could be used as a broadcast stream that's either added by Samza itself
> to each taskname, or individual groupers could do this as makes sense. Right
> now the container maintains a map of SSP to TaskInstance and delivers the ssp
> to that task instance. With this change, we'd need to change the map to SSP
> to Set[TaskInstance] and deliver the message to each TI in the set.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
