[
https://issues.apache.org/jira/browse/SAMZA-353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14104755#comment-14104755
]
Chris Riccomini commented on SAMZA-353:
---------------------------------------
Two additional thoughts (brain dumping right now):
# If we are only considering "global state" stores, then my initial concern
with the mutliple-checkpoints-for-single-SSP issue is unwarranted since global
state stores will always be reprocessed from offset 0. The individual tasks
should never need to checkpoint offsets for "global state" streams.
# There might be room for a hybrid approach somewhere between the SSPGrouper
approach and the traditional store.restore approach. I'm thinking mostly in
terms of implementation now. We could have the RunLoop.process method detect
when an incoming message is from a "global state" store, and funnel the message
directly to a shared state store, rather than sending it to a StreamTask's
process method. This approach would solve a number of things: we'd have one
"global state store" per container (rather than duplicate stores for every
task), still have the MessageChooser logic run to determine prioritization of
processing, and continue consuming the "global state" stream rather than
closing it off (as we do in the store.restore approach). Still, this approach
wouldn't solve the "my StreamTask can't modify the global state before writing
it" issue, but that might be OK.
> Support assigning the same SSP to multiple tasknames
> ----------------------------------------------------
>
> Key: SAMZA-353
> URL: https://issues.apache.org/jira/browse/SAMZA-353
> Project: Samza
> Issue Type: Bug
> Components: container
> Affects Versions: 0.8.0
> Reporter: Jakob Homan
>
> Post SAMZA-123, it is possible to add the same SSP to multiple tasknames,
> although currently we check for this and error out if this is done. We
> should think through the implications of having the same SSP appear in
> multiple tasknames and support this if it makes sense.
> This could be used as a broadcast stream that's either added by Samza itself
> to each taskname, or individual groupers could do this as makes sense. Right
> now the container maintains a map of SSP to TaskInstance and delivers the ssp
> to that task instance. With this change, we'd need to change the map to SSP
> to Set[TaskInstance] and deliver the message to each TI in the set.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
