[
https://issues.apache.org/jira/browse/SAMZA-123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13983158#comment-13983158
]
Chris Riccomini commented on SAMZA-123:
---------------------------------------
bq. I didn't see anyone pick up Jay Kreps's suggestion of storing the
SSP/cohort mapping in Zookeeper, but I think it would be worth considering.
For me, this falls into the same category as the KAFKA-1000 comment I had,
above. If we introduce this SetupLog concept for the job, why not just use it
for everything (SSP/Cohort assignment, checkpoints, and job config)? Using ZK
for SSPs/Cohort mapping, and KAFKA-1000 for checkpoints seems more complicated
than just having a single way to handle all of these use cases.
bq. SetupLog? It's essentially a list of changes required to setup a job before
it starts, right? Not sure how I feel about the name, just spit-balling.
Yes, commenting on my own comment. :) What about ConfigLog?
> Move topic partition grouping to the AM and generalize
> ------------------------------------------------------
>
> Key: SAMZA-123
> URL: https://issues.apache.org/jira/browse/SAMZA-123
> Project: Samza
> Issue Type: Sub-task
> Components: container
> Affects Versions: 0.6.0
> Reporter: Jakob Homan
> Assignee: Jakob Homan
> Attachments: SAMZA-123-design-doc.md, SAMZA-123-design-doc.pdf
>
>
> Currently the AM sends a set of all the topics and partitions to the
> container, which then groups them by partition and assigns each set to a task
> instance. By moving the grouping to the AM, we can assign arbitrary groups to
> task instances, which will allow more partitioning strategies, as discussed
> in SAMZA-71.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
