[
https://issues.apache.org/jira/browse/SAMZA-375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14236442#comment-14236442
]
Chris Riccomini commented on SAMZA-375:
---------------------------------------
[~kostassoid], very cool! I don't think you're missing anything crucial. I
suppose using the term "blocked on SAMZA-348" is a bit over-kill. We simply
were thinking of doing the integration after SAMZA-348 in order to avoid churn.
It's not yet clear how much the JobCoordinator changes are going to impact the
YARN AM, and what logic will land where. We were planning on holding on the
Mesos integration until this was more clear. The basic steps we were looking at
were:
# Add a skeleton JobCoordinator (JC). (SAMAZ-438, done)
# Make JC talk to SamzaContainer via HTTP. (SAMZA-444, done)
# Make JobRunner talk to JC via CoordinatorStream. (SAMZA-448, patch available,
blocked on SAMZA-226 to avoid merge conflicts)
# Eliminate the CheckpointManager and use CoordinatorStream instead. (SAMZA-465)
>From there, it will be clear what logic we can take out of the YARN AM and put
>in the JobCoordinator. The idea is to invert the control between the AM and
>the JC. Currently the AM tells the JC what to do. We're going to switch it to
>make the JC tell YARN what to do. Once that's done integrating with Mesos
>should be pretty trivial--we'll just have a plugin to tell Mesos what to do
>instead of YARN.
That said, this is a fair amount of work. If you've got something working, feel
free to post it. I am personally hesitant to merge it in, because the
JobCoordinator changes are already quite messy, but it may be of use to folks
in the meantime.
> Investigate Mesos Job Support
> -----------------------------
>
> Key: SAMZA-375
> URL: https://issues.apache.org/jira/browse/SAMZA-375
> Project: Samza
> Issue Type: Bug
> Components: hello-samza
> Reporter: Jon Bringhurst
> Assignee: Jon Bringhurst
> Labels: mesos, project
> Attachments: Screen Shot 2014-08-23 at 5.51.39 PM.png, Screen Shot
> 2014-09-22 at 8.59.12 AM.png, samza-mesos-multiple-jobs.jpg
>
>
> It would be nice if Samza had support for Mesos (https://mesos.apache.org/).
> The current plan is to create a MesosJob and MesosJobFactory, then look into
> what it would take to allow the AM code to act as a Mesos scheduler.
> The feasibility of this landing in trunk will be better understood after a
> rough prototype has been created.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
