Thanks for your detailed explanation. It is quite clear to me now. Guodong
On Thu, Apr 11, 2013 at 12:58 AM, Benjamin Mahler <[email protected] > wrote: > Yes, each executor ID can have only one 'source'. That is the only > constraint, which means you can share the same 'source' across different > executors. To see the benefit of the 'source' field, take this example > below. > > Say we have a framework and we want to spin up three executors to run a > job. We can have three executors: > 1: ExecutorID: "e_1", source: "page_rank_job" > 1: ExecutorID: "e_2", source: "page_rank_job" > 1: ExecutorID: "e_3", source: "page_rank_job" > > This allows us to identify the source of the executor. This is also > exported as part of the monitoring stats on each slave. > > $ curl <slave>:5051/monitor/usage.json > [ > {"executor_id":"e1", > "executor_name": "task_tracker", > "source": "page_rank_job", > "framework_id":"201301240028-2449582272-5050-61291-0000", > "resource_usage": > {"cpu_time":1.0, // CPU time (user + system) in seconds. > "cpu_usage":0.5, // CPU usage [0.5 ~ 50% of 1 core]. > "memory_rss":1048576, // RSS in bytes. > // ... Additional stats in the future. > }, > }, > {"executor_id":"e2", > "executor_name": "task_tracker", > "source": "page_rank_job", > "framework_id":"201301240028-2449582272-5050-61291-0000", > "resource_usage": > {"cpu_time":1.0, // CPU time (user + system) in seconds. > "cpu_usage":16.0, // CPU usage [16.0 ~ 1600% usage ~ 100% of 16 > cores] > "memory_rss":1048576, // RSS in bytes. > // ... Additional stats in the future. > }, > }, > > > Here we can aggregate all the resource usage for the page_rank_job by > asking all of the slaves for the usage information. But if you're looking > to write a framework, you can do without setting the source field for now, > and set it later if the need arises. > > I'll eventually have some more documentation around this: > https://issues.apache.org/jira/browse/MESOS-373 > > On Wed, Apr 10, 2013 at 2:28 AM, 王国栋 <[email protected]> wrote: > > > Hi Ben, > > > > I find some comment in mesos.proto file, > > // Source is an identifier style string used by frameworks to track > > // the source of an executor. > > > > Does this mean each executorId can only have the identical source ? > > In the old long-lived-framework, we are trying to assign different > sources > > to the same executor. So the error happens. > > > > Am I right? > > > > Thanks. > > > > Guodong > > > > > > On Wed, Apr 10, 2013 at 3:02 PM, 王国栋 <[email protected]> wrote: > > > > > Hi Ben, > > > > > > It works now. Thank you for your reply. > > > > > > I am trying to learn to write a framework on mesos. But I can not find > > the > > > exact meaning of each param in the API. eg. I do not know the meaning > of > > > "source". Where can I find the docs about the programming guide? > > > > > > Thanks. > > > > > > > > > Guodong > > > > > > > > > On Wed, Apr 10, 2013 at 2:09 AM, Benjamin Mahler < > > > [email protected]> wrote: > > > > > >> Thanks for the report! > > >> > > >> The bug here is that we set the source for each task: > > >> > > >> TaskInfo task; > > >> task.set_name("Task " + lexical_cast<string>(taskId)); > > >> > task.mutable_task_id()->set_value(lexical_cast<string>(taskId)); > > >> task.mutable_slave_id()->MergeFrom(offer.slave_id()); > > >> task.mutable_executor()->MergeFrom(executor); > > >> *task.mutable_executor()->set_source("task_" + > > >> stringify(taskId));* > > >> > > >> I'll have a review out shortly to fix this. > > >> > > >> > > >> On Tue, Apr 9, 2013 at 2:43 AM, 王国栋 <[email protected]> wrote: > > >> > > >> > hi > > >> > > > >> > I am trying to run long-lived-framework in the trunk. > > >> > But I find the following error after task 1 finished. > > >> > > > >> > W0409 17:18:03.841472 15305 master.cpp:1566] Error validating task > 1 : > > >> Task > > >> > has invalid ExecutorInfo (existing ExecutorInfo with same ExecutorID > > is > > >> not > > >> > compatible) > > >> > > > >> > Then all the tasks will be lost. > > >> > > > >> > The log of the framework is as follow: > > >> > Registered! > > >> > .Starting task 0 on guodong-Vostro-3400 > > >> > Task 0 is in state 1 > > >> > Task 0 is in state 2 > > >> > .Starting task 1 on guodong-Vostro-3400 > > >> > Task 1 is in state 5 > > >> > .Starting task 2 on guodong-Vostro-3400 > > >> > Task 2 is in state 5 > > >> > .Starting task 3 on guodong-Vostro-3400 > > >> > Task 3 is in state 5 > > >> > > > >> > > > >> > I also go through the code of LongLivedFramework. And I can not > > >> understand > > >> > this error, since the ExecutorInfo is passed as the constructor > > >> arguments > > >> > of Scheduler. > > >> > > > >> > Best regards. > > >> > > > >> > Guodong > > >> > > > >> > > > > > > > > >
