Hey Steven, We introduced a field on the ExecutorInfo called "source <https://github.com/apache/mesos/blob/0.21.0/include/mesos/mesos.proto#L283>" [1] (which we didn't document well, sorry about that). This field should also be introduced on the TaskInfo <https://issues.apache.org/jira/browse/MESOS-486> [2] for those who just use the built-in CommandExecutor.
When launching an executor or a task, you can specify a "source" that allows you to identify the executor for monitoring / reporting purposes. This is how we do our "container" monitoring at Twitter: (1) Frameworks set "source" to a meaningful string: e.g. "prod.ads-team.ads-server.0". (environment, team, service, instance). (2) An internal system collects minutely data from /monitor/statistics.json on the slave. (3) The metrics include all those you listed above (disk utilization is coming). Once these are stored in an internal TSDB, dashboards, queries, and reports can be constructed for the relevant services, teams, etc. I hope this helps, IMHO this is probably the simplest path forward for you, as you don't need to have any pluggable functionality built-in to mesos to get what you need. Ben [1] https://github.com/apache/mesos/blob/0.21.0/include/mesos/mesos.proto#L283 [2] https://issues.apache.org/jira/browse/MESOS-486 On Thu, Dec 18, 2014 at 11:56 AM, Steven Schlansker < sschlans...@opentable.com> wrote: > > I am running a corporate Mesos cluster, shared by a number of teams and > projects. > We are looking to get some insight into our usage of precious computing > resources. For example, I'd like to be able to present a report breaking > down CPU-hour and RAM GB-hour utilization by service, team, or other > relevant grouping. > > How I'd imagine this works: > > * Collect Mesos statistics per task (allocated CPU, CPU utilization, > allocated memory, memory utilization, disk utilization) periodically (say, > once a minute) > * Collect task metadata from a pluggable source (mapping from Mesos task > to service name, team name, any other metadata you wish to use to group > tasks) > * Generate dashboard / reports by aggregating task data over axes provided > by metadata input > > Has anyone started on such a project? > > Thanks, > Steven > >
