[jira] [Commented] (MESOS-6003) Add logging module for logging to an external program
[ https://issues.apache.org/jira/browse/MESOS-6003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16264839#comment-16264839 ] Will Rouesnel commented on MESOS-6003: -- Is there any further changes needed to try and advance this to being merged? > Add logging module for logging to an external program > - > > Key: MESOS-6003 > URL: https://issues.apache.org/jira/browse/MESOS-6003 > Project: Mesos > Issue Type: Improvement > Components: modules >Reporter: Will Rouesnel >Assignee: Will Rouesnel >Priority: Minor > > In the vein of the logrotate module for logging, there should be a similar > module which provides support for logging to an arbitrary log handling > program, with suitable task metadata provided by environment variables or > command line arguments. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (MESOS-6003) Add logging module for logging to an external program
[ https://issues.apache.org/jira/browse/MESOS-6003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16049820#comment-16049820 ] Will Rouesnel commented on MESOS-6003: -- [~wjoel] From a prettiness perspective I agree, but I wanted to preserve the ability to find MESOS_LOG injected env vars, and just specifying "executorinfo" didn't seem descriptive enough. Since you almost always need to post-process that field in some way, the user script/program is pretty likely to want to do something with it. > Add logging module for logging to an external program > - > > Key: MESOS-6003 > URL: https://issues.apache.org/jira/browse/MESOS-6003 > Project: Mesos > Issue Type: Improvement > Components: modules >Reporter: Will Rouesnel >Assignee: Will Rouesnel >Priority: Minor > > In the vein of the logrotate module for logging, there should be a similar > module which provides support for logging to an arbitrary log handling > program, with suitable task metadata provided by environment variables or > command line arguments. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (MESOS-1806) Etcd-based master contender/detector module
[ https://issues.apache.org/jira/browse/MESOS-1806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895593#comment-15895593 ] Will Rouesnel commented on MESOS-1806: -- So the necessary patches were just merged to this CoreOS project, which I can confirm allows Mesos and Marathon to work directly with etcd via an adapter proxy: https://github.com/coreos/zetcd > Etcd-based master contender/detector module > --- > > Key: MESOS-1806 > URL: https://issues.apache.org/jira/browse/MESOS-1806 > Project: Mesos > Issue Type: Epic > Components: leader election >Reporter: Ed Ropple >Assignee: Shuai Lin >Priority: Minor > >eropple: Could you also file a new JIRA for Mesos to drop ZK > in favor of etcd or ReplicatedLog? Would love to get some momentum going on > that one. > -- > Consider it filed. =) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (MESOS-1806) Etcd-based master contender/detector module
[ https://issues.apache.org/jira/browse/MESOS-1806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15884003#comment-15884003 ] Will Rouesnel commented on MESOS-1806: -- Is this work still ongoing (or has it been superceded by an in-tree effort in some way?) > Etcd-based master contender/detector module > --- > > Key: MESOS-1806 > URL: https://issues.apache.org/jira/browse/MESOS-1806 > Project: Mesos > Issue Type: Epic > Components: leader election >Reporter: Ed Ropple >Assignee: Shuai Lin >Priority: Minor > >eropple: Could you also file a new JIRA for Mesos to drop ZK > in favor of etcd or ReplicatedLog? Would love to get some momentum going on > that one. > -- > Consider it filed. =) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (MESOS-6003) Add logging module for logging to an external program
[ https://issues.apache.org/jira/browse/MESOS-6003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15874111#comment-15874111 ] Will Rouesnel commented on MESOS-6003: -- Huh I thought I had notified this thread - anyway - I've rebased the patchset for the changes that have been made to ContainerLogger interface, so this should be good to go again. > Add logging module for logging to an external program > - > > Key: MESOS-6003 > URL: https://issues.apache.org/jira/browse/MESOS-6003 > Project: Mesos > Issue Type: Improvement > Components: modules >Reporter: Will Rouesnel >Assignee: Will Rouesnel >Priority: Minor > > In the vein of the logrotate module for logging, there should be a similar > module which provides support for logging to an arbitrary log handling > program, with suitable task metadata provided by environment variables or > command line arguments. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Assigned] (MESOS-7125) ./configure does not run ./config.status
[ https://issues.apache.org/jira/browse/MESOS-7125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Will Rouesnel reassigned MESOS-7125: Assignee: Will Rouesnel > ./configure does not run ./config.status > > > Key: MESOS-7125 > URL: https://issues.apache.org/jira/browse/MESOS-7125 > Project: Mesos > Issue Type: Bug > Components: build, general >Affects Versions: 1.3.0 >Reporter: Will Rouesnel >Assignee: Will Rouesnel >Priority: Minor > > When checking out a fresh build with make, ./bootstrap && ./configure && make > will fail because ./config.status is not run by the ./configure script. > This is not major, but is surprising and not included in the current build > recipes provided by the documentation. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (MESOS-7125) ./configure does not run ./config.status
Will Rouesnel created MESOS-7125: Summary: ./configure does not run ./config.status Key: MESOS-7125 URL: https://issues.apache.org/jira/browse/MESOS-7125 Project: Mesos Issue Type: Bug Components: general Affects Versions: 1.3.0 Reporter: Will Rouesnel Priority: Minor When checking out a fresh build with make, ./bootstrap && ./configure && make will fail because ./config.status is not run by the ./configure script. This is not major, but is surprising and not included in the current build recipes provided by the documentation. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (MESOS-6003) Add logging module for logging to an external program
[ https://issues.apache.org/jira/browse/MESOS-6003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15864615#comment-15864615 ] Will Rouesnel commented on MESOS-6003: -- Rebased and building on master now, with added support for the new USER variable if someone could review. > Add logging module for logging to an external program > - > > Key: MESOS-6003 > URL: https://issues.apache.org/jira/browse/MESOS-6003 > Project: Mesos > Issue Type: Improvement > Components: modules >Reporter: Will Rouesnel >Assignee: Will Rouesnel >Priority: Minor > > In the vein of the logrotate module for logging, there should be a similar > module which provides support for logging to an arbitrary log handling > program, with suitable task metadata provided by environment variables or > command line arguments. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (MESOS-6529) Support ZFS sandbox isolation
Will Rouesnel created MESOS-6529: Summary: Support ZFS sandbox isolation Key: MESOS-6529 URL: https://issues.apache.org/jira/browse/MESOS-6529 Project: Mesos Issue Type: Improvement Components: containerization Reporter: Will Rouesnel Priority: Minor ZFS has excellent support for filesystem quotas under Linux/BSD operating systems. It would be good to support to support isolating workloads on separate ZFS filesystems that are dynamically created when the task is executed, and which could have disk quotas properly and invidually enforced. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MESOS-6522) Ability to set global maximum executor runtime for an agent
Will Rouesnel created MESOS-6522: Summary: Ability to set global maximum executor runtime for an agent Key: MESOS-6522 URL: https://issues.apache.org/jira/browse/MESOS-6522 Project: Mesos Issue Type: Improvement Components: slave Reporter: Will Rouesnel Priority: Minor With the developing concept of agent maintenance mode, it would be nice to have some blunt-force ability to reason about the behavior of agents on uncooperative frameworks. Ideally there would be a new parameter --executor_maximum_lifetime which would specify a maximum duration for which *any* executor on an agent can run before being terminated. Even when using persistent schedulers such as Marathon, the ability to enforce reasonable gurantees about when an agent's tasks definitely must end can help contribute to keeping the cluster turning over and prevent nodes becoming "special" or jammed up with jobs which will not end. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-6003) Add logging module for logging to an external program
[ https://issues.apache.org/jira/browse/MESOS-6003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15608482#comment-15608482 ] Will Rouesnel commented on MESOS-6003: -- Latest round of fixups is waiting for review again. > Add logging module for logging to an external program > - > > Key: MESOS-6003 > URL: https://issues.apache.org/jira/browse/MESOS-6003 > Project: Mesos > Issue Type: Improvement > Components: modules >Reporter: Will Rouesnel >Assignee: Will Rouesnel >Priority: Minor > > In the vein of the logrotate module for logging, there should be a similar > module which provides support for logging to an arbitrary log handling > program, with suitable task metadata provided by environment variables or > command line arguments. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MESOS-6358) Add watchdog timeout/action for mesos tasks which do not exit
Will Rouesnel created MESOS-6358: Summary: Add watchdog timeout/action for mesos tasks which do not exit Key: MESOS-6358 URL: https://issues.apache.org/jira/browse/MESOS-6358 Project: Mesos Issue Type: Improvement Components: slave Reporter: Will Rouesnel Priority: Minor When running with the docker containerizer, we've observed the scenario where a subproces of the docker container becomes a zombie due to a kernel bug (i.e. is completely unkillable). The effect of this was that Mesos kept reporting the task as running via it's API but not as existing to calls to delete it (being made by Marathon) as the actual docker-runc/docker-container process never exited (since none of the child processes exited waiting on the misbehaving subprocess). Mesos should include a parameter to deal with this situation - I woudl propose --task_kill_watchdog_timeout and --task_kill_watchdog_binary The idea would be that if a task still exists beyond the the length of timeout *after* the hard kill signal has been sent, then Mesos executes the watchdog binary action. The usage of such a process would be to allow alerting of an exceptional situation (if possible) and actioning to ensure worker nodes can stay up on average (i.e. if encountered, crash the node, let a BMC watchdog reboot it). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-6003) Add logging module for logging to an external program
[ https://issues.apache.org/jira/browse/MESOS-6003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15531689#comment-15531689 ] Will Rouesnel commented on MESOS-6003: -- Rebased and awaiting code review again against the latest master. > Add logging module for logging to an external program > - > > Key: MESOS-6003 > URL: https://issues.apache.org/jira/browse/MESOS-6003 > Project: Mesos > Issue Type: Improvement > Components: modules >Reporter: Will Rouesnel >Assignee: Will Rouesnel >Priority: Minor > > In the vein of the logrotate module for logging, there should be a similar > module which provides support for logging to an arbitrary log handling > program, with suitable task metadata provided by environment variables or > command line arguments. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-3401) Add labels to Resources
[ https://issues.apache.org/jira/browse/MESOS-3401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15513862#comment-15513862 ] Will Rouesnel commented on MESOS-3401: -- Another use case for this (which would need to be dynamic) is reporting container layers to a scheduler. i.e. a scheduler would like to start a docker container. When starting tasks, it would like to minimize the amount of extra network traffic, so it prioritizes starting tasks which require the fewest extra filesystem layers to be pulled. The actual implementation could be done a number of ways, but it would be nice to express the metadata via a Mesos offer rather then going out of band. > Add labels to Resources > --- > > Key: MESOS-3401 > URL: https://issues.apache.org/jira/browse/MESOS-3401 > Project: Mesos > Issue Type: Improvement > Components: slave >Reporter: Adam B > Labels: external-volumes, mesosphere, resources > > Similar to how we have added labels to tasks/executors (MESOS-2120), and even > FrameworkInfo (MESOS-2841), we should extend Resource to allow arbitrary > key/value pairs. > This could be used to specify that a cpu resource has a certain speed, that a > disk resource is SSD, or express any other metadata about a built-in or > custom resource type. Only the scalar quantity will be used for determining > fair share in the Mesos allocator. The rest will be passed onto frameworks as > info they can use for scheduling decisions. > This would require changes to how the slave specifies its `--resources` > (probably as json), how the slave/master reports resources in its web/json > API, and how resources are offered to frameworks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MESOS-6003) Add logging module for logging to an external program
Will Rouesnel created MESOS-6003: Summary: Add logging module for logging to an external program Key: MESOS-6003 URL: https://issues.apache.org/jira/browse/MESOS-6003 Project: Mesos Issue Type: Improvement Components: modules Reporter: Will Rouesnel Priority: Minor In the vein of the logrotate module for logging, there should be a similar module which provides support for logging to an arbitrary log handling program, with suitable task metadata provided by environment variables or command line arguments. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MESOS-5734) systemd.hpp header should be installed as part of Mesos header files
Will Rouesnel created MESOS-5734: Summary: systemd.hpp header should be installed as part of Mesos header files Key: MESOS-5734 URL: https://issues.apache.org/jira/browse/MESOS-5734 Project: Mesos Issue Type: Improvement Components: c++ api Affects Versions: 0.28.2, 1.0.0 Reporter: Will Rouesnel Priority: Minor When writing a container logger plugin ala the logrotate logger, there's a need to incorporate systemd functionality used internally by mesos to extend the lifespan's beyond the executor. This looks to be something almost every logging plugin would want to do when spawning subprocesses, but header files needed for it are currently buried in `src/linux` requiring a full Mesos src code installation. This functionality needs to be lifted and made available as part of the public API. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
