[jira] [Updated] (MESOS-1621) Docker run networking should be configurable and support bridge network
[ https://issues.apache.org/jira/browse/MESOS-1621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy Chen updated MESOS-1621: Shepherd: Benjamin Hindman Docker run networking should be configurable and support bridge network --- Key: MESOS-1621 URL: https://issues.apache.org/jira/browse/MESOS-1621 Project: Mesos Issue Type: Improvement Reporter: Timothy Chen Assignee: Timothy Chen Labels: Docker Currently to easily support running executors in Docker image, we hardcode --net=host into Docker run so slave and executor and reuse the same mechanism to communicate, which is to pass the slave IP/PORT for the framework to respond with it's own hostname and port information back to setup the tunnel. We want to see how to abstract this or even get rid of host networking altogether if we have a good way to not rely on it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-1621) Docker run networking should be configurable and support bridge network
[ https://issues.apache.org/jira/browse/MESOS-1621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy Chen updated MESOS-1621: Component/s: containerization Docker run networking should be configurable and support bridge network --- Key: MESOS-1621 URL: https://issues.apache.org/jira/browse/MESOS-1621 Project: Mesos Issue Type: Improvement Components: containerization Reporter: Timothy Chen Assignee: Timothy Chen Labels: Docker Currently to easily support running executors in Docker image, we hardcode --net=host into Docker run so slave and executor and reuse the same mechanism to communicate, which is to pass the slave IP/PORT for the framework to respond with it's own hostname and port information back to setup the tunnel. We want to see how to abstract this or even get rid of host networking altogether if we have a good way to not rely on it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1621) Docker run networking should be configurable and support bridge network
[ https://issues.apache.org/jira/browse/MESOS-1621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14122500#comment-14122500 ] Timothy Chen commented on MESOS-1621: - Want to give a update after discussion with different folks about the design of port mapping. Currently with bridge networking mode we must allow the user to expose ports from the container to the host otherwise it's not reachable. Docker has two options to do so: 1) Expose all ports specified in the image (-P) 2) Explicit mapping host port to container port. Technically there is a third option which is to expose just some container ports and let Docker choose what host port to map to. The conflicting factor here is that we cannot simply let the users map ports that is not part of the ports resource offer, so -P is not a viable option in this case as we cannot choose what ports are end up being assigned. Therefore I'm going for the explicit mapping ports option, and also verify that each host port specified is in range of the ports resource used. The cons of doing this is that for users that just submits a docker image through a framework, if the framework doesn't expose information about the ports resource offer it got then the user will not be able to know what ports to explicitly map to. This can be mitigated at least by framework developers to help either expose this information, or choose to randomly choose ports for the users within the resource offer range for each port the image exposes. The only information that the user will need to know is that ports within the container that it needs to be exposed. Docker run networking should be configurable and support bridge network --- Key: MESOS-1621 URL: https://issues.apache.org/jira/browse/MESOS-1621 Project: Mesos Issue Type: Improvement Reporter: Timothy Chen Assignee: Timothy Chen Labels: Docker Currently to easily support running executors in Docker image, we hardcode --net=host into Docker run so slave and executor and reuse the same mechanism to communicate, which is to pass the slave IP/PORT for the framework to respond with it's own hostname and port information back to setup the tunnel. We want to see how to abstract this or even get rid of host networking altogether if we have a good way to not rely on it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1571) Signal escalation timeout is not configurable
[
https://issues.apache.org/jira/browse/MESOS-1571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14122858#comment-14122858
]
Till Toenshoff commented on MESOS-1571:
---
Using the environment to pass that info seems to fit best when looking at the
things we already pass (e.g. {{MESOS_RECOVERY_TIMEOUT}}), whereas the
{{SlaveInfo}} protobuf is rather limited in additional execution specific
parameters.
However to me this still raises the question on why we prefer using the
environment instead of proto's for such information. One obvious reason
certainly is that we might need to supply information that is needed
immediately before or after starting the {{ExecutorProcess}} but definitely
before it successfully registered, when {{SlaveInfo}} finally becomes available
to him. Despite my above argument being we already do it that way, are there
better arguments for not adding things to the proto but instead using the
environment for passing the additional parameters?
[~benjaminhindman], [~idownes], [~tnachen] any input for this discussion?
Signal escalation timeout is not configurable
-
Key: MESOS-1571
URL: https://issues.apache.org/jira/browse/MESOS-1571
Project: Mesos
Issue Type: Bug
Reporter: Niklas Quarfot Nielsen
Assignee: Alexander Rukletsov
Even though the executor shutdown grace period is set to a larger interval,
the signal escalation timeout will still be 3 seconds. It should either be
configurable or dependent on EXECUTOR_SHUTDOWN_GRACE_PERIOD.
Thoughts?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (MESOS-1765) Use PID namespace to avoid freezing cgroup
Cong Wang created MESOS-1765: Summary: Use PID namespace to avoid freezing cgroup Key: MESOS-1765 URL: https://issues.apache.org/jira/browse/MESOS-1765 Project: Mesos Issue Type: Story Components: containerization Reporter: Cong Wang There is some known kernel issue when we freeze the whole cgroup upon OOM. Mesos probably can just use PID namespace so that we will only need to kill the init of the pid namespace, instead of freezing all the processes and killing them one by one. But I am not quite sure if this would break the existing code. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1765) Use PID namespace to avoid freezing cgroup
[ https://issues.apache.org/jira/browse/MESOS-1765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14123176#comment-14123176 ] Vinod Kone commented on MESOS-1765: --- What is the minimum kernel version required for pid namespaces? If it's not an old enough kernel, we need to figure out a way to use either freezer (w/ kill) or pid namespace depending on the kernel version. Use PID namespace to avoid freezing cgroup -- Key: MESOS-1765 URL: https://issues.apache.org/jira/browse/MESOS-1765 Project: Mesos Issue Type: Story Components: containerization Reporter: Cong Wang There is some known kernel issue when we freeze the whole cgroup upon OOM. Mesos probably can just use PID namespace so that we will only need to kill the init of the pid namespace, instead of freezing all the processes and killing them one by one. But I am not quite sure if this would break the existing code. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MESOS-1766) MasterAuthorizationTest.DuplicateRegistration test is flaky
Vinod Kone created MESOS-1766:
-
Summary: MasterAuthorizationTest.DuplicateRegistration test is
flaky
Key: MESOS-1766
URL: https://issues.apache.org/jira/browse/MESOS-1766
Project: Mesos
Issue Type: Bug
Components: test
Reporter: Vinod Kone
{code}
[ RUN ] MasterAuthorizationTest.DuplicateRegistration
Using temporary directory
'/tmp/MasterAuthorizationTest_DuplicateRegistration_pVJg7m'
I0905 15:53:16.398993 25769 leveldb.cpp:176] Opened db in 2.601036ms
I0905 15:53:16.399566 25769 leveldb.cpp:183] Compacted db in 546216ns
I0905 15:53:16.399590 25769 leveldb.cpp:198] Created db iterator in 2787ns
I0905 15:53:16.399605 25769 leveldb.cpp:204] Seeked to beginning of db in 500ns
I0905 15:53:16.399617 25769 leveldb.cpp:273] Iterated through 0 keys in the db
in 185ns
I0905 15:53:16.399633 25769 replica.cpp:741] Replica recovered with log
positions 0 - 0 with 1 holes and 0 unlearned
I0905 15:53:16.399817 25786 recover.cpp:425] Starting replica recovery
I0905 15:53:16.399952 25793 recover.cpp:451] Replica is in EMPTY status
I0905 15:53:16.400683 25795 replica.cpp:638] Replica in EMPTY status received a
broadcasted recover request
I0905 15:53:16.400795 25787 recover.cpp:188] Received a recover response from a
replica in EMPTY status
I0905 15:53:16.401005 25783 recover.cpp:542] Updating replica status to STARTING
I0905 15:53:16.401470 25786 master.cpp:286] Master
20140905-155316-3125920579-49188-25769 (penates.apache.org) started on
67.195.81.186:49188
I0905 15:53:16.401521 25786 master.cpp:332] Master only allowing authenticated
frameworks to register
I0905 15:53:16.401533 25786 master.cpp:337] Master only allowing authenticated
slaves to register
I0905 15:53:16.401543 25786 credentials.hpp:36] Loading credentials for
authentication from
'/tmp/MasterAuthorizationTest_DuplicateRegistration_pVJg7m/credentials'
I0905 15:53:16.401558 25793 leveldb.cpp:306] Persisting metadata (8 bytes) to
leveldb took 474683ns
I0905 15:53:16.401582 25793 replica.cpp:320] Persisted replica status to
STARTING
I0905 15:53:16.401667 25793 recover.cpp:451] Replica is in STARTING status
I0905 15:53:16.401669 25786 master.cpp:366] Authorization enabled
I0905 15:53:16.401898 25795 master.cpp:120] No whitelist given. Advertising
offers for all slaves
I0905 15:53:16.401936 25796 hierarchical_allocator_process.hpp:299]
Initializing hierarchical allocator process with master :
master@67.195.81.186:49188
I0905 15:53:16.402160 25784 replica.cpp:638] Replica in STARTING status
received a broadcasted recover request
I0905 15:53:16.402333 25790 master.cpp:1205] The newly elected leader is
master@67.195.81.186:49188 with id 20140905-155316-3125920579-49188-25769
I0905 15:53:16.402359 25790 master.cpp:1218] Elected as the leading master!
I0905 15:53:16.402371 25790 master.cpp:1036] Recovering from registrar
I0905 15:53:16.402472 25798 registrar.cpp:313] Recovering registrar
I0905 15:53:16.402529 25791 recover.cpp:188] Received a recover response from a
replica in STARTING status
I0905 15:53:16.402782 25788 recover.cpp:542] Updating replica status to VOTING
I0905 15:53:16.403002 25795 leveldb.cpp:306] Persisting metadata (8 bytes) to
leveldb took 116403ns
I0905 15:53:16.403020 25795 replica.cpp:320] Persisted replica status to VOTING
I0905 15:53:16.403081 25791 recover.cpp:556] Successfully joined the Paxos group
I0905 15:53:16.403197 25791 recover.cpp:440] Recover process terminated
I0905 15:53:16.403388 25796 log.cpp:656] Attempting to start the writer
I0905 15:53:16.403993 25784 replica.cpp:474] Replica received implicit promise
request with proposal 1
I0905 15:53:16.404147 25784 leveldb.cpp:306] Persisting metadata (8 bytes) to
leveldb took 132156ns
I0905 15:53:16.404167 25784 replica.cpp:342] Persisted promised to 1
I0905 15:53:16.404542 25795 coordinator.cpp:230] Coordinator attemping to fill
missing position
I0905 15:53:16.405498 25787 replica.cpp:375] Replica received explicit promise
request for position 0 with proposal 2
I0905 15:53:16.405868 25787 leveldb.cpp:343] Persisting action (8 bytes) to
leveldb took 347231ns
I0905 15:53:16.405886 25787 replica.cpp:676] Persisted action at 0
I0905 15:53:16.406553 25788 replica.cpp:508] Replica received write request for
position 0
I0905 15:53:16.406582 25788 leveldb.cpp:438] Reading position from leveldb took
11402ns
I0905 15:53:16.529067 25788 leveldb.cpp:343] Persisting action (14 bytes) to
leveldb took 535803ns
I0905 15:53:16.529088 25788 replica.cpp:676] Persisted action at 0
I0905 15:53:16.529355 25784 replica.cpp:655] Replica received learned notice
for position 0
I0905 15:53:16.529784 25784 leveldb.cpp:343] Persisting action (16 bytes) to
leveldb took 406036ns
I0905 15:53:16.529806 25784 replica.cpp:676] Persisted action at 0
I0905 15:53:16.529817 25784 replica.cpp:661] Replica learned NOP action at
position 0
I0905 15:53:16.530108 25783
[jira] [Commented] (MESOS-1765) Use PID namespace to avoid freezing cgroup
[ https://issues.apache.org/jira/browse/MESOS-1765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14123270#comment-14123270 ] Cong Wang commented on MESOS-1765: -- According to man page, clone(CLONE_NEWPID) requires kernel = 2.6.24, unshare(CLONE_NEWPID) requires 3.8 at least. Mesos probably only needs clone(CLONE_NEWPID), so it should be safe since the current network isolation code already requires kernel 3.4. Use PID namespace to avoid freezing cgroup -- Key: MESOS-1765 URL: https://issues.apache.org/jira/browse/MESOS-1765 Project: Mesos Issue Type: Story Components: containerization Reporter: Cong Wang There is some known kernel issue when we freeze the whole cgroup upon OOM. Mesos probably can just use PID namespace so that we will only need to kill the init of the pid namespace, instead of freezing all the processes and killing them one by one. But I am not quite sure if this would break the existing code. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (MESOS-186) Resource offers should be rescinded after some configurable timeout
[
https://issues.apache.org/jira/browse/MESOS-186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Mahler resolved MESOS-186.
---
Resolution: Fixed
Fix Version/s: 0.21.0
{noformat}
commit 707bf3b1d6f042ee92e7a291d3f74a20ae2d494b
Author: Kapil Arya ka...@mesosphere.io
Date: Fri Sep 5 11:15:15 2014 -0700
Added optional --offer_timeout to rescind unused offers.
The ability to set an offer timeout helps prevent unfair resource
allocations in the face of frameworks that hoard offers, or that
accidentally drop offers.
When optimistic offers are added, hoarding will not affect the
fairness for other frameworks.
Review: https://reviews.apache.org/r/22066
{noformat}
Resource offers should be rescinded after some configurable timeout
---
Key: MESOS-186
URL: https://issues.apache.org/jira/browse/MESOS-186
Project: Mesos
Issue Type: Improvement
Components: framework
Reporter: Benjamin Hindman
Assignee: Timothy Chen
Fix For: 0.21.0
Problem: a framework has a bug and holds on to resource offers by accident
for 24 hours/
One suggestion: resource offers should be rescinded after some configurable
timeout.
Possible issue: this might interfere with frameworks that are hoarding. But
one possible solution here is to add another API call which checks the status
of resource offers (i.e., remindAboutOffer).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (MESOS-1769) Segfault when using external containerizer
Iain Mckay created MESOS-1769: - Summary: Segfault when using external containerizer Key: MESOS-1769 URL: https://issues.apache.org/jira/browse/MESOS-1769 Project: Mesos Issue Type: Bug Components: containerization Affects Versions: 0.20.0 Environment: Ubuntu Trusty 64bit, Vagrant 1.6.3, 4GB RAM, 2 CPU's. Reporter: Iain Mckay I've been attemping to get the 0.4.2 release of Deimos running with Mesos 0.20.0. On launch with the following arguments, the system segfaults: bq. mesos-slave --master=zk://33.33.32.1:2181/mesos --ip=33.33.32.10 --containerizer_path=/usr/local/bin/deimos --containerizers=external You can find the core-dump [here|https://drive.google.com/file/d/0B8zUJw_2eBVYUFlYYTd0YWlLOFU/edit?usp=sharing] and I've extracted the stacktrace [here|https://gist.github.com/iainmckay/579fac480b9a89fb57ef]. I can provide more information and I'm available for further debugging if required. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MESOS-1770) Docker with command shell=true should override entrypoint
Timothy Chen created MESOS-1770: --- Summary: Docker with command shell=true should override entrypoint Key: MESOS-1770 URL: https://issues.apache.org/jira/browse/MESOS-1770 Project: Mesos Issue Type: Improvement Reporter: Timothy Chen Assignee: Timothy Chen Currently with the new CommandInfo there is a shell flag that if it's enabled, will wrap the command with /bin/sh -c with docker run. However we don't override the entrypoint, therefore when a user specified a image with a entrypoint and also have shell=true then /bin/sh -c will become part of the argument to the entrypoint. I don't think there is any example where users expect /bin/sh -c to be a argument in the entrypoint, and to make sure cases where shell is needed for expanding environment variables we also override the entrypoint. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-1770) Docker with command shell=true should override entrypoint
[ https://issues.apache.org/jira/browse/MESOS-1770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy Chen updated MESOS-1770: Target Version/s: 0.20.1 Shepherd: Benjamin Hindman Docker with command shell=true should override entrypoint - Key: MESOS-1770 URL: https://issues.apache.org/jira/browse/MESOS-1770 Project: Mesos Issue Type: Improvement Reporter: Timothy Chen Assignee: Timothy Chen Labels: docker Currently with the new CommandInfo there is a shell flag that if it's enabled, will wrap the command with /bin/sh -c with docker run. However we don't override the entrypoint, therefore when a user specified a image with a entrypoint and also have shell=true then /bin/sh -c will become part of the argument to the entrypoint. I don't think there is any example where users expect /bin/sh -c to be a argument in the entrypoint, and to make sure cases where shell is needed for expanding environment variables we also override the entrypoint. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
