[jira] [Created] (MESOS-1762) Avoid docker pull on each container run
Timothy Chen created MESOS-1762: --- Summary: Avoid docker pull on each container run Key: MESOS-1762 URL: https://issues.apache.org/jira/browse/MESOS-1762 Project: Mesos Issue Type: Improvement Reporter: Timothy Chen Assignee: Timothy Chen Currently the docker containerizer does a docker pull on each run, and this has several downsides: 1. Not able to run local images 2. Require to contact registry server on each run, therefore docker run becomes unavailable if the registry server is down. Also has scalability limits. We want to avoid doing a pull everytime. The downside ofcourse is that images without explicit tags (:latest) will not get the most updated version, but this is the same behavior when calling docker run on the cli. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1762) Avoid docker pull on each container run
[ https://issues.apache.org/jira/browse/MESOS-1762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14120998#comment-14120998 ] Timothy Chen commented on MESOS-1762: - Reviewboard: https://reviews.apache.org/r/25237 > Avoid docker pull on each container run > --- > > Key: MESOS-1762 > URL: https://issues.apache.org/jira/browse/MESOS-1762 > Project: Mesos > Issue Type: Improvement >Reporter: Timothy Chen >Assignee: Timothy Chen > > Currently the docker containerizer does a docker pull on each run, and this > has several downsides: > 1. Not able to run local images > 2. Require to contact registry server on each run, therefore docker run > becomes unavailable if the registry server is down. Also has scalability > limits. > We want to avoid doing a pull everytime. The downside ofcourse is that images > without explicit tags (:latest) will not get the most updated version, but > this is the same behavior when calling docker run on the cli. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1750) ImportError in containerizer_pb2.py
[
https://issues.apache.org/jira/browse/MESOS-1750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14120770#comment-14120770
]
Thomas Rampelberg commented on MESOS-1750:
--
It'd be possible to have dummy modules at the `mesos.interface` layer which
makes it possible to import mesos.interface.mesos_pb2 from externally and let
the generated pb2s import as they see fit.
That said, I still prefer the sed option.
> ImportError in containerizer_pb2.py
> ---
>
> Key: MESOS-1750
> URL: https://issues.apache.org/jira/browse/MESOS-1750
> Project: Mesos
> Issue Type: Bug
> Components: python api
>Reporter: Till Toenshoff
> Labels: mesos.interface, python-egg
>
> It appears as if the split of the Python egg (native/interface/protocol) has
> introduced a new issue.
> Whenever I am trying to use {{from mesos.interface import
> containerizer_pb2}}, I am receiving an {{ImportError: No module named
> mesos_pb2}}.
> For replicating this issue, try running
> {{"build/src/examples/python/test-containerizer"}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (MESOS-1761) docker containerizer should override entrypoint when starting executor.
Jay Buffington created MESOS-1761:
-
Summary: docker containerizer should override entrypoint when
starting executor.
Key: MESOS-1761
URL: https://issues.apache.org/jira/browse/MESOS-1761
Project: Mesos
Issue Type: Bug
Components: containerization
Reporter: Jay Buffington
Assignee: Timothy Chen
When I specify an ExecutorInfo with a ContainerInfo set, the docker
containerizer will do {{docker run ... /bin/sh -c ./my-executor}}.
If the docker image the user specifies has an entrypoint configured the
executor command will be appended to the entrypoint rather than overwriting it.
The docker docs says:
{quote}
Unlike the behavior of the CMD instruction, The ENTRYPOINT instruction adds an
entry command that will not be overwritten when arguments are passed to docker
run. This allows arguments to be passed to the entry point, i.e. docker run
-d will pass the -d argument to the entry point.
{quote}
This results in docker running the entrypoint inside the container with the
executorinfo's command as an argument. If you specified an executor this is
never what you want.
I suggest docker run use {{--entrypoint}} when starting an executor inside the
container.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (MESOS-1760) MasterAuthorizationTest.FrameworkRemovedBeforeReregistration is flaky
Vinod Kone created MESOS-1760:
-
Summary:
MasterAuthorizationTest.FrameworkRemovedBeforeReregistration is flaky
Key: MESOS-1760
URL: https://issues.apache.org/jira/browse/MESOS-1760
Project: Mesos
Issue Type: Bug
Components: test
Reporter: Vinod Kone
Observed this on Apache CI:
https://builds.apache.org/job/Mesos-Trunk-Ubuntu-Build-Out-Of-Src-Disable-Java-Disable-Python-Disable-Webui/2355/changes
{code}
[ RUN] MasterAuthorizationTest.FrameworkRemovedBeforeReregistration
Using temporary directory
'/tmp/MasterAuthorizationTest_FrameworkRemovedBeforeReregistration_0tw16Z'
I0903 22:04:33.520237 25565 leveldb.cpp:176] Opened db in 49.073821ms
I0903 22:04:33.538331 25565 leveldb.cpp:183] Compacted db in 18.065051ms
I0903 22:04:33.538363 25565 leveldb.cpp:198] Created db iterator in 4826ns
I0903 22:04:33.538377 25565 leveldb.cpp:204] Seeked to beginning of db in 682ns
I0903 22:04:33.538385 25565 leveldb.cpp:273] Iterated through 0 keys in the db
in 312ns
I0903 22:04:33.538399 25565 replica.cpp:741] Replica recovered with log
positions 0 -> 0 with 1 holes and 0 unlearned
I0903 22:04:33.538624 25593 recover.cpp:425] Starting replica recovery
I0903 22:04:33.538707 25598 recover.cpp:451] Replica is in EMPTY status
I0903 22:04:33.540909 25590 master.cpp:286] Master
20140903-220433-453759884-44122-25565 (hemera.apache.org) started on
140.211.11.27:44122
I0903 22:04:33.540932 25590 master.cpp:332] Master only allowing authenticated
frameworks to register
I0903 22:04:33.540936 25590 master.cpp:337] Master only allowing authenticated
slaves to register
I0903 22:04:33.540941 25590 credentials.hpp:36] Loading credentials for
authentication from
'/tmp/MasterAuthorizationTest_FrameworkRemovedBeforeReregistration_0tw16Z/credentials'
I0903 22:04:33.541337 25590 master.cpp:366] Authorization enabled
I0903 22:04:33.541508 25597 replica.cpp:638] Replica in EMPTY status received a
broadcasted recover request
I0903 22:04:33.542343 25582 hierarchical_allocator_process.hpp:299]
Initializing hierarchical allocator process with master :
master@140.211.11.27:44122
I0903 22:04:33.542445 25592 master.cpp:120] No whitelist given. Advertising
offers for all slaves
I0903 22:04:33.543175 25602 recover.cpp:188] Received a recover response from a
replica in EMPTY status
I0903 22:04:33.543637 25587 recover.cpp:542] Updating replica status to STARTING
I0903 22:04:33.544256 25579 master.cpp:1205] The newly elected leader is
master@140.211.11.27:44122 with id 20140903-220433-453759884-44122-25565
I0903 22:04:33.544275 25579 master.cpp:1218] Elected as the leading master!
I0903 22:04:33.544282 25579 master.cpp:1036] Recovering from registrar
I0903 22:04:33.544401 25579 registrar.cpp:313] Recovering registrar
I0903 22:04:33.558487 25593 leveldb.cpp:306] Persisting metadata (8 bytes) to
leveldb took 14.678563ms
I0903 22:04:33.558531 25593 replica.cpp:320] Persisted replica status to
STARTING
I0903 22:04:33.558653 25593 recover.cpp:451] Replica is in STARTING status
I0903 22:04:33.559867 25588 replica.cpp:638] Replica in STARTING status
received a broadcasted recover request
I0903 22:04:33.560057 25602 recover.cpp:188] Received a recover response from a
replica in STARTING status
I0903 22:04:33.561280 25584 recover.cpp:542] Updating replica status to VOTING
I0903 22:04:33.576900 25581 leveldb.cpp:306] Persisting metadata (8 bytes) to
leveldb took 14.712427ms
I0903 22:04:33.576942 25581 replica.cpp:320] Persisted replica status to VOTING
I0903 22:04:33.577018 25581 recover.cpp:556] Successfully joined the Paxos group
I0903 22:04:33.577108 25581 recover.cpp:440] Recover process terminated
I0903 22:04:33.577401 25581 log.cpp:656] Attempting to start the writer
I0903 22:04:33.578559 25589 replica.cpp:474] Replica received implicit promise
request with proposal 1
I0903 22:04:33.594611 25589 leveldb.cpp:306] Persisting metadata (8 bytes) to
leveldb took 16.029152ms
I0903 22:04:33.594640 25589 replica.cpp:342] Persisted promised to 1
I0903 22:04:33.595391 25584 coordinator.cpp:230] Coordinator attemping to fill
missing position
I0903 22:04:33.597512 25588 replica.cpp:375] Replica received explicit promise
request for position 0 with proposal 2
I0903 22:04:33.613037 25588 leveldb.cpp:343] Persisting action (8 bytes) to
leveldb took 15.502568ms
I0903 22:04:33.613065 25588 replica.cpp:676] Persisted action at 0
I0903 22:04:33.615435 25585 replica.cpp:508] Replica received write request for
position 0
I0903 22:04:33.615463 25585 leveldb.cpp:438] Reading position from leveldb took
10743ns
I0903 22:04:33.630801 25585 leveldb.cpp:343] Persisting action (14 bytes) to
leveldb took 15.320225ms
I0903 22:04:33.630852 25585 replica.cpp:676] Persisted action at 0
I0903 22:04:33.631126 25585 replica.cpp:655] Replica received learned notice
for position 0
I0903 22:04:33.647801 25585 leveldb.cpp
[jira] [Updated] (MESOS-1759) Provide a better/scalable registry of Mesos frameworks
[ https://issues.apache.org/jira/browse/MESOS-1759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Aniszczyk updated MESOS-1759: --- Summary: Provide a better/scalable registry of Mesos frameworks (was: Provide a registry of Mesos frameworks) > Provide a better/scalable registry of Mesos frameworks > -- > > Key: MESOS-1759 > URL: https://issues.apache.org/jira/browse/MESOS-1759 > Project: Mesos > Issue Type: Documentation > Components: documentation >Reporter: Chris Aniszczyk > > We should provide a better and more scalable registry for Mesos frameworks as > the list keeps growing and growing. There's at least a couple of ways this > can be done: > * have frameworks that are developed outside the Apache Mesos organization > developed in a GitHub organization (similar how to JenkinsCI runs its plugins > ecosystem). this would involve shared community ownership and more > importantly, have most of the frameworks in one place > * build a online registry where people could publish their frameworks in some > fashion and than provide a website where people can search for frameworks by > language, tool etc (think more like docker registry) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MESOS-1759) Provide a registry of Mesos frameworks
Chris Aniszczyk created MESOS-1759: -- Summary: Provide a registry of Mesos frameworks Key: MESOS-1759 URL: https://issues.apache.org/jira/browse/MESOS-1759 Project: Mesos Issue Type: Documentation Components: documentation Reporter: Chris Aniszczyk We should provide a better and more scalable registry for Mesos frameworks as the list keeps growing and growing. There's at least a couple of ways this can be done: * have frameworks that are developed outside the Apache Mesos organization developed in a GitHub organization (similar how to JenkinsCI runs its plugins ecosystem). this would involve shared community ownership and more importantly, have most of the frameworks in one place * build a online registry where people could publish their frameworks in some fashion and than provide a website where people can search for frameworks by language, tool etc (think more like docker registry) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MESOS-1758) Freezer failure leads to lost task during container destruction.
Benjamin Mahler created MESOS-1758:
--
Summary: Freezer failure leads to lost task during container
destruction.
Key: MESOS-1758
URL: https://issues.apache.org/jira/browse/MESOS-1758
Project: Mesos
Issue Type: Bug
Components: containerization
Reporter: Benjamin Mahler
In the past we've seen numerous issues around the freezer. Lately, on the
2.6.44 kernel, we've seen issues where we're unable to freeze the cgroup:
(1) An oom occurs.
(2) No indication of oom in the kernel logs.
(3) The slave is unable to freeze the cgroup.
(4) The task is marked as lost.
{noformat}
I0903 16:46:24.956040 25469 mem.cpp:575] Memory limit exceeded: Requested:
15488MB Maximum Used: 15488MB
MEMORY STATISTICS:
cache 7958691840
rss 8281653248
mapped_file 9474048
pgpgin 4487861
pgpgout 522933
pgfault 2533780
pgmajfault 11
inactive_anon 0
active_anon 8281653248
inactive_file 7631708160
active_file 326852608
unevictable 0
hierarchical_memory_limit 16240345088
total_cache 7958691840
total_rss 8281653248
total_mapped_file 9474048
total_pgpgin 4487861
total_pgpgout 522933
total_pgfault 2533780
total_pgmajfault 11
total_inactive_anon 0
total_active_anon 8281653248
total_inactive_file 7631728640
total_active_file 326852608
total_unevictable 0
I0903 16:46:24.956848 25469 containerizer.cpp:1041] Container
bbb9732a-d600-4c1b-b326-846338c608c3 has reached its limit for resource
mem(*):1.62403e+10 and will be terminated
I0903 16:46:24.957427 25469 containerizer.cpp:909] Destroying container
'bbb9732a-d600-4c1b-b326-846338c608c3'
I0903 16:46:24.958664 25481 cgroups.cpp:2192] Freezing cgroup
/sys/fs/cgroup/freezer/mesos/bbb9732a-d600-4c1b-b326-846338c608c3
I0903 16:46:34.959529 25488 cgroups.cpp:2209] Thawing cgroup
/sys/fs/cgroup/freezer/mesos/bbb9732a-d600-4c1b-b326-846338c608c3
I0903 16:46:34.962070 25482 cgroups.cpp:1404] Successfullly thawed cgroup
/sys/fs/cgroup/freezer/mesos/bbb9732a-d600-4c1b-b326-846338c608c3 after
1.710848ms
I0903 16:46:34.962658 25479 cgroups.cpp:2192] Freezing cgroup
/sys/fs/cgroup/freezer/mesos/bbb9732a-d600-4c1b-b326-846338c608c3
I0903 16:46:44.963349 25488 cgroups.cpp:2209] Thawing cgroup
/sys/fs/cgroup/freezer/mesos/bbb9732a-d600-4c1b-b326-846338c608c3
I0903 16:46:44.965631 25472 cgroups.cpp:1404] Successfullly thawed cgroup
/sys/fs/cgroup/freezer/mesos/bbb9732a-d600-4c1b-b326-846338c608c3 after
1.588224ms
I0903 16:46:44.966356 25472 cgroups.cpp:2192] Freezing cgroup
/sys/fs/cgroup/freezer/mesos/bbb9732a-d600-4c1b-b326-846338c608c3
I0903 16:46:54.967254 25488 cgroups.cpp:2209] Thawing cgroup
/sys/fs/cgroup/freezer/mesos/bbb9732a-d600-4c1b-b326-846338c608c3
I0903 16:46:56.008447 25475 cgroups.cpp:1404] Successfullly thawed cgroup
/sys/fs/cgroup/freezer/mesos/bbb9732a-d600-4c1b-b326-846338c608c3 after
2.15296ms
I0903 16:46:56.009071 25466 cgroups.cpp:2192] Freezing cgroup
/sys/fs/cgroup/freezer/mesos/bbb9732a-d600-4c1b-b326-846338c608c3
I0903 16:47:06.010329 25488 cgroups.cpp:2209] Thawing cgroup
/sys/fs/cgroup/freezer/mesos/bbb9732a-d600-4c1b-b326-846338c608c3
I0903 16:47:06.012538 25467 cgroups.cpp:1404] Successfullly thawed cgroup
/sys/fs/cgroup/freezer/mesos/bbb9732a-d600-4c1b-b326-846338c608c3 after
1.643008ms
I0903 16:47:06.013216 25467 cgroups.cpp:2192] Freezing cgroup
/sys/fs/cgroup/freezer/mesos/bbb9732a-d600-4c1b-b326-846338c608c3
I0903 16:47:12.516348 25480 slave.cpp:3030] Current usage 9.57%. Max allowed
age: 5.630238827780799days
I0903 16:47:16.015192 25488 cgroups.cpp:2209] Thawing cgroup
/sys/fs/cgroup/freezer/mesos/bbb9732a-d600-4c1b-b326-846338c608c3
I0903 16:47:16.017043 25486 cgroups.cpp:1404] Successfullly thawed cgroup
/sys/fs/cgroup/freezer/mesos/bbb9732a-d600-4c1b-b326-846338c608c3 after
1.511168ms
I0903 16:47:16.017555 25480 cgroups.cpp:2192] Freezing cgroup
/sys/fs/cgroup/freezer/mesos/bbb9732a-d600-4c1b-b326-846338c608c3
I0903 16:47:19.862746 25483 http.cpp:245] HTTP request for
'/slave(1)/stats.json'
E0903 16:47:24.960055 25472 slave.cpp:2557] Termination of executor 'E' of
framework '201104070004-002563-' failed: Failed to destroy container:
discarded future
I0903 16:47:24.962054 25472 slave.cpp:2087] Handling status update TASK_LOST
(UUID: c0c1633b-7221-40dc-90a2-660ef639f747) for task T of framework
201104070004-002563- from @0.0.0.0:0
I0903 16:47:24.963470 25469 mem.cpp:293] Updated 'memory.soft_limit_in_bytes'
to 128MB for container bbb9732a-d600-4c1b-b326-846338c608c3
I0903 16:47:24.963541 25471 cpushare.cpp:338] Updated 'cpu.shares' to 256 (cpus
0.25) for container bbb9732a-d600-4c1b-b326-846338c608c3
I0903 16:47:24.964756 25471 cpushare.cpp:359] Updated 'cpu.cfs_period_us' to
100ms and 'cpu.cfs_quota_us' to 25ms (cpus 0.25) for container
bbb9732a-d600-4c1b-b326-846338c608c3
I0903 16:47:43.406610 25476 status_update_manager.cpp:320] Received status
update TASK_LOST (UUID: c0c1633b-7221-40dc-90a2-660
[jira] [Commented] (MESOS-1757) Speed up the tests.
[ https://issues.apache.org/jira/browse/MESOS-1757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14120567#comment-14120567 ] Dominic Hamon commented on MESOS-1757: -- We can also consider using more mocks/stubs and refactoring code to enable unit tests as opposed to (or in support of) integration tests. This might reduce some of the larger test cases. > Speed up the tests. > --- > > Key: MESOS-1757 > URL: https://issues.apache.org/jira/browse/MESOS-1757 > Project: Mesos > Issue Type: Epic > Components: technical debt, test >Reporter: Benjamin Mahler > > The full test suite is exceeding the 7 minute mark (440 seconds on my > machine), this epic is to track techniques to improve this: > # The reaper takes a full second to reap an exited process (MESOS-1199), this > adds a second to each slave recovery test, and possibly more for things that > rely on Subprocess. > # The command executor sleeps for a second when shutting down (MESOS-442), > this adds a second to every test that uses the command executor. > # Now that the master and the slave have to perform sync'ed disk writes, > consider using a ramdisk to speed up the disk writes. > Additional options that hopefully will not be necessary: > # Use automake's [parallel test > harness|http://www.gnu.org/software/automake/manual/html_node/Parallel-Test-Harness.html] > to compile tests separately and run tests in parallel. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MESOS-1757) Speed up the tests.
Benjamin Mahler created MESOS-1757: -- Summary: Speed up the tests. Key: MESOS-1757 URL: https://issues.apache.org/jira/browse/MESOS-1757 Project: Mesos Issue Type: Epic Components: technical debt, test Reporter: Benjamin Mahler The full test suite is exceeding the 7 minute mark (440 seconds on my machine), this epic is to track techniques to improve this: # The reaper takes a full second to reap an exited process (MESOS-1199), this adds a second to each slave recovery test, and possibly more for things that rely on Subprocess. # The command executor sleeps for a second when shutting down (MESOS-442), this adds a second to every test that uses the command executor. # Now that the master and the slave have to perform sync'ed disk writes, consider using a ramdisk to speed up the disk writes. Additional options that hopefully will not be necessary: # Use automake's [parallel test harness|http://www.gnu.org/software/automake/manual/html_node/Parallel-Test-Harness.html] to compile tests separately and run tests in parallel. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (MESOS-733) Speedup slave recovery tests
[ https://issues.apache.org/jira/browse/MESOS-733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Mahler closed MESOS-733. - Resolution: Incomplete Closing this in favor of an epic to track testing speedups. > Speedup slave recovery tests > > > Key: MESOS-733 > URL: https://issues.apache.org/jira/browse/MESOS-733 > Project: Mesos > Issue Type: Bug >Reporter: Vinod Kone > Labels: twitter > > Several of the tests are slow now that they do offer checking. I suspect this > is due to the "Clock" semantics. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1754) Mesos static library has undefined symbols from 3rd party deps
[
https://issues.apache.org/jira/browse/MESOS-1754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14120501#comment-14120501
]
Timothy St. Clair commented on MESOS-1754:
--
It looks like the culprit is around the whole-archive link option:
--whole-archive
For each archive mentioned on the command line after the
--whole-archive option, include every object
file in the archive in the link, rather than searching the archive
for the required object files. This
is normally used to turn an archive file into a shared library,
forcing every object to be included in
the resulting shared library. This option may be used more than
once.
Two notes when using this option from gcc: First, gcc doesn't know
about this option, so you have to use
-Wl,-whole-archive. Second, don't forget to use
-Wl,-no-whole-archive after your list of archives,
because gcc will add its own list of archives to your link and you
may not want this flag to affect
those as well.
more specifically:
Wl,--no-whole-archive
../3rdparty/libprocess/3rdparty/glog-0.3.3/.libs/libglog.a -L/usr/lib/../lib64
../3rdparty/leveldb/libleveldb.a
../3rdparty/zookeeper-3.4.5/src/c/.libs/libzookeeper_mt.a
> Mesos static library has undefined symbols from 3rd party deps
> --
>
> Key: MESOS-1754
> URL: https://issues.apache.org/jira/browse/MESOS-1754
> Project: Mesos
> Issue Type: Bug
>Affects Versions: 0.20.0
>Reporter: Vinod Kone
>Assignee: Vinod Kone
>
> Looks like the static libmesos library that we build is missing some symbols
> from 3rd party deps (e.g, GLOG, ZooKeeper). I suspect this has to do with the
> order of libraries defined on the linker command line.
> {code}
> zookeeper_init symbol is present in .so but not in .a.
> $ nm libmesos.so | grep zookeeper_init
> 030519f0 T zookeeper_init
> $ nm libmesos.a | grep zookeeper_init
> nm: libleveldb.a: File format not recognized
> U zookeeper_init
> Same with google::InstallFailureSignalHandler.
> $ nm libmesos.so | grep InstallFailureSignalHandler
> 0301901d T _ZN6google27InstallFailureSignalHandlerEv
> $ nm libmesos.a | grep InstallFailureSignalHandler
> nm: libleveldb.a: File format not recognized
> U _ZN6google27InstallFailureSignalHandlerEv
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-1740) Bad error message when docker containerizer isn't enabled
[
https://issues.apache.org/jira/browse/MESOS-1740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Timothy Chen updated MESOS-1740:
Target Version/s: 0.20.1
Fix Version/s: (was: 0.20.1)
> Bad error message when docker containerizer isn't enabled
> -
>
> Key: MESOS-1740
> URL: https://issues.apache.org/jira/browse/MESOS-1740
> Project: Mesos
> Issue Type: Bug
> Components: containerization
>Reporter: Jay Buffington
>Assignee: Timothy Chen
>Priority: Minor
> Labels: docker
>
> If I set container in TaskInfo's executor (aka DockerInfo) but I do not start
> the slave with {{--containerizer=docker,...}} then I get this error message
> in the log:
> {noformat}
> E0827 17:53:16.422735 20090 slave.cpp:2491] Container 'xxx' for executor
> 'yyy' of framework 'zzz' failed to start: TaskInfo/ExecutorInfo not supported
> {noformat}
> A better error message would have been:
> {noformat}
> No enabled containerizers could create a container for the provided
> TaskInfo/ExecutorInfo message. Enabled containerizers are: mesos.
> {noformat}
> An even better error message would have been:
> {noformat}
> DockerInfo was sent, but docker containerizer is not enabled. Try adding
> --containerizer=docker,... to command line args
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-1740) Bad error message when docker containerizer isn't enabled
[
https://issues.apache.org/jira/browse/MESOS-1740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Timothy Chen updated MESOS-1740:
Labels: docker (was: )
> Bad error message when docker containerizer isn't enabled
> -
>
> Key: MESOS-1740
> URL: https://issues.apache.org/jira/browse/MESOS-1740
> Project: Mesos
> Issue Type: Bug
> Components: containerization
>Reporter: Jay Buffington
>Assignee: Timothy Chen
>Priority: Minor
> Labels: docker
> Fix For: 0.20.1
>
>
> If I set container in TaskInfo's executor (aka DockerInfo) but I do not start
> the slave with {{--containerizer=docker,...}} then I get this error message
> in the log:
> {noformat}
> E0827 17:53:16.422735 20090 slave.cpp:2491] Container 'xxx' for executor
> 'yyy' of framework 'zzz' failed to start: TaskInfo/ExecutorInfo not supported
> {noformat}
> A better error message would have been:
> {noformat}
> No enabled containerizers could create a container for the provided
> TaskInfo/ExecutorInfo message. Enabled containerizers are: mesos.
> {noformat}
> An even better error message would have been:
> {noformat}
> DockerInfo was sent, but docker containerizer is not enabled. Try adding
> --containerizer=docker,... to command line args
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-1740) Bad error message when docker containerizer isn't enabled
[
https://issues.apache.org/jira/browse/MESOS-1740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Timothy Chen updated MESOS-1740:
Fix Version/s: 0.20.1
> Bad error message when docker containerizer isn't enabled
> -
>
> Key: MESOS-1740
> URL: https://issues.apache.org/jira/browse/MESOS-1740
> Project: Mesos
> Issue Type: Bug
> Components: containerization
>Reporter: Jay Buffington
>Assignee: Timothy Chen
>Priority: Minor
> Fix For: 0.20.1
>
>
> If I set container in TaskInfo's executor (aka DockerInfo) but I do not start
> the slave with {{--containerizer=docker,...}} then I get this error message
> in the log:
> {noformat}
> E0827 17:53:16.422735 20090 slave.cpp:2491] Container 'xxx' for executor
> 'yyy' of framework 'zzz' failed to start: TaskInfo/ExecutorInfo not supported
> {noformat}
> A better error message would have been:
> {noformat}
> No enabled containerizers could create a container for the provided
> TaskInfo/ExecutorInfo message. Enabled containerizers are: mesos.
> {noformat}
> An even better error message would have been:
> {noformat}
> DockerInfo was sent, but docker containerizer is not enabled. Try adding
> --containerizer=docker,... to command line args
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (MESOS-1754) Mesos static library has undefined symbols from 3rd party deps
[
https://issues.apache.org/jira/browse/MESOS-1754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14118952#comment-14118952
]
Vinod Kone edited comment on MESOS-1754 at 9/3/14 8:49 PM:
---
Looks like some of the 3rd party deps are not included in the static archive
during libtool link step. Not sure if we are missing a flag or something. cc.
[~tstclair]
libtool command that gets eventually run
{code}
/bin/sh ../libtool --tag=CC --tag=CXX --mode=link gcc -g
-Wno-unused-local-typedefs -release 0.21.0 -o libmesos.la -rpath
/usr/local/liblibmesos_no_3rdparty.la
../3rdparty/libprocess/3rdparty/glog-0.3.3/libglog.la
../3rdparty/leveldb/libleveldb.a
../3rdparty/zookeeper-3.4.5/src/c/libzookeeper_mt.la
../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src/libprotobuf.la
../3rdparty/libprocess/libprocess.la libjava.la -lsasl2 -lcurl -lz -lrt
-lunwind
libtool: link: warning: `/usr/lib/../lib64/libunwind.la' seems to be moved
*** Warning: Linking the shared library libmesos.la against the
*** static library ../3rdparty/leveldb/libleveldb.a is not portable!
libtool: link: warning: `/usr/lib/../lib64/libunwind.la' seems to be moved
{code}
shared library includes libglog.a and libzookeeper_mt.a for example.
{code}
libtool: link: g++ -shared -nostdlib /usr/lib/../lib64/crti.o
/opt/rh/devtoolset-2/root/usr/lib/gcc/x86_64-redhat-linux/4.8.1/crtbeginS.o
-Wl,--whole-archive ./.libs/libmesos_no_3rdparty.a
../3rdparty/libprocess/.libs/libprocess.a ./.libs/libjava.a
-Wl,--no-whole-archive
../3rdparty/libprocess/3rdparty/glog-0.3.3/.libs/libglog.a -L/usr/lib/../lib64
../3rdparty/leveldb/libleveldb.a
../3rdparty/zookeeper-3.4.5/src/c/.libs/libzookeeper_mt.a
../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src/.libs/libprotobuf.a
/home/vinod/mesos/build/3rdparty/libprocess/3rdparty/glog-0.3.3/.libs/libglog.a
-lpthread
/home/vinod/mesos/build/3rdparty/libprocess/3rdparty/libev-4.15/.libs/libev.a
-lsasl2 -lcurl -lz -lrt /usr/lib64/libunwind.so -lgcc
-L/opt/rh/devtoolset-2/root/usr/lib/gcc/x86_64-redhat-linux/4.8.1
-L/opt/rh/devtoolset-2/root/usr/lib/gcc/x86_64-redhat-linux/4.8.1/../../../../lib64
-L/lib/../lib64
-L/opt/rh/devtoolset-2/root/usr/lib/gcc/x86_64-redhat-linux/4.8.1/../../..
-lstdc++ -lm -lc -lgcc_s
/opt/rh/devtoolset-2/root/usr/lib/gcc/x86_64-redhat-linux/4.8.1/crtendS.o
/usr/lib/../lib64/crtn.o-pthread -Wl,-soname -Wl,libmesos-0.21.0.so -o
.libs/libmesos-0.21.0.so
libtool: link: (cd ".libs" && rm -f "libmesos.so" && ln -s "libmesos-0.21.0.so"
"libmesos.so")
{code}
static archive doesn't include object files from libglog.a or libzookeeper_mt.a
but includes object files from libmesos_no_3rdpart.a, libprocess.a and
libjava.a.
{code}
libtool: link: (cd .libs/libmesos.lax/libmesos_no_3rdparty.a && ar x
"/home/vinod/mesos/build/src/./.libs/libmesos_no_3rdparty.a")
libtool: link: (cd .libs/libmesos.lax/libprocess.a && ar x
"/home/vinod/mesos/build/src/../3rdparty/libprocess/.libs/libprocess.a")
libtool: link: (cd .libs/libmesos.lax/libjava.a && ar x
"/home/vinod/mesos/build/src/./.libs/libjava.a")
libtool: link: ar cru .libs/libmesos.a ../3rdparty/leveldb/libleveldb.a
.libs/libmesos.lax/libmesos_no_3rdparty.a/libmesos_no_3rdparty_la-state.o
.libs/libmesos.lax/libmesos_no_3rdparty.a/libmesos_no_3rdparty_la-docker.o
.libs/libmesos.lax/libmesos_no_3rdparty.a/libstate_la-zookeeper.o
.libs/libmesos.lax/libmesos_no_3rdparty.a/libmesos_no_3rdparty_la-group.o
.libs/libmesos.lax/libmesos_no_3rdparty.a/libmesos_no_3rdparty_la-detector.o
.libs/libmesos.lax/libmesos_no_3rdparty.a/libmesos_no_3rdparty_la-registrar.o
.libs/libmesos.lax/libmesos_no_3rdparty.a/libmesos_no_3rdparty_la-http.o
.libs/libmesos.lax/libmesos_no_3rdparty.a/libmesos_no_3rdparty_la-auxprop.o
.libs/libmesos.lax/libmesos_no_3rdparty.a/libmesos_no_3rdparty_la-gc.o
.libs/libmesos.lax/libmesos_no_3rdparty.a/liblog_la-recover.o
.libs/libmesos.lax/libmesos_no_3rdparty.a/libmesos_no_3rdparty_la-type_utils.o
.libs/libmesos.lax/libmesos_no_3rdparty.a/libmesos_no_3rdparty_la-repairer.o
.libs/libmesos.lax/libmesos_no_3rdparty.a/libmesos_no_3rdparty_la-scheduler.pb.o
.libs/libmesos.lax/libmesos_no_3rdparty.a/libmesos_no_3rdparty_la-slave.o
.libs/libmesos.lax/libmesos_no_3rdparty.a/lt2-libmesos_no_3rdparty_la-constants.o
.libs/libmesos.lax/libmesos_no_3rdparty.a/libmesos_no_3rdparty_la-files.o
.libs/libmesos.lax/libmesos_no_3rdparty.a/libmesos_no_3rdparty_la-usage.o
.libs/libmesos.lax/libmesos_no_3rdparty.a/libmesos_no_3rdparty_la-launch.o
.libs/libmesos.lax/libmesos_no_3rdparty.a/libmesos_no_3rdparty_la-authorizer.o
.libs/libmesos.lax/libmesos_no_3rdparty.a/libmesos_no_3rdparty_la-sched.o
.libs/libmesos.lax/libmesos_no_3rdparty.a/liblog_la-consensus.o
.libs/libmesos.lax/libmesos_no_3rdparty.a/libstate_la-log.o
.libs/libmesos.lax/libmesos_no_3rdparty.a/libmesos_no_3rdparty_la-mem.o
.libs/libmesos.l
[jira] [Comment Edited] (MESOS-1754) Mesos static library has undefined symbols from 3rd party deps
[
https://issues.apache.org/jira/browse/MESOS-1754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14118952#comment-14118952
]
Vinod Kone edited comment on MESOS-1754 at 9/3/14 8:45 PM:
---
Looks like some of the 3rd party deps are not included in the static archive
during libtool link step. Not sure if we are missing a flag or something. cc.
[~tstclair]
shared library includes glog and zookeeper for example.
{code}
libtool: link: g++ -shared -nostdlib /usr/lib/../lib64/crti.o
/opt/rh/devtoolset-2/root/usr/lib/gcc/x86_64-redhat-linux/4.8.1/crtbeginS.o
-Wl,--whole-archive ./.libs/libmesos_no_3rdparty.a
../3rdparty/libprocess/.libs/libprocess.a ./.libs/libjava.a
-Wl,--no-whole-archive
../3rdparty/libprocess/3rdparty/glog-0.3.3/.libs/libglog.a -L/usr/lib/../lib64
../3rdparty/leveldb/libleveldb.a
../3rdparty/zookeeper-3.4.5/src/c/.libs/libzookeeper_mt.a
../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src/.libs/libprotobuf.a
/home/vinod/mesos/build/3rdparty/libprocess/3rdparty/glog-0.3.3/.libs/libglog.a
-lpthread
/home/vinod/mesos/build/3rdparty/libprocess/3rdparty/libev-4.15/.libs/libev.a
-lsasl2 -lcurl -lz -lrt /usr/lib64/libunwind.so -lgcc
-L/opt/rh/devtoolset-2/root/usr/lib/gcc/x86_64-redhat-linux/4.8.1
-L/opt/rh/devtoolset-2/root/usr/lib/gcc/x86_64-redhat-linux/4.8.1/../../../../lib64
-L/lib/../lib64
-L/opt/rh/devtoolset-2/root/usr/lib/gcc/x86_64-redhat-linux/4.8.1/../../..
-lstdc++ -lm -lc -lgcc_s
/opt/rh/devtoolset-2/root/usr/lib/gcc/x86_64-redhat-linux/4.8.1/crtendS.o
/usr/lib/../lib64/crtn.o-pthread -Wl,-soname -Wl,libmesos-0.21.0.so -o
.libs/libmesos-0.21.0.so
libtool: link: (cd ".libs" && rm -f "libmesos.so" && ln -s "libmesos-0.21.0.so"
"libmesos.so")
{code}
static archive doesn't include glog or zookeeper for example.
{code}
libtool: link: (cd .libs/libmesos.lax/libmesos_no_3rdparty.a && ar x
"/home/vinod/mesos/build/src/./.libs/libmesos_no_3rdparty.a")
libtool: link: (cd .libs/libmesos.lax/libprocess.a && ar x
"/home/vinod/mesos/build/src/../3rdparty/libprocess/.libs/libprocess.a")
libtool: link: (cd .libs/libmesos.lax/libjava.a && ar x
"/home/vinod/mesos/build/src/./.libs/libjava.a")
libtool: link: ar cru .libs/libmesos.a ../3rdparty/leveldb/libleveldb.a
.libs/libmesos.lax/libmesos_no_3rdparty.a/libmesos_no_3rdparty_la-state.o
.libs/libmesos.lax/libmesos_no_3rdparty.a/libmesos_no_3rdparty_la-docker.o
.libs/libmesos.lax/libmesos_no_3rdparty.a/libstate_la-zookeeper.o
.libs/libmesos.lax/libmesos_no_3rdparty.a/libmesos_no_3rdparty_la-group.o
.libs/libmesos.lax/libmesos_no_3rdparty.a/libmesos_no_3rdparty_la-detector.o
.libs/libmesos.lax/libmesos_no_3rdparty.a/libmesos_no_3rdparty_la-registrar.o
.libs/libmesos.lax/libmesos_no_3rdparty.a/libmesos_no_3rdparty_la-http.o
.libs/libmesos.lax/libmesos_no_3rdparty.a/libmesos_no_3rdparty_la-auxprop.o
.libs/libmesos.lax/libmesos_no_3rdparty.a/libmesos_no_3rdparty_la-gc.o
.libs/libmesos.lax/libmesos_no_3rdparty.a/liblog_la-recover.o
.libs/libmesos.lax/libmesos_no_3rdparty.a/libmesos_no_3rdparty_la-type_utils.o
.libs/libmesos.lax/libmesos_no_3rdparty.a/libmesos_no_3rdparty_la-repairer.o
.libs/libmesos.lax/libmesos_no_3rdparty.a/libmesos_no_3rdparty_la-scheduler.pb.o
.libs/libmesos.lax/libmesos_no_3rdparty.a/libmesos_no_3rdparty_la-slave.o
.libs/libmesos.lax/libmesos_no_3rdparty.a/lt2-libmesos_no_3rdparty_la-constants.o
.libs/libmesos.lax/libmesos_no_3rdparty.a/libmesos_no_3rdparty_la-files.o
.libs/libmesos.lax/libmesos_no_3rdparty.a/libmesos_no_3rdparty_la-usage.o
.libs/libmesos.lax/libmesos_no_3rdparty.a/libmesos_no_3rdparty_la-launch.o
.libs/libmesos.lax/libmesos_no_3rdparty.a/libmesos_no_3rdparty_la-authorizer.o
.libs/libmesos.lax/libmesos_no_3rdparty.a/libmesos_no_3rdparty_la-sched.o
.libs/libmesos.lax/libmesos_no_3rdparty.a/liblog_la-consensus.o
.libs/libmesos.lax/libmesos_no_3rdparty.a/libstate_la-log.o
.libs/libmesos.lax/libmesos_no_3rdparty.a/libmesos_no_3rdparty_la-mem.o
.libs/libmesos.lax/libmesos_no_3rdparty.a/liblog_la-coordinator.o
.libs/libmesos.lax/libmesos_no_3rdparty.a/lt5-libmesos_no_3rdparty_la-containerizer.o
.libs/libmesos.lax/libmesos_no_3rdparty.a/libmesos_no_3rdparty_la-launcher.o
.libs/libmesos.lax/libmesos_no_3rdparty.a/libmesos_no_3rdparty_la-logging.o
.libs/libmesos.lax/libmesos_no_3rdparty.a/liblog_la-initialize.o
.libs/libmesos.lax/libmesos_no_3rdparty.a/libmesos_no_3rdparty_la-isolator.o
.libs/libmesos.lax/libmesos_no_3rdparty.a/libstate_la-leveldb.o
.libs/libmesos.lax/libmesos_no_3rdparty.a/libbuild_la-build.o
.libs/libmesos.lax/libmesos_no_3rdparty.a/libmesos_no_3rdparty_la-values.o
.libs/libmesos.lax/libmesos_no_3rdparty.a/liblog_la-catchup.o
.libs/libmesos.lax/libmesos_no_3rdparty.a/libmesos_no_3rdparty_la-resources.o
.libs/libmesos.lax/libmesos_no_3rdparty.a/liblog_la-read.o
.libs/libmesos.lax/libmesos_no_3rdparty.a/libmesos_no_3rdparty_la-composing.o
.
[jira] [Commented] (MESOS-1754) Mesos static library has undefined symbols from 3rd party deps
[
https://issues.apache.org/jira/browse/MESOS-1754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14120355#comment-14120355
]
Timothy St. Clair commented on MESOS-1754:
--
For static libraries 'ar' is called to smush the .o files into a .a. Typically
there is no link step / symbol resolution for .a files (only .so and
executables) so there is no merging of .a files by default.
A versioned .so forces linkage, hence it will force symbol resolution.
> Mesos static library has undefined symbols from 3rd party deps
> --
>
> Key: MESOS-1754
> URL: https://issues.apache.org/jira/browse/MESOS-1754
> Project: Mesos
> Issue Type: Bug
>Affects Versions: 0.20.0
>Reporter: Vinod Kone
>Assignee: Vinod Kone
>
> Looks like the static libmesos library that we build is missing some symbols
> from 3rd party deps (e.g, GLOG, ZooKeeper). I suspect this has to do with the
> order of libraries defined on the linker command line.
> {code}
> zookeeper_init symbol is present in .so but not in .a.
> $ nm libmesos.so | grep zookeeper_init
> 030519f0 T zookeeper_init
> $ nm libmesos.a | grep zookeeper_init
> nm: libleveldb.a: File format not recognized
> U zookeeper_init
> Same with google::InstallFailureSignalHandler.
> $ nm libmesos.so | grep InstallFailureSignalHandler
> 0301901d T _ZN6google27InstallFailureSignalHandlerEv
> $ nm libmesos.a | grep InstallFailureSignalHandler
> nm: libleveldb.a: File format not recognized
> U _ZN6google27InstallFailureSignalHandlerEv
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (MESOS-1750) ImportError in containerizer_pb2.py
[
https://issues.apache.org/jira/browse/MESOS-1750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14120175#comment-14120175
]
Till Toenshoff edited comment on MESOS-1750 at 9/3/14 7:13 PM:
---
So the following does the job...
{noformat}
python/interface/src/mesos/interface/containerizer_pb2.py:
\
$(CONTAINERIZER_PROTO)
$(MKDIR_P) $(@D)
$(PROTOC) -I$(top_srcdir)/include/mesos/containerizer \
$(PROTOCFLAGS)
\
--python_out=python/interface/src/mesos/interface $^
sed -i '' -e 's/mesos\.mesos_pb2/mesos_pb2/' $^
{noformat}
However, I still feel that this is not really *correct* and that we should
possibly take a step back for finding a proper solution.
Another solution would be adapting the {{mesos/interface}} structure towards a
non flat hierachy:
{noformat}
mesos/mesos_pb2.py
mesos/containerizer/containerizer_pb2.py
{noformat}
Resulting into rather verbose looking python imports like:
{noformat}
from mesos.interface.mesos.containerizer import containerizer_pb2
from mesos.interface.mesos import mesos_pb2
{noformat}
The above maintains the exact namespace structure we are using for all other
bindings (but adds {{mesos.interface}} as a prefix).
I do not see any other options right now and I really would like to ask for
comments on this.
was (Author: tillt):
So the following does the job...
{noformat}
python/interface/src/mesos/interface/containerizer_pb2.py:
\
$(CONTAINERIZER_PROTO)
$(MKDIR_P) $(@D)
$(PROTOC) -I$(top_srcdir)/include/mesos/containerizer \
$(PROTOCFLAGS)
\
--python_out=python/interface/src/mesos/interface $^
sed -i -e 's/mesos\.mesos_pb2/mesos_pb2/' $^
{noformat}
However, I still feel that this is not really *correct* and that we should
possibly take a step back for finding a proper solution.
Another solution would be adapting the {{mesos/interface}} structure towards a
non flat hierachy:
{noformat}
mesos/mesos_pb2.py
mesos/containerizer/containerizer_pb2.py
{noformat}
Resulting into rather verbose looking python imports like:
{noformat}
from mesos.interface.mesos.containerizer import containerizer_pb2
from mesos.interface.mesos import mesos_pb2
{noformat}
The above maintains the exact namespace structure we are using for all other
bindings (but adds {{mesos.interface}} as a prefix).
I do not see any other options right now and I really would like to ask for
comments on this.
> ImportError in containerizer_pb2.py
> ---
>
> Key: MESOS-1750
> URL: https://issues.apache.org/jira/browse/MESOS-1750
> Project: Mesos
> Issue Type: Bug
> Components: python api
>Reporter: Till Toenshoff
> Labels: mesos.interface, python-egg
>
> It appears as if the split of the Python egg (native/interface/protocol) has
> introduced a new issue.
> Whenever I am trying to use {{from mesos.interface import
> containerizer_pb2}}, I am receiving an {{ImportError: No module named
> mesos_pb2}}.
> For replicating this issue, try running
> {{"build/src/examples/python/test-containerizer"}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-1754) Mesos static library has undefined symbols from 3rd party deps
[
https://issues.apache.org/jira/browse/MESOS-1754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14120278#comment-14120278
]
Vinod Kone commented on MESOS-1754:
---
Hi [~tstclair], not sure how versioned .so solves this problem?
Shouldn't libtool/automake merge the archives correctly when we do *_LIBADD ?
It seems to do the right thing for .so but not for .a. IIUC, the linker
will/should copy all the required/referenced symbols from the linked archive
even if it doesn't pull in all the symbols in the archive? Maybe it has to do
with the fact that we have an intermediate non-installable convenience library
(libmesos_no_3rdparty) or the fact that we have installable 3rd party libraries
(libglog.a etc) ?
> Mesos static library has undefined symbols from 3rd party deps
> --
>
> Key: MESOS-1754
> URL: https://issues.apache.org/jira/browse/MESOS-1754
> Project: Mesos
> Issue Type: Bug
>Affects Versions: 0.20.0
>Reporter: Vinod Kone
>Assignee: Vinod Kone
>
> Looks like the static libmesos library that we build is missing some symbols
> from 3rd party deps (e.g, GLOG, ZooKeeper). I suspect this has to do with the
> order of libraries defined on the linker command line.
> {code}
> zookeeper_init symbol is present in .so but not in .a.
> $ nm libmesos.so | grep zookeeper_init
> 030519f0 T zookeeper_init
> $ nm libmesos.a | grep zookeeper_init
> nm: libleveldb.a: File format not recognized
> U zookeeper_init
> Same with google::InstallFailureSignalHandler.
> $ nm libmesos.so | grep InstallFailureSignalHandler
> 0301901d T _ZN6google27InstallFailureSignalHandlerEv
> $ nm libmesos.a | grep InstallFailureSignalHandler
> nm: libleveldb.a: File format not recognized
> U _ZN6google27InstallFailureSignalHandlerEv
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (MESOS-850) Please delete old releases from mirroring system
[ https://issues.apache.org/jira/browse/MESOS-850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kone resolved MESOS-850. -- Resolution: Fixed Deleted 0.19.0 and 0.19.1 from dist/releases. Thanks for the ping [~s...@apache.org]. > Please delete old releases from mirroring system > > > Key: MESOS-850 > URL: https://issues.apache.org/jira/browse/MESOS-850 > Project: Mesos > Issue Type: Bug > Environment: http://www.apache.org/dist/mesos/ > http://mesos.apache.org/downloads/ >Reporter: Sebb >Assignee: Vinod Kone > > To reduce the load on the ASF mirrors, projects are required to delete old > releases [1] > Please can you remove all non-current releases? > Thanks! > [Note that older releases are always available from the ASF archive server] > [1] http://www.apache.org/dev/release.html#when-to-archive -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1750) ImportError in containerizer_pb2.py
[
https://issues.apache.org/jira/browse/MESOS-1750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14120175#comment-14120175
]
Till Toenshoff commented on MESOS-1750:
---
So the following does the job...
{noformat}
python/interface/src/mesos/interface/containerizer_pb2.py:
\
$(CONTAINERIZER_PROTO)
$(MKDIR_P) $(@D)
$(PROTOC) -I$(top_srcdir)/include/mesos/containerizer \
$(PROTOCFLAGS)
\
--python_out=python/interface/src/mesos/interface $^
sed -i -e 's/mesos\.mesos_pb2/mesos_pb2/' $^
{noformat}
However, I still feel that this is not really *correct* and that we should
possibly take a step back for finding a proper solution.
Another solution would be adapting the {{mesos/interface}} structure towards a
non flat hierachy:
{noformat}
mesos/mesos_pb2.py
mesos/containerizer/containerizer_pb2.py
{noformat}
Resulting into rather verbose looking python imports like:
{noformat}
from mesos.interface.mesos.containerizer import containerizer_pb2
from mesos.interface.mesos import mesos_pb2
{noformat}
The above maintains the exact namespace structure we are using for all other
bindings (but adds {{mesos.interface}} as a prefix).
I do not see any other options right now and I really would like to ask for
comments on this.
> ImportError in containerizer_pb2.py
> ---
>
> Key: MESOS-1750
> URL: https://issues.apache.org/jira/browse/MESOS-1750
> Project: Mesos
> Issue Type: Bug
> Components: python api
>Reporter: Till Toenshoff
> Labels: mesos.interface, python-egg
>
> It appears as if the split of the Python egg (native/interface/protocol) has
> introduced a new issue.
> Whenever I am trying to use {{from mesos.interface import
> containerizer_pb2}}, I am receiving an {{ImportError: No module named
> mesos_pb2}}.
> For replicating this issue, try running
> {{"build/src/examples/python/test-containerizer"}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (MESOS-1698) make check segfaults
[
https://issues.apache.org/jira/browse/MESOS-1698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vinod Kone reassigned MESOS-1698:
-
Assignee: Jie Yu (was: Vinod Kone)
assigning to [~jieyu] as he kindly offered to help out.
> make check segfaults
>
>
> Key: MESOS-1698
> URL: https://issues.apache.org/jira/browse/MESOS-1698
> Project: Mesos
> Issue Type: Bug
> Components: test
> Environment: Ubuntu
>Reporter: Vinod Kone
>Assignee: Jie Yu
>
> Observed this on Apache CI:
> https://builds.apache.org/job/Mesos-Trunk-Ubuntu-Build-Out-Of-Src-Set-JAVA_HOME/2331/consoleFull
> It looks like the segfault happens before any tests are run. So I suspect
> somewhere in the setup phase of the tests.
> {code}
> mv -f .deps/tests-time_tests.Tpo .deps/tests-time_tests.Po
> /bin/bash ./libtool --tag=CXX --mode=link g++ -g -g2 -O2
> -Wno-unused-local-typedefs -std=c++11 -o tests tests-decoder_tests.o
> tests-encoder_tests.o tests-http_tests.o tests-io_tests.o tests-main.o
> tests-mutex_tests.o tests-metrics_tests.o tests-owned_tests.o
> tests-process_tests.o tests-queue_tests.o tests-reap_tests.o
> tests-sequence_tests.o tests-shared_tests.o tests-statistics_tests.o
> tests-subprocess_tests.o tests-system_tests.o tests-timeseries_tests.o
> tests-time_tests.o 3rdparty/libgmock.la libprocess.la
> 3rdparty/glog-0.3.3/libglog.la 3rdparty/libry_http_parser.la
> 3rdparty/libev-4.15/libev.la -lz -lrt
> libtool: link: g++ -g -g2 -O2 -Wno-unused-local-typedefs -std=c++11 -o tests
> tests-decoder_tests.o tests-encoder_tests.o tests-http_tests.o
> tests-io_tests.o tests-main.o tests-mutex_tests.o tests-metrics_tests.o
> tests-owned_tests.o tests-process_tests.o tests-queue_tests.o
> tests-reap_tests.o tests-sequence_tests.o tests-shared_tests.o
> tests-statistics_tests.o tests-subprocess_tests.o tests-system_tests.o
> tests-timeseries_tests.o tests-time_tests.o 3rdparty/.libs/libgmock.a
> ./.libs/libprocess.a
> /home/jenkins/jenkins-slave/workspace/Mesos-Trunk-Ubuntu-Build-Out-Of-Src-Set-JAVA_HOME/build/3rdparty/libprocess/3rdparty/glog-0.3.3/.libs/libglog.a
>
> /home/jenkins/jenkins-slave/workspace/Mesos-Trunk-Ubuntu-Build-Out-Of-Src-Set-JAVA_HOME/build/3rdparty/libprocess/3rdparty/libev-4.15/.libs/libev.a
> 3rdparty/glog-0.3.3/.libs/libglog.a -lpthread
> 3rdparty/.libs/libry_http_parser.a 3rdparty/libev-4.15/.libs/libev.a -lm -lz
> -lrt
> make[5]: Leaving directory
> `/home/jenkins/jenkins-slave/workspace/Mesos-Trunk-Ubuntu-Build-Out-Of-Src-Set-JAVA_HOME/build/3rdparty/libprocess'
> make check-local
> make[5]: Entering directory
> `/home/jenkins/jenkins-slave/workspace/Mesos-Trunk-Ubuntu-Build-Out-Of-Src-Set-JAVA_HOME/build/3rdparty/libprocess'
> ./tests
> Note: Google Test filter =
> [==] Running 0 tests from 0 test cases.
> [==] 0 tests from 0 test cases ran. (0 ms total)
> [ PASSED ] 0 tests.
> YOU HAVE 3 DISABLED TESTS
> make[5]: *** [check-local] Segmentation fault
> make[5]: Leaving directory
> `/home/jenkins/jenkins-slave/workspace/Mesos-Trunk-Ubuntu-Build-Out-Of-Src-Set-JAVA_HOME/build/3rdparty/libprocess'
> make[4]: *** [check-am] Error 2
> make[4]: Leaving directory
> `/home/jenkins/jenkins-slave/workspace/Mesos-Trunk-Ubuntu-Build-Out-Of-Src-Set-JAVA_HOME/build/3rdparty/libprocess'
> make[3]: *** [check-recursive] Error 1
> make[3]: Leaving directory
> `/home/jenkins/jenkins-slave/workspace/Mesos-Trunk-Ubuntu-Build-Out-Of-Src-Set-JAVA_HOME/build/3rdparty/libprocess'
> make[2]: *** [check-recursive] Error 1
> make[2]: Leaving directory
> `/home/jenkins/jenkins-slave/workspace/Mesos-Trunk-Ubuntu-Build-Out-Of-Src-Set-JAVA_HOME/build/3rdparty'
> make[1]: *** [check] Error 2
> make[1]: Leaving directory
> `/home/jenkins/jenkins-slave/workspace/Mesos-Trunk-Ubuntu-Build-Out-Of-Src-Set-JAVA_HOME/build/3rdparty'
> make: *** [check-recursive] Error 1
> Build step 'Execute shell' marked build as failure
> Sending e-mails to: d...@mesos.apache.org benjamin.hind...@gmail.com
> dha...@twitter.com yujie@gmail.com
> Finished: FAILURE
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-1714) The C++ 'Resources' abstraction should keep the underlying resources flattened.
[ https://issues.apache.org/jira/browse/MESOS-1714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14120138#comment-14120138 ] Benjamin Mahler commented on MESOS-1714: For now, this review avoids constructing an unflattened Resources object: https://reviews.apache.org/r/25306/ > The C++ 'Resources' abstraction should keep the underlying resources > flattened. > --- > > Key: MESOS-1714 > URL: https://issues.apache.org/jira/browse/MESOS-1714 > Project: Mesos > Issue Type: Bug > Components: c++ api >Reporter: Benjamin Mahler > > Currently, the C++ Resources class does not ensure that the underlying > Resources protobufs are kept flat. > This is an issue because some of the methods, e.g. > [Resources::get|https://github.com/apache/mesos/blob/0.19.1/src/common/resources.cpp#L269], > assume the resources are flat. > There is code that constructs unflattened resources, e.g. > [Slave::launchExecutor|https://github.com/apache/mesos/blob/0.19.1/src/slave/slave.cpp#L3353]. > We could prevent this type of construction, however it is perfectly fine if > we ensure the C++ 'Resources' class performs flattening. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1716) The slave does not add pending tasks as part of the staging tasks metric.
[ https://issues.apache.org/jira/browse/MESOS-1716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14120110#comment-14120110 ] Benjamin Mahler commented on MESOS-1716: https://reviews.apache.org/r/25302/ https://reviews.apache.org/r/25303/ > The slave does not add pending tasks as part of the staging tasks metric. > - > > Key: MESOS-1716 > URL: https://issues.apache.org/jira/browse/MESOS-1716 > Project: Mesos > Issue Type: Bug > Components: slave >Reporter: Benjamin Mahler >Assignee: Benjamin Mahler >Priority: Trivial > > The slave does not represent pending tasks in the "tasks_staging" metric. > This should be a trivial fix. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (MESOS-1007) Python framework unable to parse framework messages
[ https://issues.apache.org/jira/browse/MESOS-1007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kone resolved MESOS-1007. --- Resolution: Cannot Reproduce Haven't seen this in a while. So closing this as "cannot reproduce". > Python framework unable to parse framework messages > --- > > Key: MESOS-1007 > URL: https://issues.apache.org/jira/browse/MESOS-1007 > Project: Mesos > Issue Type: Bug > Environment: Fedora 20. Cxx11 >Reporter: Vinod Kone > > [ RUN ] ExamplesTest.PythonFramework > Using temporary directory '/tmp/ExamplesTest_PythonFramework_bk0KRb' > WARNING: Logging before InitGoogleLogging() is written to STDERR > I0213 22:15:39.954318 17373 process.cpp:1591] libprocess is initialized on > 192.168.122.164:47130 for 8 cpus > I0213 22:15:39.955761 17373 logging.cpp:140] Logging to STDERR > I0213 22:15:39.959444 17373 master.cpp:240] Master ID: > 2014-02-13-22:15:39-2759502016-47130-17373 Hostname: fedora-20 > I0213 22:15:39.960225 17383 master.cpp:322] Master started on > 192.168.122.164:47130 > I0213 22:15:39.960244 17383 master.cpp:325] Master only allowing > authenticated frameworks to register! > I0213 22:15:39.961699 17387 master.cpp:86] No whitelist given. Advertising > offers for all slaves > I0213 22:15:39.962286 17382 hierarchical_allocator_process.hpp:302] > Initializing hierarchical allocator process with master : > master@192.168.122.164:47130 > I0213 22:15:39.963912 17373 containerizer.cpp:180] Using isolation: > posix/cpu,posix/mem > I0213 22:15:39.965071 17373 containerizer.cpp:180] Using isolation: > posix/cpu,posix/mem > I0213 22:15:39.965754 17384 slave.cpp:112] Slave started on > 1)@192.168.122.164:47130 > I0213 22:15:39.966032 17387 master.cpp:760] The newly elected leader is > master@192.168.122.164:47130 with id > 2014-02-13-22:15:39-2759502016-47130-17373 > I0213 22:15:39.966341 17387 master.cpp:770] Elected as the leading master! > I0213 22:15:39.966227 17384 slave.cpp:122] Slave resources: cpus(*):1; > mem(*):979; disk(*):1001; ports(*):[31000-32000] > I0213 22:15:39.967811 17384 slave.cpp:150] Slave hostname: fedora-20 > I0213 22:15:39.967828 17384 slave.cpp:151] Slave checkpoint: true > I0213 22:15:39.968070 17384 state.cpp:33] Recovering state from > '/tmp/mesos-KG9dIs/0/meta' > I0213 22:15:39.968147 17384 status_update_manager.cpp:188] Recovering status > update manager > I0213 22:15:39.968200 17384 mesos_containerizer.cpp:137] Recovering > containerizer > I0213 22:15:39.969071 17384 slave.cpp:2670] Finished recovery > I0213 22:15:39.969290 17384 slave.cpp:397] New master detected at > master@192.168.122.164:47130 > I0213 22:15:39.970041 17384 slave.cpp:422] Detecting new master > I0213 22:15:39.970067 17384 status_update_manager.cpp:162] New master > detected at master@192.168.122.164:47130 > I0213 22:15:39.970116 17384 master.cpp:1840] Attempting to register slave on > fedora-20 at slave(1)@192.168.122.164:47130 > I0213 22:15:39.970130 17384 master.cpp:2810] Adding slave > 2014-02-13-22:15:39-2759502016-47130-17373-0 at fedora-20 with cpus(*):1; > mem(*):979; disk(*):1001; ports(*):[31000-32000] > I0213 22:15:39.970213 17384 slave.cpp:440] Registered with master > master@192.168.122.164:47130; given slave ID > 2014-02-13-22:15:39-2759502016-47130-17373-0 > I0213 22:15:39.970365 17384 slave.cpp:453] Checkpointing SlaveInfo to > '/tmp/mesos-KG9dIs/0/meta/slaves/2014-02-13-22:15:39-2759502016-47130-17373-0/slave.info' > I0213 22:15:39.971178 17382 hierarchical_allocator_process.hpp:445] Added > slave 2014-02-13-22:15:39-2759502016-47130-17373-0 (fedora-20) with > cpus(*):1; mem(*):979; disk(*):1001; ports(*):[31000-32000] (and cpus(*):1; > mem(*):979; disk(*):1001; ports(*):[31000-32000] available) > I0213 22:15:39.971231 17382 hierarchical_allocator_process.hpp:708] Performed > allocation for slave 2014-02-13-22:15:39-2759502016-47130-17373-0 in 9023ns > I0213 22:15:39.972479 17373 containerizer.cpp:180] Using isolation: > posix/cpu,posix/mem > I0213 22:15:39.973124 17381 slave.cpp:112] Slave started on > 2)@192.168.122.164:47130 > I0213 22:15:39.973276 17381 slave.cpp:122] Slave resources: cpus(*):1; > mem(*):979; disk(*):1001; ports(*):[31000-32000] > I0213 22:15:39.974185 17381 slave.cpp:150] Slave hostname: fedora-20 > I0213 22:15:39.974202 17381 slave.cpp:151] Slave checkpoint: true > I0213 22:15:39.975014 17381 state.cpp:33] Recovering state from > '/tmp/mesos-KG9dIs/1/meta' > I0213 22:15:39.975071 17381 status_update_manager.cpp:188] Recovering status > update manager > I0213 22:15:39.975098 17381 mesos_containerizer.cpp:137] Recovering > containerizer > I0213 22:15:39.975219 17381 slave.cpp:2670] Finished recovery > I0213 22:15:39.976111 17381 slave.cpp:397] New master detected at > master@192.168.122.164:47130 > I0213 22:15:39.976758
[jira] [Commented] (MESOS-1755) Add docker support to mesos-execute
[ https://issues.apache.org/jira/browse/MESOS-1755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14120061#comment-14120061 ] Steven Schlansker commented on MESOS-1755: -- Hi Vinod and Timothy, sorry for the confusion. I looped back with Henning and Sean and this commit is indeed unrelated as you suspected. That said, it is a small commit, and I think it is worthy to be included in 0.20.1 even if Singularity does not need it -- it will make testing things somewhat easier and it seems low-risk. > Add docker support to mesos-execute > --- > > Key: MESOS-1755 > URL: https://issues.apache.org/jira/browse/MESOS-1755 > Project: Mesos > Issue Type: Bug >Reporter: Vinod Kone >Assignee: Timothy Chen > Fix For: 0.21.0 > > > The fix for this is already committed at https://reviews.apache.org/r/24808/. > I'm creating this ticket to track that this patch gets included in 0.20.1 > release, since apparently Singularity framework depends on this patch to work > with Docker !?!? > https://groups.google.com/forum/#!topic/singularity-users/GzzswbpI92E > [~tnachen]: Can you confirm if this has to be included in 0.20.1? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1554) Persistent resources support for storage-like services
[
https://issues.apache.org/jira/browse/MESOS-1554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14120041#comment-14120041
]
Steven Schlansker commented on MESOS-1554:
--
It would be nice to be able to manage e.g. Amazon EBS (or generic SAN) volumes
in this way. That would be very powerful indeed.
> Persistent resources support for storage-like services
> --
>
> Key: MESOS-1554
> URL: https://issues.apache.org/jira/browse/MESOS-1554
> Project: Mesos
> Issue Type: Epic
> Components: general, hadoop
>Reporter: Nikita Vetoshkin
>Priority: Minor
>
> This question came up in [dev mailing
> list|http://mail-archives.apache.org/mod_mbox/mesos-dev/201406.mbox/%3CCAK8jAgNDs9Fe011Sq1jeNr0h%3DE-tDD9rak6hAsap3PqHx1y%3DKQ%40mail.gmail.com%3E].
> It seems reasonable for storage like services (e.g. HDFS or Cassandra) to use
> Mesos to manage it's instances. But right now if we'd like to restart
> instance (e.g. to spin up a new version) - all previous instance version
> sandbox filesystem resources will be recycled by slave's garbage collector.
> At the moment filesystem resources can be managed out of band - i.e.
> instances can save their data in some database specific placed, that various
> instances can share (e.g. {{/var/lib/cassandra}}).
> [~benjaminhindman] suggested an idea in the mailing list (though it still
> needs some fleshing out):
> {quote}
> The idea originally came about because, even today, if we allocate some
> file system space to a task/executor, and then that task/executor
> terminates, we haven't officially "freed" those file system resources until
> after we garbage collect the task/executor sandbox! (We keep the sandbox
> around so a user/operator can get the stdout/stderr or anything else left
> around from their task/executor.)
> To solve this problem we wanted to be able to let a task/executor terminate
> but not *give up* all of it's resources, hence: persistent resources.
> Pushing this concept even further you could imagine always reallocating
> resources to a framework that had already been allocated those resources
> for a previous task/executor. Looked at from another perspective, these are
> "late-binding", or "lazy", resource reservations.
> At one point in time we had considered just doing 'right-of-first-refusal'
> for allocations after a task/executor terminate. But this is really
> insufficient for supporting storage-like frameworks well (and likely even
> harder to reliably implement then 'persistent resources' IMHO).
> There are a ton of things that need to get worked out in this model,
> including (but not limited to), how should a file system (or disk) be
> exposed in order to be made persistent? How should persistent resources be
> returned to a master? How many persistent resources can a framework get
> allocated?
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (MESOS-1750) ImportError in containerizer_pb2.py
[
https://issues.apache.org/jira/browse/MESOS-1750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14119530#comment-14119530
]
Till Toenshoff edited comment on MESOS-1750 at 9/3/14 8:19 AM:
---
That was exactly the only solution I could come up with so far as well;
patching the python proto results after they have been rendered. Not really a
beautiful solution but maybe the only option we got other than changing the
proto locations which seems just wrong considering that we would have to mix
all proto-files into one flat folder (including the internal proto's for
allowing MESOS-1731 to happen).
was (Author: tillt):
That was exactly the only solution I could come up with so far as well;
patching the python proto results after they have been rendered. Not really
beautiful solution but maybe the only option we got other than changing the
proto locations which seems just wrong considering that we would have to mix
all proto-files into one flat folder (including the internal proto's for
allowing MESOS-1731 to happen).
> ImportError in containerizer_pb2.py
> ---
>
> Key: MESOS-1750
> URL: https://issues.apache.org/jira/browse/MESOS-1750
> Project: Mesos
> Issue Type: Bug
> Components: python api
>Reporter: Till Toenshoff
> Labels: mesos.interface, python-egg
>
> It appears as if the split of the Python egg (native/interface/protocol) has
> introduced a new issue.
> Whenever I am trying to use {{from mesos.interface import
> containerizer_pb2}}, I am receiving an {{ImportError: No module named
> mesos_pb2}}.
> For replicating this issue, try running
> {{"build/src/examples/python/test-containerizer"}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-1750) ImportError in containerizer_pb2.py
[
https://issues.apache.org/jira/browse/MESOS-1750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14119530#comment-14119530
]
Till Toenshoff commented on MESOS-1750:
---
That was exactly the only solution I could come up with so far as well;
patching the python proto results after they have been rendered. Not really
beautiful solution but maybe the only option we got other than changing the
proto locations which seems just wrong considering that we would have to mix
all proto-files into one flat folder (including the internal proto's for
allowing MESOS-1731 to happen).
> ImportError in containerizer_pb2.py
> ---
>
> Key: MESOS-1750
> URL: https://issues.apache.org/jira/browse/MESOS-1750
> Project: Mesos
> Issue Type: Bug
> Components: python api
>Reporter: Till Toenshoff
> Labels: mesos.interface, python-egg
>
> It appears as if the split of the Python egg (native/interface/protocol) has
> introduced a new issue.
> Whenever I am trying to use {{from mesos.interface import
> containerizer_pb2}}, I am receiving an {{ImportError: No module named
> mesos_pb2}}.
> For replicating this issue, try running
> {{"build/src/examples/python/test-containerizer"}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
