[jira] [Commented] (MESOS-3793) Cannot start mesos local on a Debian GNU/Linux 8 docker machine
[
https://issues.apache.org/jira/browse/MESOS-3793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15278226#comment-15278226
]
igor commented on MESOS-3793:
-
--no-systemd_enable_support flag helped.
> Cannot start mesos local on a Debian GNU/Linux 8 docker machine
> ---
>
> Key: MESOS-3793
> URL: https://issues.apache.org/jira/browse/MESOS-3793
> Project: Mesos
> Issue Type: Bug
>Affects Versions: 0.25.0
> Environment: Debian GNU/Linux 8 docker machine
>Reporter: Matthias Veit
>Assignee: Jojy Varghese
> Labels: mesosphere
> Fix For: 0.26.0
>
>
> We updated the mesos version to 0.25.0 in our Marathon docker image, that
> runs our integration tests.
> We use mesos local for those tests. This fails with this message:
> {noformat}
> root@a06e4b4eb776:/marathon# mesos local
> I1022 18:42:26.852485 136 leveldb.cpp:176] Opened db in 6.103258ms
> I1022 18:42:26.853302 136 leveldb.cpp:183] Compacted db in 765740ns
> I1022 18:42:26.853343 136 leveldb.cpp:198] Created db iterator in 9001ns
> I1022 18:42:26.853355 136 leveldb.cpp:204] Seeked to beginning of db in
> 1287ns
> I1022 18:42:26.853366 136 leveldb.cpp:273] Iterated through 0 keys in the
> db in ns
> I1022 18:42:26.853406 136 replica.cpp:744] Replica recovered with log
> positions 0 -> 0 with 1 holes and 0 unlearned
> I1022 18:42:26.853775 141 recover.cpp:449] Starting replica recovery
> I1022 18:42:26.853862 141 recover.cpp:475] Replica is in EMPTY status
> I1022 18:42:26.854751 138 replica.cpp:641] Replica in EMPTY status received
> a broadcasted recover request
> I1022 18:42:26.854856 140 recover.cpp:195] Received a recover response from
> a replica in EMPTY status
> I1022 18:42:26.855002 140 recover.cpp:566] Updating replica status to
> STARTING
> I1022 18:42:26.855655 138 master.cpp:376] Master
> a3f39818-1bda-4710-b96b-2a60ed4d12b8 (a06e4b4eb776) started on
> 172.17.0.14:5050
> I1022 18:42:26.855680 138 master.cpp:378] Flags at startup:
> --allocation_interval="1secs" --allocator="HierarchicalDRF"
> --authenticate="false" --authenticate_slaves="false"
> --authenticators="crammd5" --authorizers="local" --framework_sorter="drf"
> --help="false" --hostname_lookup="true" --initialize_driver_logging="true"
> --log_auto_initialize="true" --logbufsecs="0" --logging_level="INFO"
> --max_slave_ping_timeouts="5" --quiet="false"
> --recovery_slave_removal_limit="100%" --registry="replicated_log"
> --registry_fetch_timeout="1mins" --registry_store_timeout="5secs"
> --registry_strict="false" --root_submissions="true"
> --slave_ping_timeout="15secs" --slave_reregister_timeout="10mins"
> --user_sorter="drf" --version="false" --webui_dir="/usr/share/mesos/webui"
> --work_dir="/tmp/mesos/local/AK0XpG" --zk_session_timeout="10secs"
> I1022 18:42:26.855790 138 master.cpp:425] Master allowing unauthenticated
> frameworks to register
> I1022 18:42:26.855803 138 master.cpp:430] Master allowing unauthenticated
> slaves to register
> I1022 18:42:26.855815 138 master.cpp:467] Using default 'crammd5'
> authenticator
> W1022 18:42:26.855829 138 authenticator.cpp:505] No credentials provided,
> authentication requests will be refused
> I1022 18:42:26.855840 138 authenticator.cpp:512] Initializing server SASL
> I1022 18:42:26.856442 136 containerizer.cpp:143] Using isolation:
> posix/cpu,posix/mem,filesystem/posix
> I1022 18:42:26.856943 140 leveldb.cpp:306] Persisting metadata (8 bytes) to
> leveldb took 1.888185ms
> I1022 18:42:26.856987 140 replica.cpp:323] Persisted replica status to
> STARTING
> I1022 18:42:26.857115 140 recover.cpp:475] Replica is in STARTING status
> I1022 18:42:26.857270 140 replica.cpp:641] Replica in STARTING status
> received a broadcasted recover request
> I1022 18:42:26.857312 140 recover.cpp:195] Received a recover response from
> a replica in STARTING status
> I1022 18:42:26.857368 140 recover.cpp:566] Updating replica status to VOTING
> I1022 18:42:26.857781 140 leveldb.cpp:306] Persisting metadata (8 bytes) to
> leveldb took 371121ns
> I1022 18:42:26.857841 140 replica.cpp:323] Persisted replica status to
> VOTING
> I1022 18:42:26.857895 140 recover.cpp:580] Successfully joined the Paxos
> group
> I1022 18:42:26.857928 140 recover.cpp:464] Recover process terminated
> I1022 18:42:26.862455 137 master.cpp:1603] The newly elected leader is
> master@172.17.0.14:5050 with id a3f39818-1bda-4710-b96b-2a60ed4d12b8
> I1022 18:42:26.862498 137 master.cpp:1616] Elected as the leading master!
> I1022 18:42:26.862511 137 master.cpp:1376] Recovering from registrar
> I1022 18:42:26.862560 137 registrar.cpp:309] Recovering registrar
> Failed to create a containerizer: Could not create MesosContainerizer: Failed
[jira] [Commented] (MESOS-3793) Cannot start mesos local on a Debian GNU/Linux 8 docker machine
[
https://issues.apache.org/jira/browse/MESOS-3793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15278189#comment-15278189
]
igor commented on MESOS-3793:
-
same problem on centos 7.2
# mesos-slave --master=build:5050 --containerizers=docker --no-switch_user
--launcher=posix
I0510 17:30:01.43576227 main.cpp:223] Build: 2016-04-14 15:43:08 by root
I0510 17:30:01.43644727 main.cpp:225] Version: 0.28.1
I0510 17:30:01.43646527 main.cpp:228] Git tag: 0.28.1
I0510 17:30:01.43647527 main.cpp:232] Git SHA:
555db235a34afbb9fb49940376cc33a66f1f85f0
I0510 17:30:01.44741727 systemd.cpp:236] systemd version `219` detected
I0510 17:30:01.44750727 main.cpp:240] Inializing systemd state
Failed to initialize systemd: Failed to locate systemd runtime directory:
/run/systemd/system
> Cannot start mesos local on a Debian GNU/Linux 8 docker machine
> ---
>
> Key: MESOS-3793
> URL: https://issues.apache.org/jira/browse/MESOS-3793
> Project: Mesos
> Issue Type: Bug
>Affects Versions: 0.25.0
> Environment: Debian GNU/Linux 8 docker machine
>Reporter: Matthias Veit
>Assignee: Jojy Varghese
> Labels: mesosphere
> Fix For: 0.26.0
>
>
> We updated the mesos version to 0.25.0 in our Marathon docker image, that
> runs our integration tests.
> We use mesos local for those tests. This fails with this message:
> {noformat}
> root@a06e4b4eb776:/marathon# mesos local
> I1022 18:42:26.852485 136 leveldb.cpp:176] Opened db in 6.103258ms
> I1022 18:42:26.853302 136 leveldb.cpp:183] Compacted db in 765740ns
> I1022 18:42:26.853343 136 leveldb.cpp:198] Created db iterator in 9001ns
> I1022 18:42:26.853355 136 leveldb.cpp:204] Seeked to beginning of db in
> 1287ns
> I1022 18:42:26.853366 136 leveldb.cpp:273] Iterated through 0 keys in the
> db in ns
> I1022 18:42:26.853406 136 replica.cpp:744] Replica recovered with log
> positions 0 -> 0 with 1 holes and 0 unlearned
> I1022 18:42:26.853775 141 recover.cpp:449] Starting replica recovery
> I1022 18:42:26.853862 141 recover.cpp:475] Replica is in EMPTY status
> I1022 18:42:26.854751 138 replica.cpp:641] Replica in EMPTY status received
> a broadcasted recover request
> I1022 18:42:26.854856 140 recover.cpp:195] Received a recover response from
> a replica in EMPTY status
> I1022 18:42:26.855002 140 recover.cpp:566] Updating replica status to
> STARTING
> I1022 18:42:26.855655 138 master.cpp:376] Master
> a3f39818-1bda-4710-b96b-2a60ed4d12b8 (a06e4b4eb776) started on
> 172.17.0.14:5050
> I1022 18:42:26.855680 138 master.cpp:378] Flags at startup:
> --allocation_interval="1secs" --allocator="HierarchicalDRF"
> --authenticate="false" --authenticate_slaves="false"
> --authenticators="crammd5" --authorizers="local" --framework_sorter="drf"
> --help="false" --hostname_lookup="true" --initialize_driver_logging="true"
> --log_auto_initialize="true" --logbufsecs="0" --logging_level="INFO"
> --max_slave_ping_timeouts="5" --quiet="false"
> --recovery_slave_removal_limit="100%" --registry="replicated_log"
> --registry_fetch_timeout="1mins" --registry_store_timeout="5secs"
> --registry_strict="false" --root_submissions="true"
> --slave_ping_timeout="15secs" --slave_reregister_timeout="10mins"
> --user_sorter="drf" --version="false" --webui_dir="/usr/share/mesos/webui"
> --work_dir="/tmp/mesos/local/AK0XpG" --zk_session_timeout="10secs"
> I1022 18:42:26.855790 138 master.cpp:425] Master allowing unauthenticated
> frameworks to register
> I1022 18:42:26.855803 138 master.cpp:430] Master allowing unauthenticated
> slaves to register
> I1022 18:42:26.855815 138 master.cpp:467] Using default 'crammd5'
> authenticator
> W1022 18:42:26.855829 138 authenticator.cpp:505] No credentials provided,
> authentication requests will be refused
> I1022 18:42:26.855840 138 authenticator.cpp:512] Initializing server SASL
> I1022 18:42:26.856442 136 containerizer.cpp:143] Using isolation:
> posix/cpu,posix/mem,filesystem/posix
> I1022 18:42:26.856943 140 leveldb.cpp:306] Persisting metadata (8 bytes) to
> leveldb took 1.888185ms
> I1022 18:42:26.856987 140 replica.cpp:323] Persisted replica status to
> STARTING
> I1022 18:42:26.857115 140 recover.cpp:475] Replica is in STARTING status
> I1022 18:42:26.857270 140 replica.cpp:641] Replica in STARTING status
> received a broadcasted recover request
> I1022 18:42:26.857312 140 recover.cpp:195] Received a recover response from
> a replica in STARTING status
> I1022 18:42:26.857368 140 recover.cpp:566] Updating replica status to VOTING
> I1022 18:42:26.857781 140 leveldb.cpp:306] Persisting metadata (8 bytes) to
> leveldb took 371121ns
> I1022 18:42:26.857841 140 replica.cpp:323] Persisted replica status to
> VOTING
> I
[jira] [Commented] (MESOS-3793) Cannot start mesos local on a Debian GNU/Linux 8 docker machine
[
https://issues.apache.org/jira/browse/MESOS-3793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15278190#comment-15278190
]
igor commented on MESOS-3793:
-
same problem on centos 7.2
{code}
mesos-slave --master=build:5050 --containerizers=docker --no-switch_user
--launcher=posix
I0510 17:30:01.435762 27 main.cpp:223] Build: 2016-04-14 15:43:08 by root
I0510 17:30:01.436447 27 main.cpp:225] Version: 0.28.1
I0510 17:30:01.436465 27 main.cpp:228] Git tag: 0.28.1
I0510 17:30:01.436475 27 main.cpp:232] Git SHA:
555db235a34afbb9fb49940376cc33a66f1f85f0
I0510 17:30:01.447417 27 systemd.cpp:236] systemd version `219` detected
I0510 17:30:01.447507 27 main.cpp:240] Inializing systemd state
Failed to initialize systemd: Failed to locate systemd runtime directory:
/run/systemd/system
{code}
> Cannot start mesos local on a Debian GNU/Linux 8 docker machine
> ---
>
> Key: MESOS-3793
> URL: https://issues.apache.org/jira/browse/MESOS-3793
> Project: Mesos
> Issue Type: Bug
>Affects Versions: 0.25.0
> Environment: Debian GNU/Linux 8 docker machine
>Reporter: Matthias Veit
>Assignee: Jojy Varghese
> Labels: mesosphere
> Fix For: 0.26.0
>
>
> We updated the mesos version to 0.25.0 in our Marathon docker image, that
> runs our integration tests.
> We use mesos local for those tests. This fails with this message:
> {noformat}
> root@a06e4b4eb776:/marathon# mesos local
> I1022 18:42:26.852485 136 leveldb.cpp:176] Opened db in 6.103258ms
> I1022 18:42:26.853302 136 leveldb.cpp:183] Compacted db in 765740ns
> I1022 18:42:26.853343 136 leveldb.cpp:198] Created db iterator in 9001ns
> I1022 18:42:26.853355 136 leveldb.cpp:204] Seeked to beginning of db in
> 1287ns
> I1022 18:42:26.853366 136 leveldb.cpp:273] Iterated through 0 keys in the
> db in ns
> I1022 18:42:26.853406 136 replica.cpp:744] Replica recovered with log
> positions 0 -> 0 with 1 holes and 0 unlearned
> I1022 18:42:26.853775 141 recover.cpp:449] Starting replica recovery
> I1022 18:42:26.853862 141 recover.cpp:475] Replica is in EMPTY status
> I1022 18:42:26.854751 138 replica.cpp:641] Replica in EMPTY status received
> a broadcasted recover request
> I1022 18:42:26.854856 140 recover.cpp:195] Received a recover response from
> a replica in EMPTY status
> I1022 18:42:26.855002 140 recover.cpp:566] Updating replica status to
> STARTING
> I1022 18:42:26.855655 138 master.cpp:376] Master
> a3f39818-1bda-4710-b96b-2a60ed4d12b8 (a06e4b4eb776) started on
> 172.17.0.14:5050
> I1022 18:42:26.855680 138 master.cpp:378] Flags at startup:
> --allocation_interval="1secs" --allocator="HierarchicalDRF"
> --authenticate="false" --authenticate_slaves="false"
> --authenticators="crammd5" --authorizers="local" --framework_sorter="drf"
> --help="false" --hostname_lookup="true" --initialize_driver_logging="true"
> --log_auto_initialize="true" --logbufsecs="0" --logging_level="INFO"
> --max_slave_ping_timeouts="5" --quiet="false"
> --recovery_slave_removal_limit="100%" --registry="replicated_log"
> --registry_fetch_timeout="1mins" --registry_store_timeout="5secs"
> --registry_strict="false" --root_submissions="true"
> --slave_ping_timeout="15secs" --slave_reregister_timeout="10mins"
> --user_sorter="drf" --version="false" --webui_dir="/usr/share/mesos/webui"
> --work_dir="/tmp/mesos/local/AK0XpG" --zk_session_timeout="10secs"
> I1022 18:42:26.855790 138 master.cpp:425] Master allowing unauthenticated
> frameworks to register
> I1022 18:42:26.855803 138 master.cpp:430] Master allowing unauthenticated
> slaves to register
> I1022 18:42:26.855815 138 master.cpp:467] Using default 'crammd5'
> authenticator
> W1022 18:42:26.855829 138 authenticator.cpp:505] No credentials provided,
> authentication requests will be refused
> I1022 18:42:26.855840 138 authenticator.cpp:512] Initializing server SASL
> I1022 18:42:26.856442 136 containerizer.cpp:143] Using isolation:
> posix/cpu,posix/mem,filesystem/posix
> I1022 18:42:26.856943 140 leveldb.cpp:306] Persisting metadata (8 bytes) to
> leveldb took 1.888185ms
> I1022 18:42:26.856987 140 replica.cpp:323] Persisted replica status to
> STARTING
> I1022 18:42:26.857115 140 recover.cpp:475] Replica is in STARTING status
> I1022 18:42:26.857270 140 replica.cpp:641] Replica in STARTING status
> received a broadcasted recover request
> I1022 18:42:26.857312 140 recover.cpp:195] Received a recover response from
> a replica in STARTING status
> I1022 18:42:26.857368 140 recover.cpp:566] Updating replica status to VOTING
> I1022 18:42:26.857781 140 leveldb.cpp:306] Persisting metadata (8 bytes) to
> leveldb took 371121ns
> I1022 18:42:26.857841 140 replica.cpp:323] Persisted replica status to
> VOTING
> I1022 18:
[jira] [Commented] (MESOS-3793) Cannot start mesos local on a Debian GNU/Linux 8 docker machine
[
https://issues.apache.org/jira/browse/MESOS-3793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15278191#comment-15278191
]
igor commented on MESOS-3793:
-
version 0.28.1
> Cannot start mesos local on a Debian GNU/Linux 8 docker machine
> ---
>
> Key: MESOS-3793
> URL: https://issues.apache.org/jira/browse/MESOS-3793
> Project: Mesos
> Issue Type: Bug
>Affects Versions: 0.25.0
> Environment: Debian GNU/Linux 8 docker machine
>Reporter: Matthias Veit
>Assignee: Jojy Varghese
> Labels: mesosphere
> Fix For: 0.26.0
>
>
> We updated the mesos version to 0.25.0 in our Marathon docker image, that
> runs our integration tests.
> We use mesos local for those tests. This fails with this message:
> {noformat}
> root@a06e4b4eb776:/marathon# mesos local
> I1022 18:42:26.852485 136 leveldb.cpp:176] Opened db in 6.103258ms
> I1022 18:42:26.853302 136 leveldb.cpp:183] Compacted db in 765740ns
> I1022 18:42:26.853343 136 leveldb.cpp:198] Created db iterator in 9001ns
> I1022 18:42:26.853355 136 leveldb.cpp:204] Seeked to beginning of db in
> 1287ns
> I1022 18:42:26.853366 136 leveldb.cpp:273] Iterated through 0 keys in the
> db in ns
> I1022 18:42:26.853406 136 replica.cpp:744] Replica recovered with log
> positions 0 -> 0 with 1 holes and 0 unlearned
> I1022 18:42:26.853775 141 recover.cpp:449] Starting replica recovery
> I1022 18:42:26.853862 141 recover.cpp:475] Replica is in EMPTY status
> I1022 18:42:26.854751 138 replica.cpp:641] Replica in EMPTY status received
> a broadcasted recover request
> I1022 18:42:26.854856 140 recover.cpp:195] Received a recover response from
> a replica in EMPTY status
> I1022 18:42:26.855002 140 recover.cpp:566] Updating replica status to
> STARTING
> I1022 18:42:26.855655 138 master.cpp:376] Master
> a3f39818-1bda-4710-b96b-2a60ed4d12b8 (a06e4b4eb776) started on
> 172.17.0.14:5050
> I1022 18:42:26.855680 138 master.cpp:378] Flags at startup:
> --allocation_interval="1secs" --allocator="HierarchicalDRF"
> --authenticate="false" --authenticate_slaves="false"
> --authenticators="crammd5" --authorizers="local" --framework_sorter="drf"
> --help="false" --hostname_lookup="true" --initialize_driver_logging="true"
> --log_auto_initialize="true" --logbufsecs="0" --logging_level="INFO"
> --max_slave_ping_timeouts="5" --quiet="false"
> --recovery_slave_removal_limit="100%" --registry="replicated_log"
> --registry_fetch_timeout="1mins" --registry_store_timeout="5secs"
> --registry_strict="false" --root_submissions="true"
> --slave_ping_timeout="15secs" --slave_reregister_timeout="10mins"
> --user_sorter="drf" --version="false" --webui_dir="/usr/share/mesos/webui"
> --work_dir="/tmp/mesos/local/AK0XpG" --zk_session_timeout="10secs"
> I1022 18:42:26.855790 138 master.cpp:425] Master allowing unauthenticated
> frameworks to register
> I1022 18:42:26.855803 138 master.cpp:430] Master allowing unauthenticated
> slaves to register
> I1022 18:42:26.855815 138 master.cpp:467] Using default 'crammd5'
> authenticator
> W1022 18:42:26.855829 138 authenticator.cpp:505] No credentials provided,
> authentication requests will be refused
> I1022 18:42:26.855840 138 authenticator.cpp:512] Initializing server SASL
> I1022 18:42:26.856442 136 containerizer.cpp:143] Using isolation:
> posix/cpu,posix/mem,filesystem/posix
> I1022 18:42:26.856943 140 leveldb.cpp:306] Persisting metadata (8 bytes) to
> leveldb took 1.888185ms
> I1022 18:42:26.856987 140 replica.cpp:323] Persisted replica status to
> STARTING
> I1022 18:42:26.857115 140 recover.cpp:475] Replica is in STARTING status
> I1022 18:42:26.857270 140 replica.cpp:641] Replica in STARTING status
> received a broadcasted recover request
> I1022 18:42:26.857312 140 recover.cpp:195] Received a recover response from
> a replica in STARTING status
> I1022 18:42:26.857368 140 recover.cpp:566] Updating replica status to VOTING
> I1022 18:42:26.857781 140 leveldb.cpp:306] Persisting metadata (8 bytes) to
> leveldb took 371121ns
> I1022 18:42:26.857841 140 replica.cpp:323] Persisted replica status to
> VOTING
> I1022 18:42:26.857895 140 recover.cpp:580] Successfully joined the Paxos
> group
> I1022 18:42:26.857928 140 recover.cpp:464] Recover process terminated
> I1022 18:42:26.862455 137 master.cpp:1603] The newly elected leader is
> master@172.17.0.14:5050 with id a3f39818-1bda-4710-b96b-2a60ed4d12b8
> I1022 18:42:26.862498 137 master.cpp:1616] Elected as the leading master!
> I1022 18:42:26.862511 137 master.cpp:1376] Recovering from registrar
> I1022 18:42:26.862560 137 registrar.cpp:309] Recovering registrar
> Failed to create a containerizer: Could not create MesosContainerizer: Failed
> to create launcher: Fail
[jira] [Commented] (MESOS-3793) Cannot start mesos local on a Debian GNU/Linux 8 docker machine
[
https://issues.apache.org/jira/browse/MESOS-3793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15254514#comment-15254514
]
Stavros Kontopoulos commented on MESOS-3793:
Still see the issue with mesos 0.28.1 using this images
https://hub.docker.com/r/osrg/dind-ubuntu-vivid/
and installing mesos as follows:
RUN apt-key adv --keyserver keyserver.ubuntu.com --recv E56151BF && \
DISTRO=$(lsb_release -is | tr '[:upper:]' '[:lower:]') && \
CODENAME=$(lsb_release -cs) && \
echo $CODENAME && \
echo $DISTRO && \
echo "deb http://repos.mesosphere.io/${DISTRO} ${CODENAME} main" | tee
/etc/apt/sources.list.d/mesosphere.list && \
apt-get -y update && \
VERSION=$(apt-cache madison mesos | head -1 | awk '{ print $3 }') && \
MARATHON_VERSION=$(apt-cache madison marathon | head -1 | awk '{ print $3
}') && \
apt-get -y install mesos=$VERSION && \
apt-get install marathon=$MARATHON_VERSION && \
rm -rf /var/lib/apt/lists/*
When i run:
root@stavrospc:/# /usr/sbin/mesos-slave --master=172.17.0.1:5050
--ip=172.17.0.1 --launcher=posix
I0422 19:33:32.743777 24201 logging.cpp:188] INFO level logging started!
I0422 19:33:32.743979 24201 main.cpp:223] Build: 2016-04-14 15:43:15 by root
I0422 19:33:32.743996 24201 main.cpp:225] Version: 0.28.1
I0422 19:33:32.744005 24201 main.cpp:228] Git tag: 0.28.1
I0422 19:33:32.744014 24201 main.cpp:232] Git SHA:
555db235a34afbb9fb49940376cc33a66f1f85f0
I0422 19:33:32.746184 24201 systemd.cpp:236] systemd version `219` detected
I0422 19:33:32.746215 24201 main.cpp:240] Inializing systemd state
Failed to initialize systemd: Failed to locate systemd runtime directory:
/run/systemd/system
root@stavrospc:/# ls /usr/lib/libm*
/usr/lib/libmesos-0.28.1.so /usr/lib/libmesos.la /usr/lib/libmesos.so
Any ideas?
> Cannot start mesos local on a Debian GNU/Linux 8 docker machine
> ---
>
> Key: MESOS-3793
> URL: https://issues.apache.org/jira/browse/MESOS-3793
> Project: Mesos
> Issue Type: Bug
>Affects Versions: 0.25.0
> Environment: Debian GNU/Linux 8 docker machine
>Reporter: Matthias Veit
>Assignee: Jojy Varghese
> Labels: mesosphere
> Fix For: 0.26.0
>
>
> We updated the mesos version to 0.25.0 in our Marathon docker image, that
> runs our integration tests.
> We use mesos local for those tests. This fails with this message:
> {noformat}
> root@a06e4b4eb776:/marathon# mesos local
> I1022 18:42:26.852485 136 leveldb.cpp:176] Opened db in 6.103258ms
> I1022 18:42:26.853302 136 leveldb.cpp:183] Compacted db in 765740ns
> I1022 18:42:26.853343 136 leveldb.cpp:198] Created db iterator in 9001ns
> I1022 18:42:26.853355 136 leveldb.cpp:204] Seeked to beginning of db in
> 1287ns
> I1022 18:42:26.853366 136 leveldb.cpp:273] Iterated through 0 keys in the
> db in ns
> I1022 18:42:26.853406 136 replica.cpp:744] Replica recovered with log
> positions 0 -> 0 with 1 holes and 0 unlearned
> I1022 18:42:26.853775 141 recover.cpp:449] Starting replica recovery
> I1022 18:42:26.853862 141 recover.cpp:475] Replica is in EMPTY status
> I1022 18:42:26.854751 138 replica.cpp:641] Replica in EMPTY status received
> a broadcasted recover request
> I1022 18:42:26.854856 140 recover.cpp:195] Received a recover response from
> a replica in EMPTY status
> I1022 18:42:26.855002 140 recover.cpp:566] Updating replica status to
> STARTING
> I1022 18:42:26.855655 138 master.cpp:376] Master
> a3f39818-1bda-4710-b96b-2a60ed4d12b8 (a06e4b4eb776) started on
> 172.17.0.14:5050
> I1022 18:42:26.855680 138 master.cpp:378] Flags at startup:
> --allocation_interval="1secs" --allocator="HierarchicalDRF"
> --authenticate="false" --authenticate_slaves="false"
> --authenticators="crammd5" --authorizers="local" --framework_sorter="drf"
> --help="false" --hostname_lookup="true" --initialize_driver_logging="true"
> --log_auto_initialize="true" --logbufsecs="0" --logging_level="INFO"
> --max_slave_ping_timeouts="5" --quiet="false"
> --recovery_slave_removal_limit="100%" --registry="replicated_log"
> --registry_fetch_timeout="1mins" --registry_store_timeout="5secs"
> --registry_strict="false" --root_submissions="true"
> --slave_ping_timeout="15secs" --slave_reregister_timeout="10mins"
> --user_sorter="drf" --version="false" --webui_dir="/usr/share/mesos/webui"
> --work_dir="/tmp/mesos/local/AK0XpG" --zk_session_timeout="10secs"
> I1022 18:42:26.855790 138 master.cpp:425] Master allowing unauthenticated
> frameworks to register
> I1022 18:42:26.855803 138 master.cpp:430] Master allowing unauthenticated
> slaves to register
> I1022 18:42:26.855815 138 master.cpp:467] Using default 'crammd5'
> authenticator
> W1022 18:42:26.855829 138 authenticator.cpp:505] No credentials provide
[jira] [Commented] (MESOS-3793) Cannot start mesos local on a Debian GNU/Linux 8 docker machine
[
https://issues.apache.org/jira/browse/MESOS-3793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15025837#comment-15025837
]
Jojy Varghese commented on MESOS-3793:
--
review posted: https://reviews.apache.org/r/40681
> Cannot start mesos local on a Debian GNU/Linux 8 docker machine
> ---
>
> Key: MESOS-3793
> URL: https://issues.apache.org/jira/browse/MESOS-3793
> Project: Mesos
> Issue Type: Bug
>Affects Versions: 0.25.0
> Environment: Debian GNU/Linux 8 docker machine
>Reporter: Matthias Veit
>Assignee: Jojy Varghese
> Labels: mesosphere
>
> We updated the mesos version to 0.25.0 in our Marathon docker image, that
> runs our integration tests.
> We use mesos local for those tests. This fails with this message:
> {noformat}
> root@a06e4b4eb776:/marathon# mesos local
> I1022 18:42:26.852485 136 leveldb.cpp:176] Opened db in 6.103258ms
> I1022 18:42:26.853302 136 leveldb.cpp:183] Compacted db in 765740ns
> I1022 18:42:26.853343 136 leveldb.cpp:198] Created db iterator in 9001ns
> I1022 18:42:26.853355 136 leveldb.cpp:204] Seeked to beginning of db in
> 1287ns
> I1022 18:42:26.853366 136 leveldb.cpp:273] Iterated through 0 keys in the
> db in ns
> I1022 18:42:26.853406 136 replica.cpp:744] Replica recovered with log
> positions 0 -> 0 with 1 holes and 0 unlearned
> I1022 18:42:26.853775 141 recover.cpp:449] Starting replica recovery
> I1022 18:42:26.853862 141 recover.cpp:475] Replica is in EMPTY status
> I1022 18:42:26.854751 138 replica.cpp:641] Replica in EMPTY status received
> a broadcasted recover request
> I1022 18:42:26.854856 140 recover.cpp:195] Received a recover response from
> a replica in EMPTY status
> I1022 18:42:26.855002 140 recover.cpp:566] Updating replica status to
> STARTING
> I1022 18:42:26.855655 138 master.cpp:376] Master
> a3f39818-1bda-4710-b96b-2a60ed4d12b8 (a06e4b4eb776) started on
> 172.17.0.14:5050
> I1022 18:42:26.855680 138 master.cpp:378] Flags at startup:
> --allocation_interval="1secs" --allocator="HierarchicalDRF"
> --authenticate="false" --authenticate_slaves="false"
> --authenticators="crammd5" --authorizers="local" --framework_sorter="drf"
> --help="false" --hostname_lookup="true" --initialize_driver_logging="true"
> --log_auto_initialize="true" --logbufsecs="0" --logging_level="INFO"
> --max_slave_ping_timeouts="5" --quiet="false"
> --recovery_slave_removal_limit="100%" --registry="replicated_log"
> --registry_fetch_timeout="1mins" --registry_store_timeout="5secs"
> --registry_strict="false" --root_submissions="true"
> --slave_ping_timeout="15secs" --slave_reregister_timeout="10mins"
> --user_sorter="drf" --version="false" --webui_dir="/usr/share/mesos/webui"
> --work_dir="/tmp/mesos/local/AK0XpG" --zk_session_timeout="10secs"
> I1022 18:42:26.855790 138 master.cpp:425] Master allowing unauthenticated
> frameworks to register
> I1022 18:42:26.855803 138 master.cpp:430] Master allowing unauthenticated
> slaves to register
> I1022 18:42:26.855815 138 master.cpp:467] Using default 'crammd5'
> authenticator
> W1022 18:42:26.855829 138 authenticator.cpp:505] No credentials provided,
> authentication requests will be refused
> I1022 18:42:26.855840 138 authenticator.cpp:512] Initializing server SASL
> I1022 18:42:26.856442 136 containerizer.cpp:143] Using isolation:
> posix/cpu,posix/mem,filesystem/posix
> I1022 18:42:26.856943 140 leveldb.cpp:306] Persisting metadata (8 bytes) to
> leveldb took 1.888185ms
> I1022 18:42:26.856987 140 replica.cpp:323] Persisted replica status to
> STARTING
> I1022 18:42:26.857115 140 recover.cpp:475] Replica is in STARTING status
> I1022 18:42:26.857270 140 replica.cpp:641] Replica in STARTING status
> received a broadcasted recover request
> I1022 18:42:26.857312 140 recover.cpp:195] Received a recover response from
> a replica in STARTING status
> I1022 18:42:26.857368 140 recover.cpp:566] Updating replica status to VOTING
> I1022 18:42:26.857781 140 leveldb.cpp:306] Persisting metadata (8 bytes) to
> leveldb took 371121ns
> I1022 18:42:26.857841 140 replica.cpp:323] Persisted replica status to
> VOTING
> I1022 18:42:26.857895 140 recover.cpp:580] Successfully joined the Paxos
> group
> I1022 18:42:26.857928 140 recover.cpp:464] Recover process terminated
> I1022 18:42:26.862455 137 master.cpp:1603] The newly elected leader is
> master@172.17.0.14:5050 with id a3f39818-1bda-4710-b96b-2a60ed4d12b8
> I1022 18:42:26.862498 137 master.cpp:1616] Elected as the leading master!
> I1022 18:42:26.862511 137 master.cpp:1376] Recovering from registrar
> I1022 18:42:26.862560 137 registrar.cpp:309] Recovering registrar
> Failed to create a containerizer: Could not create MesosContainerizer: Failed
> to
[jira] [Commented] (MESOS-3793) Cannot start mesos local on a Debian GNU/Linux 8 docker machine
[
https://issues.apache.org/jira/browse/MESOS-3793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15024903#comment-15024903
]
Bernd Mathiske commented on MESOS-3793:
---
[~jojy] I like #2 better also for the reasons you have stated, with the
addition that we can check first if the var is already set and not overwrite it
if it is.
> Cannot start mesos local on a Debian GNU/Linux 8 docker machine
> ---
>
> Key: MESOS-3793
> URL: https://issues.apache.org/jira/browse/MESOS-3793
> Project: Mesos
> Issue Type: Bug
>Affects Versions: 0.25.0
> Environment: Debian GNU/Linux 8 docker machine
>Reporter: Matthias Veit
>Assignee: Jojy Varghese
> Labels: mesosphere
>
> We updated the mesos version to 0.25.0 in our Marathon docker image, that
> runs our integration tests.
> We use mesos local for those tests. This fails with this message:
> {noformat}
> root@a06e4b4eb776:/marathon# mesos local
> I1022 18:42:26.852485 136 leveldb.cpp:176] Opened db in 6.103258ms
> I1022 18:42:26.853302 136 leveldb.cpp:183] Compacted db in 765740ns
> I1022 18:42:26.853343 136 leveldb.cpp:198] Created db iterator in 9001ns
> I1022 18:42:26.853355 136 leveldb.cpp:204] Seeked to beginning of db in
> 1287ns
> I1022 18:42:26.853366 136 leveldb.cpp:273] Iterated through 0 keys in the
> db in ns
> I1022 18:42:26.853406 136 replica.cpp:744] Replica recovered with log
> positions 0 -> 0 with 1 holes and 0 unlearned
> I1022 18:42:26.853775 141 recover.cpp:449] Starting replica recovery
> I1022 18:42:26.853862 141 recover.cpp:475] Replica is in EMPTY status
> I1022 18:42:26.854751 138 replica.cpp:641] Replica in EMPTY status received
> a broadcasted recover request
> I1022 18:42:26.854856 140 recover.cpp:195] Received a recover response from
> a replica in EMPTY status
> I1022 18:42:26.855002 140 recover.cpp:566] Updating replica status to
> STARTING
> I1022 18:42:26.855655 138 master.cpp:376] Master
> a3f39818-1bda-4710-b96b-2a60ed4d12b8 (a06e4b4eb776) started on
> 172.17.0.14:5050
> I1022 18:42:26.855680 138 master.cpp:378] Flags at startup:
> --allocation_interval="1secs" --allocator="HierarchicalDRF"
> --authenticate="false" --authenticate_slaves="false"
> --authenticators="crammd5" --authorizers="local" --framework_sorter="drf"
> --help="false" --hostname_lookup="true" --initialize_driver_logging="true"
> --log_auto_initialize="true" --logbufsecs="0" --logging_level="INFO"
> --max_slave_ping_timeouts="5" --quiet="false"
> --recovery_slave_removal_limit="100%" --registry="replicated_log"
> --registry_fetch_timeout="1mins" --registry_store_timeout="5secs"
> --registry_strict="false" --root_submissions="true"
> --slave_ping_timeout="15secs" --slave_reregister_timeout="10mins"
> --user_sorter="drf" --version="false" --webui_dir="/usr/share/mesos/webui"
> --work_dir="/tmp/mesos/local/AK0XpG" --zk_session_timeout="10secs"
> I1022 18:42:26.855790 138 master.cpp:425] Master allowing unauthenticated
> frameworks to register
> I1022 18:42:26.855803 138 master.cpp:430] Master allowing unauthenticated
> slaves to register
> I1022 18:42:26.855815 138 master.cpp:467] Using default 'crammd5'
> authenticator
> W1022 18:42:26.855829 138 authenticator.cpp:505] No credentials provided,
> authentication requests will be refused
> I1022 18:42:26.855840 138 authenticator.cpp:512] Initializing server SASL
> I1022 18:42:26.856442 136 containerizer.cpp:143] Using isolation:
> posix/cpu,posix/mem,filesystem/posix
> I1022 18:42:26.856943 140 leveldb.cpp:306] Persisting metadata (8 bytes) to
> leveldb took 1.888185ms
> I1022 18:42:26.856987 140 replica.cpp:323] Persisted replica status to
> STARTING
> I1022 18:42:26.857115 140 recover.cpp:475] Replica is in STARTING status
> I1022 18:42:26.857270 140 replica.cpp:641] Replica in STARTING status
> received a broadcasted recover request
> I1022 18:42:26.857312 140 recover.cpp:195] Received a recover response from
> a replica in STARTING status
> I1022 18:42:26.857368 140 recover.cpp:566] Updating replica status to VOTING
> I1022 18:42:26.857781 140 leveldb.cpp:306] Persisting metadata (8 bytes) to
> leveldb took 371121ns
> I1022 18:42:26.857841 140 replica.cpp:323] Persisted replica status to
> VOTING
> I1022 18:42:26.857895 140 recover.cpp:580] Successfully joined the Paxos
> group
> I1022 18:42:26.857928 140 recover.cpp:464] Recover process terminated
> I1022 18:42:26.862455 137 master.cpp:1603] The newly elected leader is
> master@172.17.0.14:5050 with id a3f39818-1bda-4710-b96b-2a60ed4d12b8
> I1022 18:42:26.862498 137 master.cpp:1616] Elected as the leading master!
> I1022 18:42:26.862511 137 master.cpp:1376] Recovering from registrar
> I1022 18:42:26.862560 137 registrar.
[jira] [Commented] (MESOS-3793) Cannot start mesos local on a Debian GNU/Linux 8 docker machine
[
https://issues.apache.org/jira/browse/MESOS-3793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15024670#comment-15024670
]
Jojy Varghese commented on MESOS-3793:
--
[~bernd-mesos], [~jieyu]
Since there are 2 ways proposed, here is my take on both:
1. We change the ``` bin/mesos-local-flags.sh.in ``` file to add ```export
MESOS_LAUNCHER=posix```
- Adding the new env variable along with MESOS_WORK_DIR looks out of place.
- mesos-slave is an executable that can be launched by itself. If launched by
itself, it will bypass the shell script and the issue will surface
2. Change local/main.cpp to add the line ```os::setenv("MESOS_LAUNCHER",
"posix");```.
- We set LIBPROCESS_IP and LIbPROCESS_PORT this way in that file. So the new
env variable could be added along with the existing ones.
- This approach will make sure that the code path of setting the new env
variable is called in all cases.
I like #2 approach. Please let me know if that sounds ok.
> Cannot start mesos local on a Debian GNU/Linux 8 docker machine
> ---
>
> Key: MESOS-3793
> URL: https://issues.apache.org/jira/browse/MESOS-3793
> Project: Mesos
> Issue Type: Bug
>Affects Versions: 0.25.0
> Environment: Debian GNU/Linux 8 docker machine
>Reporter: Matthias Veit
>Assignee: Jojy Varghese
> Labels: mesosphere
>
> We updated the mesos version to 0.25.0 in our Marathon docker image, that
> runs our integration tests.
> We use mesos local for those tests. This fails with this message:
> {noformat}
> root@a06e4b4eb776:/marathon# mesos local
> I1022 18:42:26.852485 136 leveldb.cpp:176] Opened db in 6.103258ms
> I1022 18:42:26.853302 136 leveldb.cpp:183] Compacted db in 765740ns
> I1022 18:42:26.853343 136 leveldb.cpp:198] Created db iterator in 9001ns
> I1022 18:42:26.853355 136 leveldb.cpp:204] Seeked to beginning of db in
> 1287ns
> I1022 18:42:26.853366 136 leveldb.cpp:273] Iterated through 0 keys in the
> db in ns
> I1022 18:42:26.853406 136 replica.cpp:744] Replica recovered with log
> positions 0 -> 0 with 1 holes and 0 unlearned
> I1022 18:42:26.853775 141 recover.cpp:449] Starting replica recovery
> I1022 18:42:26.853862 141 recover.cpp:475] Replica is in EMPTY status
> I1022 18:42:26.854751 138 replica.cpp:641] Replica in EMPTY status received
> a broadcasted recover request
> I1022 18:42:26.854856 140 recover.cpp:195] Received a recover response from
> a replica in EMPTY status
> I1022 18:42:26.855002 140 recover.cpp:566] Updating replica status to
> STARTING
> I1022 18:42:26.855655 138 master.cpp:376] Master
> a3f39818-1bda-4710-b96b-2a60ed4d12b8 (a06e4b4eb776) started on
> 172.17.0.14:5050
> I1022 18:42:26.855680 138 master.cpp:378] Flags at startup:
> --allocation_interval="1secs" --allocator="HierarchicalDRF"
> --authenticate="false" --authenticate_slaves="false"
> --authenticators="crammd5" --authorizers="local" --framework_sorter="drf"
> --help="false" --hostname_lookup="true" --initialize_driver_logging="true"
> --log_auto_initialize="true" --logbufsecs="0" --logging_level="INFO"
> --max_slave_ping_timeouts="5" --quiet="false"
> --recovery_slave_removal_limit="100%" --registry="replicated_log"
> --registry_fetch_timeout="1mins" --registry_store_timeout="5secs"
> --registry_strict="false" --root_submissions="true"
> --slave_ping_timeout="15secs" --slave_reregister_timeout="10mins"
> --user_sorter="drf" --version="false" --webui_dir="/usr/share/mesos/webui"
> --work_dir="/tmp/mesos/local/AK0XpG" --zk_session_timeout="10secs"
> I1022 18:42:26.855790 138 master.cpp:425] Master allowing unauthenticated
> frameworks to register
> I1022 18:42:26.855803 138 master.cpp:430] Master allowing unauthenticated
> slaves to register
> I1022 18:42:26.855815 138 master.cpp:467] Using default 'crammd5'
> authenticator
> W1022 18:42:26.855829 138 authenticator.cpp:505] No credentials provided,
> authentication requests will be refused
> I1022 18:42:26.855840 138 authenticator.cpp:512] Initializing server SASL
> I1022 18:42:26.856442 136 containerizer.cpp:143] Using isolation:
> posix/cpu,posix/mem,filesystem/posix
> I1022 18:42:26.856943 140 leveldb.cpp:306] Persisting metadata (8 bytes) to
> leveldb took 1.888185ms
> I1022 18:42:26.856987 140 replica.cpp:323] Persisted replica status to
> STARTING
> I1022 18:42:26.857115 140 recover.cpp:475] Replica is in STARTING status
> I1022 18:42:26.857270 140 replica.cpp:641] Replica in STARTING status
> received a broadcasted recover request
> I1022 18:42:26.857312 140 recover.cpp:195] Received a recover response from
> a replica in STARTING status
> I1022 18:42:26.857368 140 recover.cpp:566] Updating replica status to VOTING
> I1022 18:42:26.857781 140 leveldb.cpp:306] Persist
[jira] [Commented] (MESOS-3793) Cannot start mesos local on a Debian GNU/Linux 8 docker machine
[
https://issues.apache.org/jira/browse/MESOS-3793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15024242#comment-15024242
]
Bernd Mathiske commented on MESOS-3793:
---
[~jojy] Can you please implement the solution proposed by Jie and file a patch?
> Cannot start mesos local on a Debian GNU/Linux 8 docker machine
> ---
>
> Key: MESOS-3793
> URL: https://issues.apache.org/jira/browse/MESOS-3793
> Project: Mesos
> Issue Type: Bug
>Affects Versions: 0.25.0
> Environment: Debian GNU/Linux 8 docker machine
>Reporter: Matthias Veit
>Assignee: Jojy Varghese
> Labels: mesosphere
>
> We updated the mesos version to 0.25.0 in our Marathon docker image, that
> runs our integration tests.
> We use mesos local for those tests. This fails with this message:
> {noformat}
> root@a06e4b4eb776:/marathon# mesos local
> I1022 18:42:26.852485 136 leveldb.cpp:176] Opened db in 6.103258ms
> I1022 18:42:26.853302 136 leveldb.cpp:183] Compacted db in 765740ns
> I1022 18:42:26.853343 136 leveldb.cpp:198] Created db iterator in 9001ns
> I1022 18:42:26.853355 136 leveldb.cpp:204] Seeked to beginning of db in
> 1287ns
> I1022 18:42:26.853366 136 leveldb.cpp:273] Iterated through 0 keys in the
> db in ns
> I1022 18:42:26.853406 136 replica.cpp:744] Replica recovered with log
> positions 0 -> 0 with 1 holes and 0 unlearned
> I1022 18:42:26.853775 141 recover.cpp:449] Starting replica recovery
> I1022 18:42:26.853862 141 recover.cpp:475] Replica is in EMPTY status
> I1022 18:42:26.854751 138 replica.cpp:641] Replica in EMPTY status received
> a broadcasted recover request
> I1022 18:42:26.854856 140 recover.cpp:195] Received a recover response from
> a replica in EMPTY status
> I1022 18:42:26.855002 140 recover.cpp:566] Updating replica status to
> STARTING
> I1022 18:42:26.855655 138 master.cpp:376] Master
> a3f39818-1bda-4710-b96b-2a60ed4d12b8 (a06e4b4eb776) started on
> 172.17.0.14:5050
> I1022 18:42:26.855680 138 master.cpp:378] Flags at startup:
> --allocation_interval="1secs" --allocator="HierarchicalDRF"
> --authenticate="false" --authenticate_slaves="false"
> --authenticators="crammd5" --authorizers="local" --framework_sorter="drf"
> --help="false" --hostname_lookup="true" --initialize_driver_logging="true"
> --log_auto_initialize="true" --logbufsecs="0" --logging_level="INFO"
> --max_slave_ping_timeouts="5" --quiet="false"
> --recovery_slave_removal_limit="100%" --registry="replicated_log"
> --registry_fetch_timeout="1mins" --registry_store_timeout="5secs"
> --registry_strict="false" --root_submissions="true"
> --slave_ping_timeout="15secs" --slave_reregister_timeout="10mins"
> --user_sorter="drf" --version="false" --webui_dir="/usr/share/mesos/webui"
> --work_dir="/tmp/mesos/local/AK0XpG" --zk_session_timeout="10secs"
> I1022 18:42:26.855790 138 master.cpp:425] Master allowing unauthenticated
> frameworks to register
> I1022 18:42:26.855803 138 master.cpp:430] Master allowing unauthenticated
> slaves to register
> I1022 18:42:26.855815 138 master.cpp:467] Using default 'crammd5'
> authenticator
> W1022 18:42:26.855829 138 authenticator.cpp:505] No credentials provided,
> authentication requests will be refused
> I1022 18:42:26.855840 138 authenticator.cpp:512] Initializing server SASL
> I1022 18:42:26.856442 136 containerizer.cpp:143] Using isolation:
> posix/cpu,posix/mem,filesystem/posix
> I1022 18:42:26.856943 140 leveldb.cpp:306] Persisting metadata (8 bytes) to
> leveldb took 1.888185ms
> I1022 18:42:26.856987 140 replica.cpp:323] Persisted replica status to
> STARTING
> I1022 18:42:26.857115 140 recover.cpp:475] Replica is in STARTING status
> I1022 18:42:26.857270 140 replica.cpp:641] Replica in STARTING status
> received a broadcasted recover request
> I1022 18:42:26.857312 140 recover.cpp:195] Received a recover response from
> a replica in STARTING status
> I1022 18:42:26.857368 140 recover.cpp:566] Updating replica status to VOTING
> I1022 18:42:26.857781 140 leveldb.cpp:306] Persisting metadata (8 bytes) to
> leveldb took 371121ns
> I1022 18:42:26.857841 140 replica.cpp:323] Persisted replica status to
> VOTING
> I1022 18:42:26.857895 140 recover.cpp:580] Successfully joined the Paxos
> group
> I1022 18:42:26.857928 140 recover.cpp:464] Recover process terminated
> I1022 18:42:26.862455 137 master.cpp:1603] The newly elected leader is
> master@172.17.0.14:5050 with id a3f39818-1bda-4710-b96b-2a60ed4d12b8
> I1022 18:42:26.862498 137 master.cpp:1616] Elected as the leading master!
> I1022 18:42:26.862511 137 master.cpp:1376] Recovering from registrar
> I1022 18:42:26.862560 137 registrar.cpp:309] Recovering registrar
> Failed to create a containerizer: Could not create M
[jira] [Commented] (MESOS-3793) Cannot start mesos local on a Debian GNU/Linux 8 docker machine
[
https://issues.apache.org/jira/browse/MESOS-3793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15014039#comment-15014039
]
Matthias Veit commented on MESOS-3793:
--
Hey [~jojy] we can apply this workaround and it will work for our integration
tests. Thanks.
> Cannot start mesos local on a Debian GNU/Linux 8 docker machine
> ---
>
> Key: MESOS-3793
> URL: https://issues.apache.org/jira/browse/MESOS-3793
> Project: Mesos
> Issue Type: Bug
>Affects Versions: 0.25.0
> Environment: Debian GNU/Linux 8 docker machine
>Reporter: Matthias Veit
>Assignee: Jojy Varghese
> Labels: mesosphere
>
> We updated the mesos version to 0.25.0 in our Marathon docker image, that
> runs our integration tests.
> We use mesos local for those tests. This fails with this message:
> {noformat}
> root@a06e4b4eb776:/marathon# mesos local
> I1022 18:42:26.852485 136 leveldb.cpp:176] Opened db in 6.103258ms
> I1022 18:42:26.853302 136 leveldb.cpp:183] Compacted db in 765740ns
> I1022 18:42:26.853343 136 leveldb.cpp:198] Created db iterator in 9001ns
> I1022 18:42:26.853355 136 leveldb.cpp:204] Seeked to beginning of db in
> 1287ns
> I1022 18:42:26.853366 136 leveldb.cpp:273] Iterated through 0 keys in the
> db in ns
> I1022 18:42:26.853406 136 replica.cpp:744] Replica recovered with log
> positions 0 -> 0 with 1 holes and 0 unlearned
> I1022 18:42:26.853775 141 recover.cpp:449] Starting replica recovery
> I1022 18:42:26.853862 141 recover.cpp:475] Replica is in EMPTY status
> I1022 18:42:26.854751 138 replica.cpp:641] Replica in EMPTY status received
> a broadcasted recover request
> I1022 18:42:26.854856 140 recover.cpp:195] Received a recover response from
> a replica in EMPTY status
> I1022 18:42:26.855002 140 recover.cpp:566] Updating replica status to
> STARTING
> I1022 18:42:26.855655 138 master.cpp:376] Master
> a3f39818-1bda-4710-b96b-2a60ed4d12b8 (a06e4b4eb776) started on
> 172.17.0.14:5050
> I1022 18:42:26.855680 138 master.cpp:378] Flags at startup:
> --allocation_interval="1secs" --allocator="HierarchicalDRF"
> --authenticate="false" --authenticate_slaves="false"
> --authenticators="crammd5" --authorizers="local" --framework_sorter="drf"
> --help="false" --hostname_lookup="true" --initialize_driver_logging="true"
> --log_auto_initialize="true" --logbufsecs="0" --logging_level="INFO"
> --max_slave_ping_timeouts="5" --quiet="false"
> --recovery_slave_removal_limit="100%" --registry="replicated_log"
> --registry_fetch_timeout="1mins" --registry_store_timeout="5secs"
> --registry_strict="false" --root_submissions="true"
> --slave_ping_timeout="15secs" --slave_reregister_timeout="10mins"
> --user_sorter="drf" --version="false" --webui_dir="/usr/share/mesos/webui"
> --work_dir="/tmp/mesos/local/AK0XpG" --zk_session_timeout="10secs"
> I1022 18:42:26.855790 138 master.cpp:425] Master allowing unauthenticated
> frameworks to register
> I1022 18:42:26.855803 138 master.cpp:430] Master allowing unauthenticated
> slaves to register
> I1022 18:42:26.855815 138 master.cpp:467] Using default 'crammd5'
> authenticator
> W1022 18:42:26.855829 138 authenticator.cpp:505] No credentials provided,
> authentication requests will be refused
> I1022 18:42:26.855840 138 authenticator.cpp:512] Initializing server SASL
> I1022 18:42:26.856442 136 containerizer.cpp:143] Using isolation:
> posix/cpu,posix/mem,filesystem/posix
> I1022 18:42:26.856943 140 leveldb.cpp:306] Persisting metadata (8 bytes) to
> leveldb took 1.888185ms
> I1022 18:42:26.856987 140 replica.cpp:323] Persisted replica status to
> STARTING
> I1022 18:42:26.857115 140 recover.cpp:475] Replica is in STARTING status
> I1022 18:42:26.857270 140 replica.cpp:641] Replica in STARTING status
> received a broadcasted recover request
> I1022 18:42:26.857312 140 recover.cpp:195] Received a recover response from
> a replica in STARTING status
> I1022 18:42:26.857368 140 recover.cpp:566] Updating replica status to VOTING
> I1022 18:42:26.857781 140 leveldb.cpp:306] Persisting metadata (8 bytes) to
> leveldb took 371121ns
> I1022 18:42:26.857841 140 replica.cpp:323] Persisted replica status to
> VOTING
> I1022 18:42:26.857895 140 recover.cpp:580] Successfully joined the Paxos
> group
> I1022 18:42:26.857928 140 recover.cpp:464] Recover process terminated
> I1022 18:42:26.862455 137 master.cpp:1603] The newly elected leader is
> master@172.17.0.14:5050 with id a3f39818-1bda-4710-b96b-2a60ed4d12b8
> I1022 18:42:26.862498 137 master.cpp:1616] Elected as the leading master!
> I1022 18:42:26.862511 137 master.cpp:1376] Recovering from registrar
> I1022 18:42:26.862560 137 registrar.cpp:309] Recovering registrar
> Failed to create a containerizer: Could
[jira] [Commented] (MESOS-3793) Cannot start mesos local on a Debian GNU/Linux 8 docker machine
[
https://issues.apache.org/jira/browse/MESOS-3793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15013996#comment-15013996
]
Jojy Varghese commented on MESOS-3793:
--
Hi [~aquamatthias]
Thanks for the repro steps. I could reproduce the issue on my ubuntu 14.14
with your steps. I added env variable MESOS_LAUNCHER=posix and the issue went
away. Did you try this ?
-Jojy
> Cannot start mesos local on a Debian GNU/Linux 8 docker machine
> ---
>
> Key: MESOS-3793
> URL: https://issues.apache.org/jira/browse/MESOS-3793
> Project: Mesos
> Issue Type: Bug
>Affects Versions: 0.25.0
> Environment: Debian GNU/Linux 8 docker machine
>Reporter: Matthias Veit
>Assignee: Jojy Varghese
> Labels: mesosphere
>
> We updated the mesos version to 0.25.0 in our Marathon docker image, that
> runs our integration tests.
> We use mesos local for those tests. This fails with this message:
> {noformat}
> root@a06e4b4eb776:/marathon# mesos local
> I1022 18:42:26.852485 136 leveldb.cpp:176] Opened db in 6.103258ms
> I1022 18:42:26.853302 136 leveldb.cpp:183] Compacted db in 765740ns
> I1022 18:42:26.853343 136 leveldb.cpp:198] Created db iterator in 9001ns
> I1022 18:42:26.853355 136 leveldb.cpp:204] Seeked to beginning of db in
> 1287ns
> I1022 18:42:26.853366 136 leveldb.cpp:273] Iterated through 0 keys in the
> db in ns
> I1022 18:42:26.853406 136 replica.cpp:744] Replica recovered with log
> positions 0 -> 0 with 1 holes and 0 unlearned
> I1022 18:42:26.853775 141 recover.cpp:449] Starting replica recovery
> I1022 18:42:26.853862 141 recover.cpp:475] Replica is in EMPTY status
> I1022 18:42:26.854751 138 replica.cpp:641] Replica in EMPTY status received
> a broadcasted recover request
> I1022 18:42:26.854856 140 recover.cpp:195] Received a recover response from
> a replica in EMPTY status
> I1022 18:42:26.855002 140 recover.cpp:566] Updating replica status to
> STARTING
> I1022 18:42:26.855655 138 master.cpp:376] Master
> a3f39818-1bda-4710-b96b-2a60ed4d12b8 (a06e4b4eb776) started on
> 172.17.0.14:5050
> I1022 18:42:26.855680 138 master.cpp:378] Flags at startup:
> --allocation_interval="1secs" --allocator="HierarchicalDRF"
> --authenticate="false" --authenticate_slaves="false"
> --authenticators="crammd5" --authorizers="local" --framework_sorter="drf"
> --help="false" --hostname_lookup="true" --initialize_driver_logging="true"
> --log_auto_initialize="true" --logbufsecs="0" --logging_level="INFO"
> --max_slave_ping_timeouts="5" --quiet="false"
> --recovery_slave_removal_limit="100%" --registry="replicated_log"
> --registry_fetch_timeout="1mins" --registry_store_timeout="5secs"
> --registry_strict="false" --root_submissions="true"
> --slave_ping_timeout="15secs" --slave_reregister_timeout="10mins"
> --user_sorter="drf" --version="false" --webui_dir="/usr/share/mesos/webui"
> --work_dir="/tmp/mesos/local/AK0XpG" --zk_session_timeout="10secs"
> I1022 18:42:26.855790 138 master.cpp:425] Master allowing unauthenticated
> frameworks to register
> I1022 18:42:26.855803 138 master.cpp:430] Master allowing unauthenticated
> slaves to register
> I1022 18:42:26.855815 138 master.cpp:467] Using default 'crammd5'
> authenticator
> W1022 18:42:26.855829 138 authenticator.cpp:505] No credentials provided,
> authentication requests will be refused
> I1022 18:42:26.855840 138 authenticator.cpp:512] Initializing server SASL
> I1022 18:42:26.856442 136 containerizer.cpp:143] Using isolation:
> posix/cpu,posix/mem,filesystem/posix
> I1022 18:42:26.856943 140 leveldb.cpp:306] Persisting metadata (8 bytes) to
> leveldb took 1.888185ms
> I1022 18:42:26.856987 140 replica.cpp:323] Persisted replica status to
> STARTING
> I1022 18:42:26.857115 140 recover.cpp:475] Replica is in STARTING status
> I1022 18:42:26.857270 140 replica.cpp:641] Replica in STARTING status
> received a broadcasted recover request
> I1022 18:42:26.857312 140 recover.cpp:195] Received a recover response from
> a replica in STARTING status
> I1022 18:42:26.857368 140 recover.cpp:566] Updating replica status to VOTING
> I1022 18:42:26.857781 140 leveldb.cpp:306] Persisting metadata (8 bytes) to
> leveldb took 371121ns
> I1022 18:42:26.857841 140 replica.cpp:323] Persisted replica status to
> VOTING
> I1022 18:42:26.857895 140 recover.cpp:580] Successfully joined the Paxos
> group
> I1022 18:42:26.857928 140 recover.cpp:464] Recover process terminated
> I1022 18:42:26.862455 137 master.cpp:1603] The newly elected leader is
> master@172.17.0.14:5050 with id a3f39818-1bda-4710-b96b-2a60ed4d12b8
> I1022 18:42:26.862498 137 master.cpp:1616] Elected as the leading master!
> I1022 18:42:26.862511 137 master.cpp:1376] Recovering from reg
[jira] [Commented] (MESOS-3793) Cannot start mesos local on a Debian GNU/Linux 8 docker machine
[
https://issues.apache.org/jira/browse/MESOS-3793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15013127#comment-15013127
]
Matthias Veit commented on MESOS-3793:
--
Hey Jojy,
don't know if this is related to the docker linux host os. I tried several
different ones all with the same result. Also on Ubuntu 14.04.
Steps to reproduce:
{noformat}
$> docker run -it java:8-jdk /bin/bash
$> apt-key adv --keyserver keyserver.ubuntu.com --recv E56151BF && \
echo "deb http://repos.mesosphere.io/debian jessie main" | tee
/etc/apt/sources.list.d/mesosphere.list && \
echo "deb http://dl.bintray.com/sbt/debian /" | tee -a
/etc/apt/sources.list.d/sbt.list && \
apt-get update && \
apt-get install --no-install-recommends -y --force-yes
mesos=0.25.0-0.2.70.debian81
$> mesos local
{noformat}
After doing this, you should see the very same result.
> Cannot start mesos local on a Debian GNU/Linux 8 docker machine
> ---
>
> Key: MESOS-3793
> URL: https://issues.apache.org/jira/browse/MESOS-3793
> Project: Mesos
> Issue Type: Bug
>Affects Versions: 0.25.0
> Environment: Debian GNU/Linux 8 docker machine
>Reporter: Matthias Veit
>Assignee: Jojy Varghese
> Labels: mesosphere
>
> We updated the mesos version to 0.25.0 in our Marathon docker image, that
> runs our integration tests.
> We use mesos local for those tests. This fails with this message:
> {noformat}
> root@a06e4b4eb776:/marathon# mesos local
> I1022 18:42:26.852485 136 leveldb.cpp:176] Opened db in 6.103258ms
> I1022 18:42:26.853302 136 leveldb.cpp:183] Compacted db in 765740ns
> I1022 18:42:26.853343 136 leveldb.cpp:198] Created db iterator in 9001ns
> I1022 18:42:26.853355 136 leveldb.cpp:204] Seeked to beginning of db in
> 1287ns
> I1022 18:42:26.853366 136 leveldb.cpp:273] Iterated through 0 keys in the
> db in ns
> I1022 18:42:26.853406 136 replica.cpp:744] Replica recovered with log
> positions 0 -> 0 with 1 holes and 0 unlearned
> I1022 18:42:26.853775 141 recover.cpp:449] Starting replica recovery
> I1022 18:42:26.853862 141 recover.cpp:475] Replica is in EMPTY status
> I1022 18:42:26.854751 138 replica.cpp:641] Replica in EMPTY status received
> a broadcasted recover request
> I1022 18:42:26.854856 140 recover.cpp:195] Received a recover response from
> a replica in EMPTY status
> I1022 18:42:26.855002 140 recover.cpp:566] Updating replica status to
> STARTING
> I1022 18:42:26.855655 138 master.cpp:376] Master
> a3f39818-1bda-4710-b96b-2a60ed4d12b8 (a06e4b4eb776) started on
> 172.17.0.14:5050
> I1022 18:42:26.855680 138 master.cpp:378] Flags at startup:
> --allocation_interval="1secs" --allocator="HierarchicalDRF"
> --authenticate="false" --authenticate_slaves="false"
> --authenticators="crammd5" --authorizers="local" --framework_sorter="drf"
> --help="false" --hostname_lookup="true" --initialize_driver_logging="true"
> --log_auto_initialize="true" --logbufsecs="0" --logging_level="INFO"
> --max_slave_ping_timeouts="5" --quiet="false"
> --recovery_slave_removal_limit="100%" --registry="replicated_log"
> --registry_fetch_timeout="1mins" --registry_store_timeout="5secs"
> --registry_strict="false" --root_submissions="true"
> --slave_ping_timeout="15secs" --slave_reregister_timeout="10mins"
> --user_sorter="drf" --version="false" --webui_dir="/usr/share/mesos/webui"
> --work_dir="/tmp/mesos/local/AK0XpG" --zk_session_timeout="10secs"
> I1022 18:42:26.855790 138 master.cpp:425] Master allowing unauthenticated
> frameworks to register
> I1022 18:42:26.855803 138 master.cpp:430] Master allowing unauthenticated
> slaves to register
> I1022 18:42:26.855815 138 master.cpp:467] Using default 'crammd5'
> authenticator
> W1022 18:42:26.855829 138 authenticator.cpp:505] No credentials provided,
> authentication requests will be refused
> I1022 18:42:26.855840 138 authenticator.cpp:512] Initializing server SASL
> I1022 18:42:26.856442 136 containerizer.cpp:143] Using isolation:
> posix/cpu,posix/mem,filesystem/posix
> I1022 18:42:26.856943 140 leveldb.cpp:306] Persisting metadata (8 bytes) to
> leveldb took 1.888185ms
> I1022 18:42:26.856987 140 replica.cpp:323] Persisted replica status to
> STARTING
> I1022 18:42:26.857115 140 recover.cpp:475] Replica is in STARTING status
> I1022 18:42:26.857270 140 replica.cpp:641] Replica in STARTING status
> received a broadcasted recover request
> I1022 18:42:26.857312 140 recover.cpp:195] Received a recover response from
> a replica in STARTING status
> I1022 18:42:26.857368 140 recover.cpp:566] Updating replica status to VOTING
> I1022 18:42:26.857781 140 leveldb.cpp:306] Persisting metadata (8 bytes) to
> leveldb took 371121ns
> I1022 18:42:26.857841 140 replica.cpp:323] Pers
[jira] [Commented] (MESOS-3793) Cannot start mesos local on a Debian GNU/Linux 8 docker machine
[
https://issues.apache.org/jira/browse/MESOS-3793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15012311#comment-15012311
]
Jojy Varghese commented on MESOS-3793:
--
[~aquamatthias]: Not sure about the latest status of this issue. I tried
reproducing it on my ubuntu 14.04 but without much success. I used the
Dockerfile you provided.
My slave start flags are:
{quote}
Flags at startup: --appc_store_dir="/tmp/mesos/store/appc"
--authenticatee="crammd5" --cgroups_cpu_enable_pids_and_tids_count="false"
--cgroups_enable_cfs="false" --cgroups_hierarchy="/sys/fs/cgroup"
--cgroups_limit_swap="false" --cgroups_root="mesos"
--container_disk_watch_interval="15secs" --containerizers="mesos"
--default_role="*" --disk_watch_interval="1mins" --docker="docker"
--docker_kill_orphans="true" --docker_remove_delay="6hrs"
--docker_socket="/var/run/docker.sock" --docker_stop_timeout="0ns"
--enforce_container_disk_quota="false" --executor_registration_timeout="5mins"
--executor_shutdown_grace_period="5secs" --fetcher_cache_dir="/tmp/mesos/fetch"
--fetcher_cache_size="2GB" --frameworks_home="" --gc_delay="1weeks"
--gc_disk_headroom="0.1" --hadoop_home="" --help="false"
--hostname_lookup="true" --image_provisioner_backend="copy"
--initialize_driver_logging="true" --isolation="posix/cpu,posix/mem"
--launcher="posix" --launcher_dir="/usr/libexec/mesos" --logbufsecs="0"
--logging_level="INFO" --master="192.168.87.227:5050"
--oversubscribed_resources_interval="15secs" --perf_duration="10secs"
--perf_interval="1mins" --port="5051" --qos_correction_interval_min="0ns"
--quiet="false" --recover="reconnect" --recovery_timeout="15mins"
--registration_backoff_factor="1secs" --resource_monitoring_interval="1secs"
--revocable_cpu_low_priority="true" --sandbox_directory="/mnt/mesos/sandbox"
--strict="true" --switch_user="false"
--systemd_runtime_directory="/run/systemd/system" --version="false"
--work_dir="/tmp/mesos"
{quote}
Could you post the start flags on the slave instance you are having issues?
> Cannot start mesos local on a Debian GNU/Linux 8 docker machine
> ---
>
> Key: MESOS-3793
> URL: https://issues.apache.org/jira/browse/MESOS-3793
> Project: Mesos
> Issue Type: Bug
>Affects Versions: 0.25.0
> Environment: Debian GNU/Linux 8 docker machine
>Reporter: Matthias Veit
>Assignee: Jojy Varghese
> Labels: mesosphere
>
> We updated the mesos version to 0.25.0 in our Marathon docker image, that
> runs our integration tests.
> We use mesos local for those tests. This fails with this message:
> {noformat}
> root@a06e4b4eb776:/marathon# mesos local
> I1022 18:42:26.852485 136 leveldb.cpp:176] Opened db in 6.103258ms
> I1022 18:42:26.853302 136 leveldb.cpp:183] Compacted db in 765740ns
> I1022 18:42:26.853343 136 leveldb.cpp:198] Created db iterator in 9001ns
> I1022 18:42:26.853355 136 leveldb.cpp:204] Seeked to beginning of db in
> 1287ns
> I1022 18:42:26.853366 136 leveldb.cpp:273] Iterated through 0 keys in the
> db in ns
> I1022 18:42:26.853406 136 replica.cpp:744] Replica recovered with log
> positions 0 -> 0 with 1 holes and 0 unlearned
> I1022 18:42:26.853775 141 recover.cpp:449] Starting replica recovery
> I1022 18:42:26.853862 141 recover.cpp:475] Replica is in EMPTY status
> I1022 18:42:26.854751 138 replica.cpp:641] Replica in EMPTY status received
> a broadcasted recover request
> I1022 18:42:26.854856 140 recover.cpp:195] Received a recover response from
> a replica in EMPTY status
> I1022 18:42:26.855002 140 recover.cpp:566] Updating replica status to
> STARTING
> I1022 18:42:26.855655 138 master.cpp:376] Master
> a3f39818-1bda-4710-b96b-2a60ed4d12b8 (a06e4b4eb776) started on
> 172.17.0.14:5050
> I1022 18:42:26.855680 138 master.cpp:378] Flags at startup:
> --allocation_interval="1secs" --allocator="HierarchicalDRF"
> --authenticate="false" --authenticate_slaves="false"
> --authenticators="crammd5" --authorizers="local" --framework_sorter="drf"
> --help="false" --hostname_lookup="true" --initialize_driver_logging="true"
> --log_auto_initialize="true" --logbufsecs="0" --logging_level="INFO"
> --max_slave_ping_timeouts="5" --quiet="false"
> --recovery_slave_removal_limit="100%" --registry="replicated_log"
> --registry_fetch_timeout="1mins" --registry_store_timeout="5secs"
> --registry_strict="false" --root_submissions="true"
> --slave_ping_timeout="15secs" --slave_reregister_timeout="10mins"
> --user_sorter="drf" --version="false" --webui_dir="/usr/share/mesos/webui"
> --work_dir="/tmp/mesos/local/AK0XpG" --zk_session_timeout="10secs"
> I1022 18:42:26.855790 138 master.cpp:425] Master allowing unauthenticated
> frameworks to register
> I1022 18:42:26.855803 138 master.cpp:430] Master allowing unau
[jira] [Commented] (MESOS-3793) Cannot start mesos local on a Debian GNU/Linux 8 docker machine
[
https://issues.apache.org/jira/browse/MESOS-3793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15011785#comment-15011785
]
Jie Yu commented on MESOS-3793:
---
Is it possible to add MESOS_LAUNCHER=posix to the startup script?
There is some recent change that tries to use linux launcher whenever possible
(running under linux with root permission). Previously, posix launcher is
always used for local cluster.
> Cannot start mesos local on a Debian GNU/Linux 8 docker machine
> ---
>
> Key: MESOS-3793
> URL: https://issues.apache.org/jira/browse/MESOS-3793
> Project: Mesos
> Issue Type: Bug
>Affects Versions: 0.25.0
> Environment: Debian GNU/Linux 8 docker machine
>Reporter: Matthias Veit
>Assignee: Jojy Varghese
> Labels: mesosphere
>
> We updated the mesos version to 0.25.0 in our Marathon docker image, that
> runs our integration tests.
> We use mesos local for those tests. This fails with this message:
> {noformat}
> root@a06e4b4eb776:/marathon# mesos local
> I1022 18:42:26.852485 136 leveldb.cpp:176] Opened db in 6.103258ms
> I1022 18:42:26.853302 136 leveldb.cpp:183] Compacted db in 765740ns
> I1022 18:42:26.853343 136 leveldb.cpp:198] Created db iterator in 9001ns
> I1022 18:42:26.853355 136 leveldb.cpp:204] Seeked to beginning of db in
> 1287ns
> I1022 18:42:26.853366 136 leveldb.cpp:273] Iterated through 0 keys in the
> db in ns
> I1022 18:42:26.853406 136 replica.cpp:744] Replica recovered with log
> positions 0 -> 0 with 1 holes and 0 unlearned
> I1022 18:42:26.853775 141 recover.cpp:449] Starting replica recovery
> I1022 18:42:26.853862 141 recover.cpp:475] Replica is in EMPTY status
> I1022 18:42:26.854751 138 replica.cpp:641] Replica in EMPTY status received
> a broadcasted recover request
> I1022 18:42:26.854856 140 recover.cpp:195] Received a recover response from
> a replica in EMPTY status
> I1022 18:42:26.855002 140 recover.cpp:566] Updating replica status to
> STARTING
> I1022 18:42:26.855655 138 master.cpp:376] Master
> a3f39818-1bda-4710-b96b-2a60ed4d12b8 (a06e4b4eb776) started on
> 172.17.0.14:5050
> I1022 18:42:26.855680 138 master.cpp:378] Flags at startup:
> --allocation_interval="1secs" --allocator="HierarchicalDRF"
> --authenticate="false" --authenticate_slaves="false"
> --authenticators="crammd5" --authorizers="local" --framework_sorter="drf"
> --help="false" --hostname_lookup="true" --initialize_driver_logging="true"
> --log_auto_initialize="true" --logbufsecs="0" --logging_level="INFO"
> --max_slave_ping_timeouts="5" --quiet="false"
> --recovery_slave_removal_limit="100%" --registry="replicated_log"
> --registry_fetch_timeout="1mins" --registry_store_timeout="5secs"
> --registry_strict="false" --root_submissions="true"
> --slave_ping_timeout="15secs" --slave_reregister_timeout="10mins"
> --user_sorter="drf" --version="false" --webui_dir="/usr/share/mesos/webui"
> --work_dir="/tmp/mesos/local/AK0XpG" --zk_session_timeout="10secs"
> I1022 18:42:26.855790 138 master.cpp:425] Master allowing unauthenticated
> frameworks to register
> I1022 18:42:26.855803 138 master.cpp:430] Master allowing unauthenticated
> slaves to register
> I1022 18:42:26.855815 138 master.cpp:467] Using default 'crammd5'
> authenticator
> W1022 18:42:26.855829 138 authenticator.cpp:505] No credentials provided,
> authentication requests will be refused
> I1022 18:42:26.855840 138 authenticator.cpp:512] Initializing server SASL
> I1022 18:42:26.856442 136 containerizer.cpp:143] Using isolation:
> posix/cpu,posix/mem,filesystem/posix
> I1022 18:42:26.856943 140 leveldb.cpp:306] Persisting metadata (8 bytes) to
> leveldb took 1.888185ms
> I1022 18:42:26.856987 140 replica.cpp:323] Persisted replica status to
> STARTING
> I1022 18:42:26.857115 140 recover.cpp:475] Replica is in STARTING status
> I1022 18:42:26.857270 140 replica.cpp:641] Replica in STARTING status
> received a broadcasted recover request
> I1022 18:42:26.857312 140 recover.cpp:195] Received a recover response from
> a replica in STARTING status
> I1022 18:42:26.857368 140 recover.cpp:566] Updating replica status to VOTING
> I1022 18:42:26.857781 140 leveldb.cpp:306] Persisting metadata (8 bytes) to
> leveldb took 371121ns
> I1022 18:42:26.857841 140 replica.cpp:323] Persisted replica status to
> VOTING
> I1022 18:42:26.857895 140 recover.cpp:580] Successfully joined the Paxos
> group
> I1022 18:42:26.857928 140 recover.cpp:464] Recover process terminated
> I1022 18:42:26.862455 137 master.cpp:1603] The newly elected leader is
> master@172.17.0.14:5050 with id a3f39818-1bda-4710-b96b-2a60ed4d12b8
> I1022 18:42:26.862498 137 master.cpp:1616] Elected as the leading master!
> I1022 18:42:26.862511 137 master.cpp
[jira] [Commented] (MESOS-3793) Cannot start mesos local on a Debian GNU/Linux 8 docker machine
[
https://issues.apache.org/jira/browse/MESOS-3793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15011371#comment-15011371
]
Jojy Varghese commented on MESOS-3793:
--
Since we are debugging the issue, this is just something we will do for now to
understand the problem.
> Cannot start mesos local on a Debian GNU/Linux 8 docker machine
> ---
>
> Key: MESOS-3793
> URL: https://issues.apache.org/jira/browse/MESOS-3793
> Project: Mesos
> Issue Type: Bug
>Affects Versions: 0.25.0
> Environment: Debian GNU/Linux 8 docker machine
>Reporter: Matthias Veit
>Assignee: Jojy Varghese
> Labels: mesosphere
>
> We updated the mesos version to 0.25.0 in our Marathon docker image, that
> runs our integration tests.
> We use mesos local for those tests. This fails with this message:
> {noformat}
> root@a06e4b4eb776:/marathon# mesos local
> I1022 18:42:26.852485 136 leveldb.cpp:176] Opened db in 6.103258ms
> I1022 18:42:26.853302 136 leveldb.cpp:183] Compacted db in 765740ns
> I1022 18:42:26.853343 136 leveldb.cpp:198] Created db iterator in 9001ns
> I1022 18:42:26.853355 136 leveldb.cpp:204] Seeked to beginning of db in
> 1287ns
> I1022 18:42:26.853366 136 leveldb.cpp:273] Iterated through 0 keys in the
> db in ns
> I1022 18:42:26.853406 136 replica.cpp:744] Replica recovered with log
> positions 0 -> 0 with 1 holes and 0 unlearned
> I1022 18:42:26.853775 141 recover.cpp:449] Starting replica recovery
> I1022 18:42:26.853862 141 recover.cpp:475] Replica is in EMPTY status
> I1022 18:42:26.854751 138 replica.cpp:641] Replica in EMPTY status received
> a broadcasted recover request
> I1022 18:42:26.854856 140 recover.cpp:195] Received a recover response from
> a replica in EMPTY status
> I1022 18:42:26.855002 140 recover.cpp:566] Updating replica status to
> STARTING
> I1022 18:42:26.855655 138 master.cpp:376] Master
> a3f39818-1bda-4710-b96b-2a60ed4d12b8 (a06e4b4eb776) started on
> 172.17.0.14:5050
> I1022 18:42:26.855680 138 master.cpp:378] Flags at startup:
> --allocation_interval="1secs" --allocator="HierarchicalDRF"
> --authenticate="false" --authenticate_slaves="false"
> --authenticators="crammd5" --authorizers="local" --framework_sorter="drf"
> --help="false" --hostname_lookup="true" --initialize_driver_logging="true"
> --log_auto_initialize="true" --logbufsecs="0" --logging_level="INFO"
> --max_slave_ping_timeouts="5" --quiet="false"
> --recovery_slave_removal_limit="100%" --registry="replicated_log"
> --registry_fetch_timeout="1mins" --registry_store_timeout="5secs"
> --registry_strict="false" --root_submissions="true"
> --slave_ping_timeout="15secs" --slave_reregister_timeout="10mins"
> --user_sorter="drf" --version="false" --webui_dir="/usr/share/mesos/webui"
> --work_dir="/tmp/mesos/local/AK0XpG" --zk_session_timeout="10secs"
> I1022 18:42:26.855790 138 master.cpp:425] Master allowing unauthenticated
> frameworks to register
> I1022 18:42:26.855803 138 master.cpp:430] Master allowing unauthenticated
> slaves to register
> I1022 18:42:26.855815 138 master.cpp:467] Using default 'crammd5'
> authenticator
> W1022 18:42:26.855829 138 authenticator.cpp:505] No credentials provided,
> authentication requests will be refused
> I1022 18:42:26.855840 138 authenticator.cpp:512] Initializing server SASL
> I1022 18:42:26.856442 136 containerizer.cpp:143] Using isolation:
> posix/cpu,posix/mem,filesystem/posix
> I1022 18:42:26.856943 140 leveldb.cpp:306] Persisting metadata (8 bytes) to
> leveldb took 1.888185ms
> I1022 18:42:26.856987 140 replica.cpp:323] Persisted replica status to
> STARTING
> I1022 18:42:26.857115 140 recover.cpp:475] Replica is in STARTING status
> I1022 18:42:26.857270 140 replica.cpp:641] Replica in STARTING status
> received a broadcasted recover request
> I1022 18:42:26.857312 140 recover.cpp:195] Received a recover response from
> a replica in STARTING status
> I1022 18:42:26.857368 140 recover.cpp:566] Updating replica status to VOTING
> I1022 18:42:26.857781 140 leveldb.cpp:306] Persisting metadata (8 bytes) to
> leveldb took 371121ns
> I1022 18:42:26.857841 140 replica.cpp:323] Persisted replica status to
> VOTING
> I1022 18:42:26.857895 140 recover.cpp:580] Successfully joined the Paxos
> group
> I1022 18:42:26.857928 140 recover.cpp:464] Recover process terminated
> I1022 18:42:26.862455 137 master.cpp:1603] The newly elected leader is
> master@172.17.0.14:5050 with id a3f39818-1bda-4710-b96b-2a60ed4d12b8
> I1022 18:42:26.862498 137 master.cpp:1616] Elected as the leading master!
> I1022 18:42:26.862511 137 master.cpp:1376] Recovering from registrar
> I1022 18:42:26.862560 137 registrar.cpp:309] Recovering registrar
> Failed to create a containeriz
[jira] [Commented] (MESOS-3793) Cannot start mesos local on a Debian GNU/Linux 8 docker machine
[
https://issues.apache.org/jira/browse/MESOS-3793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15011312#comment-15011312
]
Matthias Veit commented on MESOS-3793:
--
I can do that. But do you expect people to do that?
We install mesos via apt-get install mesos and the run mesos.
Why should I do anything else?
> Cannot start mesos local on a Debian GNU/Linux 8 docker machine
> ---
>
> Key: MESOS-3793
> URL: https://issues.apache.org/jira/browse/MESOS-3793
> Project: Mesos
> Issue Type: Bug
>Affects Versions: 0.25.0
> Environment: Debian GNU/Linux 8 docker machine
>Reporter: Matthias Veit
>Assignee: Jojy Varghese
> Labels: mesosphere
>
> We updated the mesos version to 0.25.0 in our Marathon docker image, that
> runs our integration tests.
> We use mesos local for those tests. This fails with this message:
> {noformat}
> root@a06e4b4eb776:/marathon# mesos local
> I1022 18:42:26.852485 136 leveldb.cpp:176] Opened db in 6.103258ms
> I1022 18:42:26.853302 136 leveldb.cpp:183] Compacted db in 765740ns
> I1022 18:42:26.853343 136 leveldb.cpp:198] Created db iterator in 9001ns
> I1022 18:42:26.853355 136 leveldb.cpp:204] Seeked to beginning of db in
> 1287ns
> I1022 18:42:26.853366 136 leveldb.cpp:273] Iterated through 0 keys in the
> db in ns
> I1022 18:42:26.853406 136 replica.cpp:744] Replica recovered with log
> positions 0 -> 0 with 1 holes and 0 unlearned
> I1022 18:42:26.853775 141 recover.cpp:449] Starting replica recovery
> I1022 18:42:26.853862 141 recover.cpp:475] Replica is in EMPTY status
> I1022 18:42:26.854751 138 replica.cpp:641] Replica in EMPTY status received
> a broadcasted recover request
> I1022 18:42:26.854856 140 recover.cpp:195] Received a recover response from
> a replica in EMPTY status
> I1022 18:42:26.855002 140 recover.cpp:566] Updating replica status to
> STARTING
> I1022 18:42:26.855655 138 master.cpp:376] Master
> a3f39818-1bda-4710-b96b-2a60ed4d12b8 (a06e4b4eb776) started on
> 172.17.0.14:5050
> I1022 18:42:26.855680 138 master.cpp:378] Flags at startup:
> --allocation_interval="1secs" --allocator="HierarchicalDRF"
> --authenticate="false" --authenticate_slaves="false"
> --authenticators="crammd5" --authorizers="local" --framework_sorter="drf"
> --help="false" --hostname_lookup="true" --initialize_driver_logging="true"
> --log_auto_initialize="true" --logbufsecs="0" --logging_level="INFO"
> --max_slave_ping_timeouts="5" --quiet="false"
> --recovery_slave_removal_limit="100%" --registry="replicated_log"
> --registry_fetch_timeout="1mins" --registry_store_timeout="5secs"
> --registry_strict="false" --root_submissions="true"
> --slave_ping_timeout="15secs" --slave_reregister_timeout="10mins"
> --user_sorter="drf" --version="false" --webui_dir="/usr/share/mesos/webui"
> --work_dir="/tmp/mesos/local/AK0XpG" --zk_session_timeout="10secs"
> I1022 18:42:26.855790 138 master.cpp:425] Master allowing unauthenticated
> frameworks to register
> I1022 18:42:26.855803 138 master.cpp:430] Master allowing unauthenticated
> slaves to register
> I1022 18:42:26.855815 138 master.cpp:467] Using default 'crammd5'
> authenticator
> W1022 18:42:26.855829 138 authenticator.cpp:505] No credentials provided,
> authentication requests will be refused
> I1022 18:42:26.855840 138 authenticator.cpp:512] Initializing server SASL
> I1022 18:42:26.856442 136 containerizer.cpp:143] Using isolation:
> posix/cpu,posix/mem,filesystem/posix
> I1022 18:42:26.856943 140 leveldb.cpp:306] Persisting metadata (8 bytes) to
> leveldb took 1.888185ms
> I1022 18:42:26.856987 140 replica.cpp:323] Persisted replica status to
> STARTING
> I1022 18:42:26.857115 140 recover.cpp:475] Replica is in STARTING status
> I1022 18:42:26.857270 140 replica.cpp:641] Replica in STARTING status
> received a broadcasted recover request
> I1022 18:42:26.857312 140 recover.cpp:195] Received a recover response from
> a replica in STARTING status
> I1022 18:42:26.857368 140 recover.cpp:566] Updating replica status to VOTING
> I1022 18:42:26.857781 140 leveldb.cpp:306] Persisting metadata (8 bytes) to
> leveldb took 371121ns
> I1022 18:42:26.857841 140 replica.cpp:323] Persisted replica status to
> VOTING
> I1022 18:42:26.857895 140 recover.cpp:580] Successfully joined the Paxos
> group
> I1022 18:42:26.857928 140 recover.cpp:464] Recover process terminated
> I1022 18:42:26.862455 137 master.cpp:1603] The newly elected leader is
> master@172.17.0.14:5050 with id a3f39818-1bda-4710-b96b-2a60ed4d12b8
> I1022 18:42:26.862498 137 master.cpp:1616] Elected as the leading master!
> I1022 18:42:26.862511 137 master.cpp:1376] Recovering from registrar
> I1022 18:42:26.862560 137 registrar.cpp:309] Recovering r
[jira] [Commented] (MESOS-3793) Cannot start mesos local on a Debian GNU/Linux 8 docker machine
[
https://issues.apache.org/jira/browse/MESOS-3793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15011274#comment-15011274
]
Jojy Varghese commented on MESOS-3793:
--
Have you tried mounting /run/systemd also inside the container?
> Cannot start mesos local on a Debian GNU/Linux 8 docker machine
> ---
>
> Key: MESOS-3793
> URL: https://issues.apache.org/jira/browse/MESOS-3793
> Project: Mesos
> Issue Type: Bug
>Affects Versions: 0.25.0
> Environment: Debian GNU/Linux 8 docker machine
>Reporter: Matthias Veit
>Assignee: Jojy Varghese
> Labels: mesosphere
>
> We updated the mesos version to 0.25.0 in our Marathon docker image, that
> runs our integration tests.
> We use mesos local for those tests. This fails with this message:
> {noformat}
> root@a06e4b4eb776:/marathon# mesos local
> I1022 18:42:26.852485 136 leveldb.cpp:176] Opened db in 6.103258ms
> I1022 18:42:26.853302 136 leveldb.cpp:183] Compacted db in 765740ns
> I1022 18:42:26.853343 136 leveldb.cpp:198] Created db iterator in 9001ns
> I1022 18:42:26.853355 136 leveldb.cpp:204] Seeked to beginning of db in
> 1287ns
> I1022 18:42:26.853366 136 leveldb.cpp:273] Iterated through 0 keys in the
> db in ns
> I1022 18:42:26.853406 136 replica.cpp:744] Replica recovered with log
> positions 0 -> 0 with 1 holes and 0 unlearned
> I1022 18:42:26.853775 141 recover.cpp:449] Starting replica recovery
> I1022 18:42:26.853862 141 recover.cpp:475] Replica is in EMPTY status
> I1022 18:42:26.854751 138 replica.cpp:641] Replica in EMPTY status received
> a broadcasted recover request
> I1022 18:42:26.854856 140 recover.cpp:195] Received a recover response from
> a replica in EMPTY status
> I1022 18:42:26.855002 140 recover.cpp:566] Updating replica status to
> STARTING
> I1022 18:42:26.855655 138 master.cpp:376] Master
> a3f39818-1bda-4710-b96b-2a60ed4d12b8 (a06e4b4eb776) started on
> 172.17.0.14:5050
> I1022 18:42:26.855680 138 master.cpp:378] Flags at startup:
> --allocation_interval="1secs" --allocator="HierarchicalDRF"
> --authenticate="false" --authenticate_slaves="false"
> --authenticators="crammd5" --authorizers="local" --framework_sorter="drf"
> --help="false" --hostname_lookup="true" --initialize_driver_logging="true"
> --log_auto_initialize="true" --logbufsecs="0" --logging_level="INFO"
> --max_slave_ping_timeouts="5" --quiet="false"
> --recovery_slave_removal_limit="100%" --registry="replicated_log"
> --registry_fetch_timeout="1mins" --registry_store_timeout="5secs"
> --registry_strict="false" --root_submissions="true"
> --slave_ping_timeout="15secs" --slave_reregister_timeout="10mins"
> --user_sorter="drf" --version="false" --webui_dir="/usr/share/mesos/webui"
> --work_dir="/tmp/mesos/local/AK0XpG" --zk_session_timeout="10secs"
> I1022 18:42:26.855790 138 master.cpp:425] Master allowing unauthenticated
> frameworks to register
> I1022 18:42:26.855803 138 master.cpp:430] Master allowing unauthenticated
> slaves to register
> I1022 18:42:26.855815 138 master.cpp:467] Using default 'crammd5'
> authenticator
> W1022 18:42:26.855829 138 authenticator.cpp:505] No credentials provided,
> authentication requests will be refused
> I1022 18:42:26.855840 138 authenticator.cpp:512] Initializing server SASL
> I1022 18:42:26.856442 136 containerizer.cpp:143] Using isolation:
> posix/cpu,posix/mem,filesystem/posix
> I1022 18:42:26.856943 140 leveldb.cpp:306] Persisting metadata (8 bytes) to
> leveldb took 1.888185ms
> I1022 18:42:26.856987 140 replica.cpp:323] Persisted replica status to
> STARTING
> I1022 18:42:26.857115 140 recover.cpp:475] Replica is in STARTING status
> I1022 18:42:26.857270 140 replica.cpp:641] Replica in STARTING status
> received a broadcasted recover request
> I1022 18:42:26.857312 140 recover.cpp:195] Received a recover response from
> a replica in STARTING status
> I1022 18:42:26.857368 140 recover.cpp:566] Updating replica status to VOTING
> I1022 18:42:26.857781 140 leveldb.cpp:306] Persisting metadata (8 bytes) to
> leveldb took 371121ns
> I1022 18:42:26.857841 140 replica.cpp:323] Persisted replica status to
> VOTING
> I1022 18:42:26.857895 140 recover.cpp:580] Successfully joined the Paxos
> group
> I1022 18:42:26.857928 140 recover.cpp:464] Recover process terminated
> I1022 18:42:26.862455 137 master.cpp:1603] The newly elected leader is
> master@172.17.0.14:5050 with id a3f39818-1bda-4710-b96b-2a60ed4d12b8
> I1022 18:42:26.862498 137 master.cpp:1616] Elected as the leading master!
> I1022 18:42:26.862511 137 master.cpp:1376] Recovering from registrar
> I1022 18:42:26.862560 137 registrar.cpp:309] Recovering registrar
> Failed to create a containerizer: Could not create MesosContainerizer:
[jira] [Commented] (MESOS-3793) Cannot start mesos local on a Debian GNU/Linux 8 docker machine
[
https://issues.apache.org/jira/browse/MESOS-3793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15011004#comment-15011004
]
Matthias Veit commented on MESOS-3793:
--
Hey [~jojy] this is blocking Marathon from using Mesos 0.25.0 (we still
use/recommend 0.24). Given the fact, that 0.26 is around the corner we could
jump to that version. So the question is: is that problem solved with mesos
0.26? Will there be a fix for 0.25?
> Cannot start mesos local on a Debian GNU/Linux 8 docker machine
> ---
>
> Key: MESOS-3793
> URL: https://issues.apache.org/jira/browse/MESOS-3793
> Project: Mesos
> Issue Type: Bug
>Affects Versions: 0.25.0
> Environment: Debian GNU/Linux 8 docker machine
>Reporter: Matthias Veit
>Assignee: Jojy Varghese
> Labels: mesosphere
>
> We updated the mesos version to 0.25.0 in our Marathon docker image, that
> runs our integration tests.
> We use mesos local for those tests. This fails with this message:
> {noformat}
> root@a06e4b4eb776:/marathon# mesos local
> I1022 18:42:26.852485 136 leveldb.cpp:176] Opened db in 6.103258ms
> I1022 18:42:26.853302 136 leveldb.cpp:183] Compacted db in 765740ns
> I1022 18:42:26.853343 136 leveldb.cpp:198] Created db iterator in 9001ns
> I1022 18:42:26.853355 136 leveldb.cpp:204] Seeked to beginning of db in
> 1287ns
> I1022 18:42:26.853366 136 leveldb.cpp:273] Iterated through 0 keys in the
> db in ns
> I1022 18:42:26.853406 136 replica.cpp:744] Replica recovered with log
> positions 0 -> 0 with 1 holes and 0 unlearned
> I1022 18:42:26.853775 141 recover.cpp:449] Starting replica recovery
> I1022 18:42:26.853862 141 recover.cpp:475] Replica is in EMPTY status
> I1022 18:42:26.854751 138 replica.cpp:641] Replica in EMPTY status received
> a broadcasted recover request
> I1022 18:42:26.854856 140 recover.cpp:195] Received a recover response from
> a replica in EMPTY status
> I1022 18:42:26.855002 140 recover.cpp:566] Updating replica status to
> STARTING
> I1022 18:42:26.855655 138 master.cpp:376] Master
> a3f39818-1bda-4710-b96b-2a60ed4d12b8 (a06e4b4eb776) started on
> 172.17.0.14:5050
> I1022 18:42:26.855680 138 master.cpp:378] Flags at startup:
> --allocation_interval="1secs" --allocator="HierarchicalDRF"
> --authenticate="false" --authenticate_slaves="false"
> --authenticators="crammd5" --authorizers="local" --framework_sorter="drf"
> --help="false" --hostname_lookup="true" --initialize_driver_logging="true"
> --log_auto_initialize="true" --logbufsecs="0" --logging_level="INFO"
> --max_slave_ping_timeouts="5" --quiet="false"
> --recovery_slave_removal_limit="100%" --registry="replicated_log"
> --registry_fetch_timeout="1mins" --registry_store_timeout="5secs"
> --registry_strict="false" --root_submissions="true"
> --slave_ping_timeout="15secs" --slave_reregister_timeout="10mins"
> --user_sorter="drf" --version="false" --webui_dir="/usr/share/mesos/webui"
> --work_dir="/tmp/mesos/local/AK0XpG" --zk_session_timeout="10secs"
> I1022 18:42:26.855790 138 master.cpp:425] Master allowing unauthenticated
> frameworks to register
> I1022 18:42:26.855803 138 master.cpp:430] Master allowing unauthenticated
> slaves to register
> I1022 18:42:26.855815 138 master.cpp:467] Using default 'crammd5'
> authenticator
> W1022 18:42:26.855829 138 authenticator.cpp:505] No credentials provided,
> authentication requests will be refused
> I1022 18:42:26.855840 138 authenticator.cpp:512] Initializing server SASL
> I1022 18:42:26.856442 136 containerizer.cpp:143] Using isolation:
> posix/cpu,posix/mem,filesystem/posix
> I1022 18:42:26.856943 140 leveldb.cpp:306] Persisting metadata (8 bytes) to
> leveldb took 1.888185ms
> I1022 18:42:26.856987 140 replica.cpp:323] Persisted replica status to
> STARTING
> I1022 18:42:26.857115 140 recover.cpp:475] Replica is in STARTING status
> I1022 18:42:26.857270 140 replica.cpp:641] Replica in STARTING status
> received a broadcasted recover request
> I1022 18:42:26.857312 140 recover.cpp:195] Received a recover response from
> a replica in STARTING status
> I1022 18:42:26.857368 140 recover.cpp:566] Updating replica status to VOTING
> I1022 18:42:26.857781 140 leveldb.cpp:306] Persisting metadata (8 bytes) to
> leveldb took 371121ns
> I1022 18:42:26.857841 140 replica.cpp:323] Persisted replica status to
> VOTING
> I1022 18:42:26.857895 140 recover.cpp:580] Successfully joined the Paxos
> group
> I1022 18:42:26.857928 140 recover.cpp:464] Recover process terminated
> I1022 18:42:26.862455 137 master.cpp:1603] The newly elected leader is
> master@172.17.0.14:5050 with id a3f39818-1bda-4710-b96b-2a60ed4d12b8
> I1022 18:42:26.862498 137 master.cpp:1616] Elected as the leading master!
> I1022 18:42:2
[jira] [Commented] (MESOS-3793) Cannot start mesos local on a Debian GNU/Linux 8 docker machine
[
https://issues.apache.org/jira/browse/MESOS-3793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14989332#comment-14989332
]
Peter Kolloch commented on MESOS-3793:
--
I found this related CHANGELOG entry
(https://github.com/apache/mesos/blob/master/CHANGELOG#L109):
{code}
* [MESOS-3425] - Modify LinuxLauncher to support Systemd.
{code}
Maybe MESOS-3425 introduced a hard dependency on systemd utilities? Maybe
MESOS-1159 is about fixing that but I am not sure.
> Cannot start mesos local on a Debian GNU/Linux 8 docker machine
> ---
>
> Key: MESOS-3793
> URL: https://issues.apache.org/jira/browse/MESOS-3793
> Project: Mesos
> Issue Type: Bug
>Affects Versions: 0.25.0
> Environment: Debian GNU/Linux 8 docker machine
>Reporter: Matthias Veit
>Assignee: Jojy Varghese
> Labels: mesosphere
>
> We updated the mesos version to 0.25.0 in our Marathon docker image, that
> runs our integration tests.
> We use mesos local for those tests. This fails with this message:
> {noformat}
> root@a06e4b4eb776:/marathon# mesos local
> I1022 18:42:26.852485 136 leveldb.cpp:176] Opened db in 6.103258ms
> I1022 18:42:26.853302 136 leveldb.cpp:183] Compacted db in 765740ns
> I1022 18:42:26.853343 136 leveldb.cpp:198] Created db iterator in 9001ns
> I1022 18:42:26.853355 136 leveldb.cpp:204] Seeked to beginning of db in
> 1287ns
> I1022 18:42:26.853366 136 leveldb.cpp:273] Iterated through 0 keys in the
> db in ns
> I1022 18:42:26.853406 136 replica.cpp:744] Replica recovered with log
> positions 0 -> 0 with 1 holes and 0 unlearned
> I1022 18:42:26.853775 141 recover.cpp:449] Starting replica recovery
> I1022 18:42:26.853862 141 recover.cpp:475] Replica is in EMPTY status
> I1022 18:42:26.854751 138 replica.cpp:641] Replica in EMPTY status received
> a broadcasted recover request
> I1022 18:42:26.854856 140 recover.cpp:195] Received a recover response from
> a replica in EMPTY status
> I1022 18:42:26.855002 140 recover.cpp:566] Updating replica status to
> STARTING
> I1022 18:42:26.855655 138 master.cpp:376] Master
> a3f39818-1bda-4710-b96b-2a60ed4d12b8 (a06e4b4eb776) started on
> 172.17.0.14:5050
> I1022 18:42:26.855680 138 master.cpp:378] Flags at startup:
> --allocation_interval="1secs" --allocator="HierarchicalDRF"
> --authenticate="false" --authenticate_slaves="false"
> --authenticators="crammd5" --authorizers="local" --framework_sorter="drf"
> --help="false" --hostname_lookup="true" --initialize_driver_logging="true"
> --log_auto_initialize="true" --logbufsecs="0" --logging_level="INFO"
> --max_slave_ping_timeouts="5" --quiet="false"
> --recovery_slave_removal_limit="100%" --registry="replicated_log"
> --registry_fetch_timeout="1mins" --registry_store_timeout="5secs"
> --registry_strict="false" --root_submissions="true"
> --slave_ping_timeout="15secs" --slave_reregister_timeout="10mins"
> --user_sorter="drf" --version="false" --webui_dir="/usr/share/mesos/webui"
> --work_dir="/tmp/mesos/local/AK0XpG" --zk_session_timeout="10secs"
> I1022 18:42:26.855790 138 master.cpp:425] Master allowing unauthenticated
> frameworks to register
> I1022 18:42:26.855803 138 master.cpp:430] Master allowing unauthenticated
> slaves to register
> I1022 18:42:26.855815 138 master.cpp:467] Using default 'crammd5'
> authenticator
> W1022 18:42:26.855829 138 authenticator.cpp:505] No credentials provided,
> authentication requests will be refused
> I1022 18:42:26.855840 138 authenticator.cpp:512] Initializing server SASL
> I1022 18:42:26.856442 136 containerizer.cpp:143] Using isolation:
> posix/cpu,posix/mem,filesystem/posix
> I1022 18:42:26.856943 140 leveldb.cpp:306] Persisting metadata (8 bytes) to
> leveldb took 1.888185ms
> I1022 18:42:26.856987 140 replica.cpp:323] Persisted replica status to
> STARTING
> I1022 18:42:26.857115 140 recover.cpp:475] Replica is in STARTING status
> I1022 18:42:26.857270 140 replica.cpp:641] Replica in STARTING status
> received a broadcasted recover request
> I1022 18:42:26.857312 140 recover.cpp:195] Received a recover response from
> a replica in STARTING status
> I1022 18:42:26.857368 140 recover.cpp:566] Updating replica status to VOTING
> I1022 18:42:26.857781 140 leveldb.cpp:306] Persisting metadata (8 bytes) to
> leveldb took 371121ns
> I1022 18:42:26.857841 140 replica.cpp:323] Persisted replica status to
> VOTING
> I1022 18:42:26.857895 140 recover.cpp:580] Successfully joined the Paxos
> group
> I1022 18:42:26.857928 140 recover.cpp:464] Recover process terminated
> I1022 18:42:26.862455 137 master.cpp:1603] The newly elected leader is
> master@172.17.0.14:5050 with id a3f39818-1bda-4710-b96b-2a60ed4d12b8
> I1022 18:42:26.862498 137 master.cpp:1616] Elected
[jira] [Commented] (MESOS-3793) Cannot start mesos local on a Debian GNU/Linux 8 docker machine
[
https://issues.apache.org/jira/browse/MESOS-3793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14989308#comment-14989308
]
Peter Kolloch commented on MESOS-3793:
--
[~karlkfi]Is it correct that you encountered this problem, too? Did you find a
workaround?
> Cannot start mesos local on a Debian GNU/Linux 8 docker machine
> ---
>
> Key: MESOS-3793
> URL: https://issues.apache.org/jira/browse/MESOS-3793
> Project: Mesos
> Issue Type: Bug
>Affects Versions: 0.25.0
> Environment: Debian GNU/Linux 8 docker machine
>Reporter: Matthias Veit
>Assignee: Jojy Varghese
> Labels: mesosphere
>
> We updated the mesos version to 0.25.0 in our Marathon docker image, that
> runs our integration tests.
> We use mesos local for those tests. This fails with this message:
> {noformat}
> root@a06e4b4eb776:/marathon# mesos local
> I1022 18:42:26.852485 136 leveldb.cpp:176] Opened db in 6.103258ms
> I1022 18:42:26.853302 136 leveldb.cpp:183] Compacted db in 765740ns
> I1022 18:42:26.853343 136 leveldb.cpp:198] Created db iterator in 9001ns
> I1022 18:42:26.853355 136 leveldb.cpp:204] Seeked to beginning of db in
> 1287ns
> I1022 18:42:26.853366 136 leveldb.cpp:273] Iterated through 0 keys in the
> db in ns
> I1022 18:42:26.853406 136 replica.cpp:744] Replica recovered with log
> positions 0 -> 0 with 1 holes and 0 unlearned
> I1022 18:42:26.853775 141 recover.cpp:449] Starting replica recovery
> I1022 18:42:26.853862 141 recover.cpp:475] Replica is in EMPTY status
> I1022 18:42:26.854751 138 replica.cpp:641] Replica in EMPTY status received
> a broadcasted recover request
> I1022 18:42:26.854856 140 recover.cpp:195] Received a recover response from
> a replica in EMPTY status
> I1022 18:42:26.855002 140 recover.cpp:566] Updating replica status to
> STARTING
> I1022 18:42:26.855655 138 master.cpp:376] Master
> a3f39818-1bda-4710-b96b-2a60ed4d12b8 (a06e4b4eb776) started on
> 172.17.0.14:5050
> I1022 18:42:26.855680 138 master.cpp:378] Flags at startup:
> --allocation_interval="1secs" --allocator="HierarchicalDRF"
> --authenticate="false" --authenticate_slaves="false"
> --authenticators="crammd5" --authorizers="local" --framework_sorter="drf"
> --help="false" --hostname_lookup="true" --initialize_driver_logging="true"
> --log_auto_initialize="true" --logbufsecs="0" --logging_level="INFO"
> --max_slave_ping_timeouts="5" --quiet="false"
> --recovery_slave_removal_limit="100%" --registry="replicated_log"
> --registry_fetch_timeout="1mins" --registry_store_timeout="5secs"
> --registry_strict="false" --root_submissions="true"
> --slave_ping_timeout="15secs" --slave_reregister_timeout="10mins"
> --user_sorter="drf" --version="false" --webui_dir="/usr/share/mesos/webui"
> --work_dir="/tmp/mesos/local/AK0XpG" --zk_session_timeout="10secs"
> I1022 18:42:26.855790 138 master.cpp:425] Master allowing unauthenticated
> frameworks to register
> I1022 18:42:26.855803 138 master.cpp:430] Master allowing unauthenticated
> slaves to register
> I1022 18:42:26.855815 138 master.cpp:467] Using default 'crammd5'
> authenticator
> W1022 18:42:26.855829 138 authenticator.cpp:505] No credentials provided,
> authentication requests will be refused
> I1022 18:42:26.855840 138 authenticator.cpp:512] Initializing server SASL
> I1022 18:42:26.856442 136 containerizer.cpp:143] Using isolation:
> posix/cpu,posix/mem,filesystem/posix
> I1022 18:42:26.856943 140 leveldb.cpp:306] Persisting metadata (8 bytes) to
> leveldb took 1.888185ms
> I1022 18:42:26.856987 140 replica.cpp:323] Persisted replica status to
> STARTING
> I1022 18:42:26.857115 140 recover.cpp:475] Replica is in STARTING status
> I1022 18:42:26.857270 140 replica.cpp:641] Replica in STARTING status
> received a broadcasted recover request
> I1022 18:42:26.857312 140 recover.cpp:195] Received a recover response from
> a replica in STARTING status
> I1022 18:42:26.857368 140 recover.cpp:566] Updating replica status to VOTING
> I1022 18:42:26.857781 140 leveldb.cpp:306] Persisting metadata (8 bytes) to
> leveldb took 371121ns
> I1022 18:42:26.857841 140 replica.cpp:323] Persisted replica status to
> VOTING
> I1022 18:42:26.857895 140 recover.cpp:580] Successfully joined the Paxos
> group
> I1022 18:42:26.857928 140 recover.cpp:464] Recover process terminated
> I1022 18:42:26.862455 137 master.cpp:1603] The newly elected leader is
> master@172.17.0.14:5050 with id a3f39818-1bda-4710-b96b-2a60ed4d12b8
> I1022 18:42:26.862498 137 master.cpp:1616] Elected as the leading master!
> I1022 18:42:26.862511 137 master.cpp:1376] Recovering from registrar
> I1022 18:42:26.862560 137 registrar.cpp:309] Recovering registrar
> Failed to create a containerizer: Could no
[jira] [Commented] (MESOS-3793) Cannot start mesos local on a Debian GNU/Linux 8 docker machine
[
https://issues.apache.org/jira/browse/MESOS-3793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14989299#comment-14989299
]
Peter Kolloch commented on MESOS-3793:
--
The last log line ( Failed to locate systemd runtime directory:
/run/systemd/system) looks as if mesos depended on systemd? Is that correct and
expected?
> Cannot start mesos local on a Debian GNU/Linux 8 docker machine
> ---
>
> Key: MESOS-3793
> URL: https://issues.apache.org/jira/browse/MESOS-3793
> Project: Mesos
> Issue Type: Bug
>Affects Versions: 0.25.0
> Environment: Debian GNU/Linux 8 docker machine
>Reporter: Matthias Veit
>Assignee: Jojy Varghese
> Labels: mesosphere
>
> We updated the mesos version to 0.25.0 in our Marathon docker image, that
> runs our integration tests.
> We use mesos local for those tests. This fails with this message:
> {noformat}
> root@a06e4b4eb776:/marathon# mesos local
> I1022 18:42:26.852485 136 leveldb.cpp:176] Opened db in 6.103258ms
> I1022 18:42:26.853302 136 leveldb.cpp:183] Compacted db in 765740ns
> I1022 18:42:26.853343 136 leveldb.cpp:198] Created db iterator in 9001ns
> I1022 18:42:26.853355 136 leveldb.cpp:204] Seeked to beginning of db in
> 1287ns
> I1022 18:42:26.853366 136 leveldb.cpp:273] Iterated through 0 keys in the
> db in ns
> I1022 18:42:26.853406 136 replica.cpp:744] Replica recovered with log
> positions 0 -> 0 with 1 holes and 0 unlearned
> I1022 18:42:26.853775 141 recover.cpp:449] Starting replica recovery
> I1022 18:42:26.853862 141 recover.cpp:475] Replica is in EMPTY status
> I1022 18:42:26.854751 138 replica.cpp:641] Replica in EMPTY status received
> a broadcasted recover request
> I1022 18:42:26.854856 140 recover.cpp:195] Received a recover response from
> a replica in EMPTY status
> I1022 18:42:26.855002 140 recover.cpp:566] Updating replica status to
> STARTING
> I1022 18:42:26.855655 138 master.cpp:376] Master
> a3f39818-1bda-4710-b96b-2a60ed4d12b8 (a06e4b4eb776) started on
> 172.17.0.14:5050
> I1022 18:42:26.855680 138 master.cpp:378] Flags at startup:
> --allocation_interval="1secs" --allocator="HierarchicalDRF"
> --authenticate="false" --authenticate_slaves="false"
> --authenticators="crammd5" --authorizers="local" --framework_sorter="drf"
> --help="false" --hostname_lookup="true" --initialize_driver_logging="true"
> --log_auto_initialize="true" --logbufsecs="0" --logging_level="INFO"
> --max_slave_ping_timeouts="5" --quiet="false"
> --recovery_slave_removal_limit="100%" --registry="replicated_log"
> --registry_fetch_timeout="1mins" --registry_store_timeout="5secs"
> --registry_strict="false" --root_submissions="true"
> --slave_ping_timeout="15secs" --slave_reregister_timeout="10mins"
> --user_sorter="drf" --version="false" --webui_dir="/usr/share/mesos/webui"
> --work_dir="/tmp/mesos/local/AK0XpG" --zk_session_timeout="10secs"
> I1022 18:42:26.855790 138 master.cpp:425] Master allowing unauthenticated
> frameworks to register
> I1022 18:42:26.855803 138 master.cpp:430] Master allowing unauthenticated
> slaves to register
> I1022 18:42:26.855815 138 master.cpp:467] Using default 'crammd5'
> authenticator
> W1022 18:42:26.855829 138 authenticator.cpp:505] No credentials provided,
> authentication requests will be refused
> I1022 18:42:26.855840 138 authenticator.cpp:512] Initializing server SASL
> I1022 18:42:26.856442 136 containerizer.cpp:143] Using isolation:
> posix/cpu,posix/mem,filesystem/posix
> I1022 18:42:26.856943 140 leveldb.cpp:306] Persisting metadata (8 bytes) to
> leveldb took 1.888185ms
> I1022 18:42:26.856987 140 replica.cpp:323] Persisted replica status to
> STARTING
> I1022 18:42:26.857115 140 recover.cpp:475] Replica is in STARTING status
> I1022 18:42:26.857270 140 replica.cpp:641] Replica in STARTING status
> received a broadcasted recover request
> I1022 18:42:26.857312 140 recover.cpp:195] Received a recover response from
> a replica in STARTING status
> I1022 18:42:26.857368 140 recover.cpp:566] Updating replica status to VOTING
> I1022 18:42:26.857781 140 leveldb.cpp:306] Persisting metadata (8 bytes) to
> leveldb took 371121ns
> I1022 18:42:26.857841 140 replica.cpp:323] Persisted replica status to
> VOTING
> I1022 18:42:26.857895 140 recover.cpp:580] Successfully joined the Paxos
> group
> I1022 18:42:26.857928 140 recover.cpp:464] Recover process terminated
> I1022 18:42:26.862455 137 master.cpp:1603] The newly elected leader is
> master@172.17.0.14:5050 with id a3f39818-1bda-4710-b96b-2a60ed4d12b8
> I1022 18:42:26.862498 137 master.cpp:1616] Elected as the leading master!
> I1022 18:42:26.862511 137 master.cpp:1376] Recovering from registrar
> I1022 18:42:26.862560 137 registrar.cpp:309] R
[jira] [Commented] (MESOS-3793) Cannot start mesos local on a Debian GNU/Linux 8 docker machine
[
https://issues.apache.org/jira/browse/MESOS-3793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14984970#comment-14984970
]
Matthias Veit commented on MESOS-3793:
--
Starting mesos local with --launcher=posix has no effect.
With env variable export MESOS_LAUNCHER=posix I can start mesos local.
Mounting /sys/fs/cgroup and starting mesos local fails with this error:
{noformat}
➔ docker run -v /sys/fs/cgroup:/sys/fs/cgroup:rw -it marathon-buildbase:test sh
# mesos local
I1102 09:35:15.839287 5 leveldb.cpp:176] Opened db in 4.975612ms
I1102 09:35:15.840312 5 leveldb.cpp:183] Compacted db in 981189ns
I1102 09:35:15.840348 5 leveldb.cpp:198] Created db iterator in 9033ns
I1102 09:35:15.840353 5 leveldb.cpp:204] Seeked to beginning of db in 1414ns
I1102 09:35:15.840358 5 leveldb.cpp:273] Iterated through 0 keys in the db
in 1025ns
I1102 09:35:15.840389 5 replica.cpp:744] Replica recovered with log
positions 0 -> 0 with 1 holes and 0 unlearned
I1102 09:35:15.840790 9 recover.cpp:449] Starting replica recovery
I1102 09:35:15.84099110 recover.cpp:475] Replica is in EMPTY status
I1102 09:35:15.841492 9 replica.cpp:641] Replica in EMPTY status received a
broadcasted recover request
I1102 09:35:15.841908 6 recover.cpp:195] Received a recover response from a
replica in EMPTY status
I1102 09:35:15.842003 6 recover.cpp:566] Updating replica status to STARTING
I1102 09:35:15.843122 7 master.cpp:376] Master
af8c1547-e308-4348-99d4-93879f06d853 (833b280a4c4a) started on 172.17.0.7:5050
I1102 09:35:15.843327 7 master.cpp:378] Flags at startup:
--allocation_interval="1secs" --allocator="HierarchicalDRF"
--authenticate="false" --authenticate_slaves="false" --authenticators="crammd5"
--authorizers="local" --framework_sorter="drf" --help="false"
--hostname_lookup="true" --initialize_driver_logging="true"
--log_auto_initialize="true" --logbufsecs="0" --logging_level="INFO"
--max_slave_ping_timeouts="5" --quiet="false"
--recovery_slave_removal_limit="100%" --registry="replicated_log"
--registry_fetch_timeout="1mins" --registry_store_timeout="5secs"
--registry_strict="false" --root_submissions="true"
--slave_ping_timeout="15secs" --slave_reregister_timeout="10mins"
--user_sorter="drf" --version="false" --webui_dir="/usr/share/mesos/webui"
--work_dir="/tmp/mesos/local/JU6SZj" --zk_session_timeout="10secs"
I1102 09:35:15.843575 7 master.cpp:425] Master allowing unauthenticated
frameworks to register
I1102 09:35:15.843822 7 master.cpp:430] Master allowing unauthenticated
slaves to register
I1102 09:35:15.843950 7 master.cpp:467] Using default 'crammd5'
authenticator
W1102 09:35:15.844105 7 authenticator.cpp:505] No credentials provided,
authentication requests will be refused
I1102 09:35:15.844224 7 authenticator.cpp:512] Initializing server SASL
I1102 09:35:15.843875 5 containerizer.cpp:143] Using isolation:
posix/cpu,posix/mem,filesystem/posix
I1102 09:35:15.84323111 leveldb.cpp:306] Persisting metadata (8 bytes) to
leveldb took 1.186846ms
I1102 09:35:15.84482011 replica.cpp:323] Persisted replica status to
STARTING
I1102 09:35:15.84521211 recover.cpp:475] Replica is in STARTING status
I1102 09:35:15.84557711 replica.cpp:641] Replica in STARTING status
received a broadcasted recover request
I1102 09:35:15.84588111 recover.cpp:195] Received a recover response from a
replica in STARTING status
I1102 09:35:15.84621711 recover.cpp:566] Updating replica status to VOTING
I1102 09:35:15.84665011 leveldb.cpp:306] Persisting metadata (8 bytes) to
leveldb took 265224ns
I1102 09:35:15.84668311 replica.cpp:323] Persisted replica status to VOTING
I1102 09:35:15.84672111 recover.cpp:580] Successfully joined the Paxos group
I1102 09:35:15.84683511 recover.cpp:464] Recover process terminated
I1102 09:35:15.849839 7 master.cpp:1603] The newly elected leader is
master@172.17.0.7:5050 with id af8c1547-e308-4348-99d4-93879f06d853
I1102 09:35:15.853528 7 master.cpp:1616] Elected as the leading master!
I1102 09:35:15.853793 7 master.cpp:1376] Recovering from registrar
I1102 09:35:15.85403313 registrar.cpp:309] Recovering registrar
I1102 09:35:15.854266 9 log.cpp:661] Attempting to start the writer
I1102 09:35:15.854802 9 replica.cpp:477] Replica received implicit promise
request with proposal 1
I1102 09:35:15.853359 5 linux_launcher.cpp:103] Using
/sys/fs/cgroup/freezer as the freezer hierarchy for the Linux launcher
I1102 09:35:15.856086 9 leveldb.cpp:306] Persisting metadata (8 bytes) to
leveldb took 1.148617ms
I1102 09:35:15.856168 9 replica.cpp:345] Persisted promised to 1
I1102 09:35:15.857818 6 coordinator.cpp:231] Coordinator attemping to fill
missing position
I1102 09:35:15.85872313 replica.cpp:378] Replica received explicit promise
request for position 0 with proposa
[jira] [Commented] (MESOS-3793) Cannot start mesos local on a Debian GNU/Linux 8 docker machine
[
https://issues.apache.org/jira/browse/MESOS-3793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14971183#comment-14971183
]
haosdent commented on MESOS-3793:
-
Please add --launcher=posix or mount cgroup as rw when launch docker container.
http://search-hadoop.com/m/0Vlr6zfCev1S7gRF1
> Cannot start mesos local on a Debian GNU/Linux 8 docker machine
> ---
>
> Key: MESOS-3793
> URL: https://issues.apache.org/jira/browse/MESOS-3793
> Project: Mesos
> Issue Type: Bug
>Affects Versions: 0.25.0
> Environment: Debian GNU/Linux 8 docker machine
>Reporter: Matthias Veit
>Assignee: Jojy Varghese
> Labels: mesosphere
>
> We updated the mesos version to 0.25.0 in our Marathon docker image, that
> runs our integration tests.
> We use mesos local for those tests. This fails with this message:
> {noformat}
> root@a06e4b4eb776:/marathon# mesos local
> I1022 18:42:26.852485 136 leveldb.cpp:176] Opened db in 6.103258ms
> I1022 18:42:26.853302 136 leveldb.cpp:183] Compacted db in 765740ns
> I1022 18:42:26.853343 136 leveldb.cpp:198] Created db iterator in 9001ns
> I1022 18:42:26.853355 136 leveldb.cpp:204] Seeked to beginning of db in
> 1287ns
> I1022 18:42:26.853366 136 leveldb.cpp:273] Iterated through 0 keys in the
> db in ns
> I1022 18:42:26.853406 136 replica.cpp:744] Replica recovered with log
> positions 0 -> 0 with 1 holes and 0 unlearned
> I1022 18:42:26.853775 141 recover.cpp:449] Starting replica recovery
> I1022 18:42:26.853862 141 recover.cpp:475] Replica is in EMPTY status
> I1022 18:42:26.854751 138 replica.cpp:641] Replica in EMPTY status received
> a broadcasted recover request
> I1022 18:42:26.854856 140 recover.cpp:195] Received a recover response from
> a replica in EMPTY status
> I1022 18:42:26.855002 140 recover.cpp:566] Updating replica status to
> STARTING
> I1022 18:42:26.855655 138 master.cpp:376] Master
> a3f39818-1bda-4710-b96b-2a60ed4d12b8 (a06e4b4eb776) started on
> 172.17.0.14:5050
> I1022 18:42:26.855680 138 master.cpp:378] Flags at startup:
> --allocation_interval="1secs" --allocator="HierarchicalDRF"
> --authenticate="false" --authenticate_slaves="false"
> --authenticators="crammd5" --authorizers="local" --framework_sorter="drf"
> --help="false" --hostname_lookup="true" --initialize_driver_logging="true"
> --log_auto_initialize="true" --logbufsecs="0" --logging_level="INFO"
> --max_slave_ping_timeouts="5" --quiet="false"
> --recovery_slave_removal_limit="100%" --registry="replicated_log"
> --registry_fetch_timeout="1mins" --registry_store_timeout="5secs"
> --registry_strict="false" --root_submissions="true"
> --slave_ping_timeout="15secs" --slave_reregister_timeout="10mins"
> --user_sorter="drf" --version="false" --webui_dir="/usr/share/mesos/webui"
> --work_dir="/tmp/mesos/local/AK0XpG" --zk_session_timeout="10secs"
> I1022 18:42:26.855790 138 master.cpp:425] Master allowing unauthenticated
> frameworks to register
> I1022 18:42:26.855803 138 master.cpp:430] Master allowing unauthenticated
> slaves to register
> I1022 18:42:26.855815 138 master.cpp:467] Using default 'crammd5'
> authenticator
> W1022 18:42:26.855829 138 authenticator.cpp:505] No credentials provided,
> authentication requests will be refused
> I1022 18:42:26.855840 138 authenticator.cpp:512] Initializing server SASL
> I1022 18:42:26.856442 136 containerizer.cpp:143] Using isolation:
> posix/cpu,posix/mem,filesystem/posix
> I1022 18:42:26.856943 140 leveldb.cpp:306] Persisting metadata (8 bytes) to
> leveldb took 1.888185ms
> I1022 18:42:26.856987 140 replica.cpp:323] Persisted replica status to
> STARTING
> I1022 18:42:26.857115 140 recover.cpp:475] Replica is in STARTING status
> I1022 18:42:26.857270 140 replica.cpp:641] Replica in STARTING status
> received a broadcasted recover request
> I1022 18:42:26.857312 140 recover.cpp:195] Received a recover response from
> a replica in STARTING status
> I1022 18:42:26.857368 140 recover.cpp:566] Updating replica status to VOTING
> I1022 18:42:26.857781 140 leveldb.cpp:306] Persisting metadata (8 bytes) to
> leveldb took 371121ns
> I1022 18:42:26.857841 140 replica.cpp:323] Persisted replica status to
> VOTING
> I1022 18:42:26.857895 140 recover.cpp:580] Successfully joined the Paxos
> group
> I1022 18:42:26.857928 140 recover.cpp:464] Recover process terminated
> I1022 18:42:26.862455 137 master.cpp:1603] The newly elected leader is
> master@172.17.0.14:5050 with id a3f39818-1bda-4710-b96b-2a60ed4d12b8
> I1022 18:42:26.862498 137 master.cpp:1616] Elected as the leading master!
> I1022 18:42:26.862511 137 master.cpp:1376] Recovering from registrar
> I1022 18:42:26.862560 137 registrar.cpp:309] Recovering registrar
> Failed to create a
