[jira] [Commented] (MESOS-5176) LinuxFilesystemIsolatorTest.ROOT_RecoverOrphanedPersistentVolume is flaky
[
https://issues.apache.org/jira/browse/MESOS-5176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15239520#comment-15239520
]
Jan Schlicht commented on MESOS-5176:
-
Seeing the same with Fedora 23:
{noformat}
[16:04:44] : [Step 10/10] [ RUN ]
LinuxFilesystemIsolatorTest.ROOT_RecoverOrphanedPersistentVolume
[16:04:44]W: [Step 10/10] I0413 16:04:44.410837 13448 cluster.cpp:149]
Creating default 'local' authorizer
[16:04:44]W: [Step 10/10] I0413 16:04:44.426347 13448 leveldb.cpp:174]
Opened db in 15.339598ms
[16:04:44]W: [Step 10/10] I0413 16:04:44.426901 13448 leveldb.cpp:181]
Compacted db in 529242ns
[16:04:44]W: [Step 10/10] I0413 16:04:44.426939 13448 leveldb.cpp:196]
Created db iterator in 16070ns
[16:04:44]W: [Step 10/10] I0413 16:04:44.426949 13448 leveldb.cpp:202]
Seeked to beginning of db in 1489ns
[16:04:44]W: [Step 10/10] I0413 16:04:44.426957 13448 leveldb.cpp:271]
Iterated through 0 keys in the db in 306ns
[16:04:44]W: [Step 10/10] I0413 16:04:44.426995 13448 replica.cpp:779]
Replica recovered with log positions 0 -> 0 with 1 holes and 0 unlearned
[16:04:44]W: [Step 10/10] I0413 16:04:44.427425 13462 recover.cpp:447]
Starting replica recovery
[16:04:44]W: [Step 10/10] I0413 16:04:44.427654 13462 recover.cpp:473]
Replica is in EMPTY status
[16:04:44]W: [Step 10/10] I0413 16:04:44.428771 13469 replica.cpp:673]
Replica in EMPTY status received a broadcasted recover request from
(17340)@172.30.2.50:44656
[16:04:44]W: [Step 10/10] I0413 16:04:44.429219 13464 recover.cpp:193]
Received a recover response from a replica in EMPTY status
[16:04:44]W: [Step 10/10] I0413 16:04:44.429733 13469 recover.cpp:564]
Updating replica status to STARTING
[16:04:44]W: [Step 10/10] I0413 16:04:44.430845 13466 master.cpp:382]
Master 6d883c86-8eb4-49e5-90d1-a8ad2d8010ae
(ip-172-30-2-50.ec2.internal.mesosphere.io) started on 172.30.2.50:44656
[16:04:44]W: [Step 10/10] I0413 16:04:44.430863 13466 master.cpp:384] Flags
at startup: --acls="" --allocation_interval="1secs"
--allocator="HierarchicalDRF" --authenticate="true" --authenticate_http="true"
--authenticate_slaves="true" --authenticators="crammd5" --authorizers="local"
--credentials="/tmp/wE0Flz/credentials" --framework_sorter="drf" --help="false"
--hostname_lookup="true" --http_authenticators="basic"
--initialize_driver_logging="true" --log_auto_initialize="true"
--logbufsecs="0" --logging_level="INFO" --max_completed_frameworks="50"
--max_completed_tasks_per_framework="1000" --max_slave_ping_timeouts="5"
--quiet="false" --recovery_slave_removal_limit="100%"
--registry="replicated_log" --registry_fetch_timeout="1mins"
--registry_store_timeout="100secs" --registry_strict="true"
--root_submissions="true" --slave_ping_timeout="15secs"
--slave_reregister_timeout="10mins" --user_sorter="drf" --version="false"
--webui_dir="/usr/local/share/mesos/webui" --work_dir="/tmp/wE0Flz/master"
--zk_session_timeout="10secs"
[16:04:44]W: [Step 10/10] I0413 16:04:44.431073 13466 master.cpp:433]
Master only allowing authenticated frameworks to register
[16:04:44]W: [Step 10/10] I0413 16:04:44.431083 13466 master.cpp:438]
Master only allowing authenticated agents to register
[16:04:44]W: [Step 10/10] I0413 16:04:44.431089 13466 credentials.hpp:37]
Loading credentials for authentication from '/tmp/wE0Flz/credentials'
[16:04:44]W: [Step 10/10] I0413 16:04:44.440948 13466 master.cpp:480] Using
default 'crammd5' authenticator
[16:04:44]W: [Step 10/10] I0413 16:04:44.441088 13466 master.cpp:551] Using
default 'basic' HTTP authenticator
[16:04:44]W: [Step 10/10] I0413 16:04:44.441226 13468 leveldb.cpp:304]
Persisting metadata (8 bytes) to leveldb took 11.276196ms
[16:04:44]W: [Step 10/10] I0413 16:04:44.441226 13466 master.cpp:589]
Authorization enabled
[16:04:44]W: [Step 10/10] I0413 16:04:44.441263 13468 replica.cpp:320]
Persisted replica status to STARTING
[16:04:44]W: [Step 10/10] I0413 16:04:44.441478 13469
whitelist_watcher.cpp:77] No whitelist given
[16:04:44]W: [Step 10/10] I0413 16:04:44.441475 13462 hierarchical.cpp:142]
Initialized hierarchical allocator process
[16:04:44]W: [Step 10/10] I0413 16:04:44.441474 13463 recover.cpp:473]
Replica is in STARTING status
[16:04:44]W: [Step 10/10] I0413 16:04:44.442275 13467 replica.cpp:673]
Replica in STARTING status received a broadcasted recover request from
(17342)@172.30.2.50:44656
[16:04:44]W: [Step 10/10] I0413 16:04:44.442739 13469 recover.cpp:193]
Received a recover response from a replica in STARTING status
[16:04:44]W: [Step 10/10] I0413 16:04:44.443559 13468 recover.cpp:564]
Updating replica status to VOTING
[16:04:44]W: [Step 10/10] I0413 16:04:44.443832 13462 master.cpp:1832] The
newly elected leader is master@172.30.2.50:44656 with id
6d883c86-8eb4-49e5-90d1-a8ad2d8010ae
[16:0
[jira] [Commented] (MESOS-5176) LinuxFilesystemIsolatorTest.ROOT_RecoverOrphanedPersistentVolume is flaky
[
https://issues.apache.org/jira/browse/MESOS-5176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15235893#comment-15235893
]
Greg Mann commented on MESOS-5176:
--
[~kaysoky]
> LinuxFilesystemIsolatorTest.ROOT_RecoverOrphanedPersistentVolume is flaky
> -
>
> Key: MESOS-5176
> URL: https://issues.apache.org/jira/browse/MESOS-5176
> Project: Mesos
> Issue Type: Bug
> Components: tests
> Environment: CentOS 7, with libevent and SSL enabled
>Reporter: Greg Mann
> Labels: mesosphere
>
> Observed on the internal Mesosphere CI:
> {code}
> [07:10:58] : [Step 11/11] [ RUN ]
> LinuxFilesystemIsolatorTest.ROOT_RecoverOrphanedPersistentVolume
> [07:10:58]W: [Step 11/11] I0410 07:10:58.289384 32129 cluster.cpp:149]
> Creating default 'local' authorizer
> [07:10:58]W: [Step 11/11] I0410 07:10:58.317526 32129 leveldb.cpp:174]
> Opened db in 27.91929ms
> [07:10:58]W: [Step 11/11] I0410 07:10:58.318943 32129 leveldb.cpp:181]
> Compacted db in 1.383973ms
> [07:10:58]W: [Step 11/11] I0410 07:10:58.318989 32129 leveldb.cpp:196]
> Created db iterator in 18603ns
> [07:10:58]W: [Step 11/11] I0410 07:10:58.319000 32129 leveldb.cpp:202]
> Seeked to beginning of db in 1529ns
> [07:10:58]W: [Step 11/11] I0410 07:10:58.319008 32129 leveldb.cpp:271]
> Iterated through 0 keys in the db in 358ns
> [07:10:58]W: [Step 11/11] I0410 07:10:58.319046 32129 replica.cpp:779]
> Replica recovered with log positions 0 -> 0 with 1 holes and 0 unlearned
> [07:10:58]W: [Step 11/11] I0410 07:10:58.319627 32143 recover.cpp:447]
> Starting replica recovery
> [07:10:58]W: [Step 11/11] I0410 07:10:58.319852 32143 recover.cpp:473]
> Replica is in EMPTY status
> [07:10:58]W: [Step 11/11] I0410 07:10:58.320796 32145 replica.cpp:673]
> Replica in EMPTY status received a broadcasted recover request from
> (17047)@172.30.2.121:48158
> [07:10:58]W: [Step 11/11] I0410 07:10:58.321202 32146 recover.cpp:193]
> Received a recover response from a replica in EMPTY status
> [07:10:58]W: [Step 11/11] I0410 07:10:58.321650 32150 recover.cpp:564]
> Updating replica status to STARTING
> [07:10:58]W: [Step 11/11] I0410 07:10:58.323005 32149 master.cpp:382]
> Master 57a2cf4e-da76-4801-a887-c0c84ad59d0d (ip-172-30-2-121.mesosphere.io)
> started on 172.30.2.121:48158
> [07:10:58]W: [Step 11/11] I0410 07:10:58.323022 32149 master.cpp:384] Flags
> at startup: --acls="" --allocation_interval="1secs"
> --allocator="HierarchicalDRF" --authenticate="true"
> --authenticate_http="true" --authenticate_slaves="true"
> --authenticators="crammd5" --authorizers="local"
> --credentials="/tmp/fWC4sn/credentials" --framework_sorter="drf"
> --help="false" --hostname_lookup="true" --http_authenticators="basic"
> --initialize_driver_logging="true" --log_auto_initialize="true"
> --logbufsecs="0" --logging_level="INFO" --max_completed_frameworks="50"
> --max_completed_tasks_per_framework="1000" --max_slave_ping_timeouts="5"
> --quiet="false" --recovery_slave_removal_limit="100%"
> --registry="replicated_log" --registry_fetch_timeout="1mins"
> --registry_store_timeout="100secs" --registry_strict="true"
> --root_submissions="true" --slave_ping_timeout="15secs"
> --slave_reregister_timeout="10mins" --user_sorter="drf" --version="false"
> --webui_dir="/usr/local/share/mesos/webui" --work_dir="/tmp/fWC4sn/master"
> --zk_session_timeout="10secs"
> [07:10:58]W: [Step 11/11] I0410 07:10:58.323227 32149 master.cpp:433]
> Master only allowing authenticated frameworks to register
> [07:10:58]W: [Step 11/11] I0410 07:10:58.323237 32149 master.cpp:438]
> Master only allowing authenticated agents to register
> [07:10:58]W: [Step 11/11] I0410 07:10:58.323243 32149 credentials.hpp:37]
> Loading credentials for authentication from '/tmp/fWC4sn/credentials'
> [07:10:58]W: [Step 11/11] I0410 07:10:58.323498 32149 master.cpp:480] Using
> default 'crammd5' authenticator
> [07:10:58]W: [Step 11/11] I0410 07:10:58.323616 32149 master.cpp:551] Using
> default 'basic' HTTP authenticator
> [07:10:58]W: [Step 11/11] I0410 07:10:58.323739 32149 master.cpp:589]
> Authorization enabled
> [07:10:58]W: [Step 11/11] I0410 07:10:58.323884 32150
> whitelist_watcher.cpp:77] No whitelist given
> [07:10:58]W: [Step 11/11] I0410 07:10:58.323920 32143 hierarchical.cpp:142]
> Initialized hierarchical allocator process
> [07:10:58]W: [Step 11/11] I0410 07:10:58.324103 32148 leveldb.cpp:304]
> Persisting metadata (8 bytes) to leveldb took 2.27166ms
> [07:10:58]W: [Step 11/11] I0410 07:10:58.324126 32148 replica.cpp:320]
> Persisted replica status to STARTING
> [07:10:58]W: [Step 11/11] I0410 07:10:58.324322 32146 recover.cpp:473]
> Replica is in STARTING status
> [07:10:58]W: [Step 11/11] I0410
