[ https://issues.apache.org/jira/browse/MESOS-4318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15090311#comment-15090311 ]
Greg Mann commented on MESOS-4318: ---------------------------------- Review here: https://reviews.apache.org/r/42096/ > PersistentVolumeTest.BadACLNoPrincipal is flaky > ----------------------------------------------- > > Key: MESOS-4318 > URL: https://issues.apache.org/jira/browse/MESOS-4318 > Project: Mesos > Issue Type: Bug > Reporter: Jie Yu > Assignee: Greg Mann > Labels: flaky-test > > https://builds.apache.org/job/Mesos/1457/COMPILER=gcc,CONFIGURATION=--verbose%20--enable-libevent%20--enable-ssl,OS=centos:7,label_exp=docker%7C%7CHadoop/consoleFull > {noformat} > [ RUN ] PersistentVolumeTest.BadACLNoPrincipal > I0108 01:13:16.117883 1325 leveldb.cpp:174] Opened db in 2.614722ms > I0108 01:13:16.118650 1325 leveldb.cpp:181] Compacted db in 706567ns > I0108 01:13:16.118702 1325 leveldb.cpp:196] Created db iterator in 24489ns > I0108 01:13:16.118723 1325 leveldb.cpp:202] Seeked to beginning of db in > 2436ns > I0108 01:13:16.118738 1325 leveldb.cpp:271] Iterated through 0 keys in the > db in 397ns > I0108 01:13:16.118793 1325 replica.cpp:779] Replica recovered with log > positions 0 -> 0 with 1 holes and 0 unlearned > I0108 01:13:16.119627 1348 recover.cpp:447] Starting replica recovery > I0108 01:13:16.120352 1348 recover.cpp:473] Replica is in EMPTY status > I0108 01:13:16.121750 1357 replica.cpp:673] Replica in EMPTY status received > a broadcasted recover request from (7084)@172.17.0.2:32801 > I0108 01:13:16.122297 1353 recover.cpp:193] Received a recover response from > a replica in EMPTY status > I0108 01:13:16.122747 1350 recover.cpp:564] Updating replica status to > STARTING > I0108 01:13:16.123625 1354 master.cpp:365] Master > 773d31e8-383d-4e4b-aa68-f9a3fb9f1fc2 (d9632dd1c41e) started on > 172.17.0.2:32801 > I0108 01:13:16.123946 1347 leveldb.cpp:304] Persisting metadata (8 bytes) to > leveldb took 728242ns > I0108 01:13:16.123999 1347 replica.cpp:320] Persisted replica status to > STARTING > I0108 01:13:16.123708 1354 master.cpp:367] Flags at startup: > --acls="create_volumes { > principals { > values: "test-principal" > } > volume_types { > type: ANY > } > } > create_volumes { > principals { > type: ANY > } > volume_types { > type: NONE > } > } > " --allocation_interval="1secs" --allocator="HierarchicalDRF" > --authenticate="false" --authenticate_slaves="true" > --authenticators="crammd5" --authorizers="local" > --credentials="/tmp/f2rA75/credentials" --framework_sorter="drf" > --help="false" --hostname_lookup="true" --initialize_driver_logging="true" > --log_auto_initialize="true" --logbufsecs="0" --logging_level="INFO" > --max_slave_ping_timeouts="5" --quiet="false" > --recovery_slave_removal_limit="100%" --registry="replicated_log" > --registry_fetch_timeout="1mins" --registry_store_timeout="25secs" > --registry_strict="true" --roles="role1" --root_submissions="true" > --slave_ping_timeout="15secs" --slave_reregister_timeout="10mins" > --user_sorter="drf" --version="false" > --webui_dir="/mesos/mesos-0.27.0/_inst/share/mesos/webui" > --work_dir="/tmp/f2rA75/master" --zk_session_timeout="10secs" > I0108 01:13:16.124219 1354 master.cpp:414] Master allowing unauthenticated > frameworks to register > I0108 01:13:16.124236 1354 master.cpp:417] Master only allowing > authenticated slaves to register > I0108 01:13:16.124248 1354 credentials.hpp:35] Loading credentials for > authentication from '/tmp/f2rA75/credentials' > I0108 01:13:16.124294 1358 recover.cpp:473] Replica is in STARTING status > I0108 01:13:16.124644 1354 master.cpp:456] Using default 'crammd5' > authenticator > I0108 01:13:16.124820 1354 master.cpp:493] Authorization enabled > W0108 01:13:16.124843 1354 master.cpp:553] The '--roles' flag is deprecated. > This flag will be removed in the future. See the Mesos 0.27 upgrade notes for > more information > I0108 01:13:16.125154 1348 hierarchical.cpp:147] Initialized hierarchical > allocator process > I0108 01:13:16.125334 1345 whitelist_watcher.cpp:77] No whitelist given > I0108 01:13:16.126065 1346 replica.cpp:673] Replica in STARTING status > received a broadcasted recover request from (7085)@172.17.0.2:32801 > I0108 01:13:16.126806 1348 recover.cpp:193] Received a recover response from > a replica in STARTING status > I0108 01:13:16.128237 1354 recover.cpp:564] Updating replica status to VOTING > I0108 01:13:16.128402 1359 master.cpp:1629] The newly elected leader is > master@172.17.0.2:32801 with id 773d31e8-383d-4e4b-aa68-f9a3fb9f1fc2 > I0108 01:13:16.128489 1359 master.cpp:1642] Elected as the leading master! > I0108 01:13:16.128523 1359 master.cpp:1387] Recovering from registrar > I0108 01:13:16.128756 1355 registrar.cpp:307] Recovering registrar > I0108 01:13:16.129259 1344 leveldb.cpp:304] Persisting metadata (8 bytes) to > leveldb took 531437ns > I0108 01:13:16.129292 1344 replica.cpp:320] Persisted replica status to > VOTING > I0108 01:13:16.129425 1358 recover.cpp:578] Successfully joined the Paxos > group > I0108 01:13:16.129680 1358 recover.cpp:462] Recover process terminated > I0108 01:13:16.130187 1358 log.cpp:659] Attempting to start the writer > I0108 01:13:16.131613 1352 replica.cpp:493] Replica received implicit > promise request from (7086)@172.17.0.2:32801 with proposal 1 > I0108 01:13:16.131983 1352 leveldb.cpp:304] Persisting metadata (8 bytes) to > leveldb took 333646ns > I0108 01:13:16.132004 1352 replica.cpp:342] Persisted promised to 1 > I0108 01:13:16.132627 1348 coordinator.cpp:238] Coordinator attempting to > fill missing positions > I0108 01:13:16.133896 1349 replica.cpp:388] Replica received explicit > promise request from (7087)@172.17.0.2:32801 for position 0 with proposal 2 > I0108 01:13:16.134289 1349 leveldb.cpp:341] Persisting action (8 bytes) to > leveldb took 349652ns > I0108 01:13:16.134317 1349 replica.cpp:712] Persisted action at 0 > I0108 01:13:16.135470 1351 replica.cpp:537] Replica received write request > for position 0 from (7088)@172.17.0.2:32801 > I0108 01:13:16.135537 1351 leveldb.cpp:436] Reading position from leveldb > took 36181ns > I0108 01:13:16.135901 1351 leveldb.cpp:341] Persisting action (14 bytes) to > leveldb took 308752ns > I0108 01:13:16.135924 1351 replica.cpp:712] Persisted action at 0 > I0108 01:13:16.136529 1347 replica.cpp:691] Replica received learned notice > for position 0 from @0.0.0.0:0 > I0108 01:13:16.136889 1347 leveldb.cpp:341] Persisting action (16 bytes) to > leveldb took 327106ns > I0108 01:13:16.136916 1347 replica.cpp:712] Persisted action at 0 > I0108 01:13:16.136943 1347 replica.cpp:697] Replica learned NOP action at > position 0 > I0108 01:13:16.137707 1359 log.cpp:675] Writer started with ending position 0 > I0108 01:13:16.138844 1348 leveldb.cpp:436] Reading position from leveldb > took 31371ns > I0108 01:13:16.139878 1356 registrar.cpp:340] Successfully fetched the > registry (0B) in 0ns > I0108 01:13:16.140012 1356 registrar.cpp:439] Applied 1 operations in > 42063ns; attempting to update the 'registry' > I0108 01:13:16.140797 1355 log.cpp:683] Attempting to append 170 bytes to > the log > I0108 01:13:16.140974 1345 coordinator.cpp:348] Coordinator attempting to > write APPEND action at position 1 > I0108 01:13:16.141744 1354 replica.cpp:537] Replica received write request > for position 1 from (7089)@172.17.0.2:32801 > I0108 01:13:16.142226 1354 leveldb.cpp:341] Persisting action (189 bytes) to > leveldb took 441971ns > I0108 01:13:16.142251 1354 replica.cpp:712] Persisted action at 1 > I0108 01:13:16.142860 1351 replica.cpp:691] Replica received learned notice > for position 1 from @0.0.0.0:0 > I0108 01:13:16.143198 1351 leveldb.cpp:341] Persisting action (191 bytes) to > leveldb took 305928ns > I0108 01:13:16.143223 1351 replica.cpp:712] Persisted action at 1 > I0108 01:13:16.143241 1351 replica.cpp:697] Replica learned APPEND action at > position 1 > I0108 01:13:16.144271 1354 registrar.cpp:484] Successfully updated the > 'registry' in 0ns > I0108 01:13:16.144435 1354 registrar.cpp:370] Successfully recovered > registrar > I0108 01:13:16.144567 1359 log.cpp:702] Attempting to truncate the log to 1 > I0108 01:13:16.144780 1359 coordinator.cpp:348] Coordinator attempting to > write TRUNCATE action at position 2 > I0108 01:13:16.144989 1348 hierarchical.cpp:165] Skipping recovery of > hierarchical allocator: nothing to recover > I0108 01:13:16.144928 1354 master.cpp:1439] Recovered 0 slaves from the > Registry (131B) ; allowing 10mins for slaves to re-register > I0108 01:13:16.145690 1357 replica.cpp:537] Replica received write request > for position 2 from (7090)@172.17.0.2:32801 > I0108 01:13:16.146072 1357 leveldb.cpp:341] Persisting action (16 bytes) to > leveldb took 345113ns > I0108 01:13:16.146097 1357 replica.cpp:712] Persisted action at 2 > I0108 01:13:16.146667 1358 replica.cpp:691] Replica received learned notice > for position 2 from @0.0.0.0:0 > I0108 01:13:16.147060 1358 leveldb.cpp:341] Persisting action (18 bytes) to > leveldb took 283648ns > I0108 01:13:16.147116 1358 leveldb.cpp:399] Deleting ~1 keys from leveldb > took 32174ns > I0108 01:13:16.147135 1358 replica.cpp:712] Persisted action at 2 > I0108 01:13:16.147153 1358 replica.cpp:697] Replica learned TRUNCATE action > at position 2 > I0108 01:13:16.166832 1325 containerizer.cpp:139] Using isolation: > posix/cpu,posix/mem,filesystem/posix > W0108 01:13:16.167556 1325 backend.cpp:48] Failed to create 'bind' backend: > BindBackend requires root privileges > I0108 01:13:16.170526 1349 slave.cpp:191] Slave started on > 231)@172.17.0.2:32801 > I0108 01:13:16.170718 1349 slave.cpp:192] Flags at startup: > --appc_store_dir="/tmp/mesos/store/appc" --authenticatee="crammd5" > --cgroups_cpu_enable_pids_and_tids_count="false" --cgroups_enable_cfs="false" > --cgroups_hierarchy="/sys/fs/cgroup" --cgroups_limit_swap="false" > --cgroups_root="mesos" --container_disk_watch_interval="15secs" > --containerizers="mesos" > --credential="/tmp/PersistentVolumeTest_BadACLNoPrincipal_yqJjLY/credential" > --default_role="*" --disk_watch_interval="1mins" --docker="docker" > --docker_auth_server="https://auth.docker.io" --docker_kill_orphans="true" > --docker_puller_timeout="60" --docker_registry="https://registry-1.docker.io" > --docker_remove_delay="6hrs" --docker_socket="/var/run/docker.sock" > --docker_stop_timeout="0ns" --docker_store_dir="/tmp/mesos/store/docker" > --enforce_container_disk_quota="false" > --executor_registration_timeout="1mins" > --executor_shutdown_grace_period="5secs" > --fetcher_cache_dir="/tmp/PersistentVolumeTest_BadACLNoPrincipal_yqJjLY/fetch" > --fetcher_cache_size="2GB" --frameworks_home="" --gc_delay="1weeks" > --gc_disk_headroom="0.1" --hadoop_home="" --help="false" > --hostname_lookup="true" --image_provisioner_backend="copy" > --initialize_driver_logging="true" --isolation="posix/cpu,posix/mem" > --launcher_dir="/mesos/mesos-0.27.0/_build/src" --logbufsecs="0" > --logging_level="INFO" --oversubscribed_resources_interval="15secs" > --perf_duration="10secs" --perf_interval="1mins" > --qos_correction_interval_min="0ns" --quiet="false" --recover="reconnect" > --recovery_timeout="15mins" --registration_backoff_factor="10ms" > --resources="cpus:2;mem:1024;disk(role1):2048" > --revocable_cpu_low_priority="true" --sandbox_directory="/mnt/mesos/sandbox" > --strict="true" --switch_user="true" > --systemd_runtime_directory="/run/systemd/system" --version="false" > --work_dir="/tmp/PersistentVolumeTest_BadACLNoPrincipal_yqJjLY" > I0108 01:13:16.171269 1349 credentials.hpp:83] Loading credential for > authentication from > '/tmp/PersistentVolumeTest_BadACLNoPrincipal_yqJjLY/credential' > I0108 01:13:16.171505 1349 slave.cpp:322] Slave using credential for: > test-principal > I0108 01:13:16.171747 1349 resources.cpp:481] Parsing resources as JSON > failed: cpus:2;mem:1024;disk(role1):2048 > Trying semicolon-delimited string format instead > I0108 01:13:16.172266 1349 slave.cpp:392] Slave resources: cpus(*):2; > mem(*):1024; disk(role1):2048; ports(*):[31000-32000] > I0108 01:13:16.172327 1349 slave.cpp:400] Slave attributes: [ ] > I0108 01:13:16.172340 1349 slave.cpp:405] Slave hostname: d9632dd1c41e > I0108 01:13:16.172353 1349 slave.cpp:410] Slave checkpoint: true > I0108 01:13:16.173418 1353 state.cpp:58] Recovering state from > '/tmp/PersistentVolumeTest_BadACLNoPrincipal_yqJjLY/meta' > I0108 01:13:16.173521 1325 sched.cpp:164] Version: 0.27.0 > I0108 01:13:16.174054 1345 status_update_manager.cpp:200] Recovering status > update manager > I0108 01:13:16.174289 1353 containerizer.cpp:387] Recovering containerizer > I0108 01:13:16.174295 1356 sched.cpp:268] New master detected at > master@172.17.0.2:32801 > I0108 01:13:16.174387 1356 sched.cpp:278] No credentials provided. > Attempting to register without authentication > I0108 01:13:16.174409 1356 sched.cpp:722] Sending SUBSCRIBE call to > master@172.17.0.2:32801 > I0108 01:13:16.174515 1356 sched.cpp:755] Will retry registration in > 1.699889272secs if necessary > I0108 01:13:16.174653 1349 master.cpp:2197] Received SUBSCRIBE call for > framework 'no-principal' at > scheduler-bf0ed267-b4c4-412d-9fb0-84c85cd2fbce@172.17.0.2:32801 > I0108 01:13:16.174823 1349 master.cpp:1668] Authorizing framework principal > '' to receive offers for role 'role1' > I0108 01:13:16.175250 1347 master.cpp:2268] Subscribing framework > no-principal with checkpointing disabled and capabilities [ ] > I0108 01:13:16.175359 1353 slave.cpp:4429] Finished recovery > I0108 01:13:16.175715 1345 hierarchical.cpp:260] Added framework > 773d31e8-383d-4e4b-aa68-f9a3fb9f1fc2-0000 > I0108 01:13:16.175734 1351 sched.cpp:649] Framework registered with > 773d31e8-383d-4e4b-aa68-f9a3fb9f1fc2-0000 > I0108 01:13:16.175792 1345 hierarchical.cpp:1329] No resources available to > allocate! > I0108 01:13:16.175833 1345 hierarchical.cpp:1423] No inverse offers to send > out! > I0108 01:13:16.175853 1353 slave.cpp:4601] Querying resource estimator for > oversubscribable resources > I0108 01:13:16.175869 1345 hierarchical.cpp:1079] Performed allocation for 0 > slaves in 127881ns > I0108 01:13:16.175923 1351 sched.cpp:663] Scheduler::registered took 27956ns > I0108 01:13:16.176110 1353 slave.cpp:729] New master detected at > master@172.17.0.2:32801 > I0108 01:13:16.176187 1353 slave.cpp:792] Authenticating with master > master@172.17.0.2:32801 > I0108 01:13:16.176216 1353 slave.cpp:797] Using default CRAM-MD5 > authenticatee > I0108 01:13:16.176398 1357 status_update_manager.cpp:174] Pausing sending > status updates > I0108 01:13:16.176404 1353 slave.cpp:765] Detecting new master > I0108 01:13:16.176463 1358 authenticatee.cpp:121] Creating new client SASL > connection > I0108 01:13:16.176553 1353 slave.cpp:4615] Received oversubscribable > resources from the resource estimator > I0108 01:13:16.176709 1353 master.cpp:5445] Authenticating > slave(231)@172.17.0.2:32801 > I0108 01:13:16.176823 1359 authenticator.cpp:413] Starting authentication > session for crammd5_authenticatee(516)@172.17.0.2:32801 > I0108 01:13:16.177135 1348 authenticator.cpp:98] Creating new server SASL > connection > I0108 01:13:16.177373 1356 authenticatee.cpp:212] Received SASL > authentication mechanisms: CRAM-MD5 > I0108 01:13:16.177399 1356 authenticatee.cpp:238] Attempting to authenticate > with mechanism 'CRAM-MD5' > I0108 01:13:16.177502 1344 authenticator.cpp:203] Received SASL > authentication start > I0108 01:13:16.177563 1344 authenticator.cpp:325] Authentication requires > more steps > I0108 01:13:16.177680 1346 authenticatee.cpp:258] Received SASL > authentication step > I0108 01:13:16.177848 1354 authenticator.cpp:231] Received SASL > authentication step > I0108 01:13:16.177883 1354 auxprop.cpp:107] Request to lookup properties for > user: 'test-principal' realm: 'd9632dd1c41e' server FQDN: 'd9632dd1c41e' > SASL_AUXPROP_VERIFY_AGAINST_HASH: false SASL_AUXPROP_OVERRIDE: false > SASL_AUXPROP_AUTHZID: false > I0108 01:13:16.177894 1354 auxprop.cpp:179] Looking up auxiliary property > '*userPassword' > I0108 01:13:16.177944 1354 auxprop.cpp:179] Looking up auxiliary property > '*cmusaslsecretCRAM-MD5' > I0108 01:13:16.177994 1354 auxprop.cpp:107] Request to lookup properties for > user: 'test-principal' realm: 'd9632dd1c41e' server FQDN: 'd9632dd1c41e' > SASL_AUXPROP_VERIFY_AGAINST_HASH: false SASL_AUXPROP_OVERRIDE: false > SASL_AUXPROP_AUTHZID: true > I0108 01:13:16.178014 1354 auxprop.cpp:129] Skipping auxiliary property > '*userPassword' since SASL_AUXPROP_AUTHZID == true > I0108 01:13:16.178040 1354 auxprop.cpp:129] Skipping auxiliary property > '*cmusaslsecretCRAM-MD5' since SASL_AUXPROP_AUTHZID == true > I0108 01:13:16.178066 1354 authenticator.cpp:317] Authentication success > I0108 01:13:16.178256 1355 authenticatee.cpp:298] Authentication success > I0108 01:13:16.178315 1354 master.cpp:5475] Successfully authenticated > principal 'test-principal' at slave(231)@172.17.0.2:32801 > I0108 01:13:16.178356 1355 authenticator.cpp:431] Authentication session > cleanup for crammd5_authenticatee(516)@172.17.0.2:32801 > I0108 01:13:16.178710 1354 slave.cpp:860] Successfully authenticated with > master master@172.17.0.2:32801 > I0108 01:13:16.178865 1354 slave.cpp:1254] Will retry registration in > 13.009431ms if necessary > I0108 01:13:16.179138 1350 master.cpp:4154] Registering slave at > slave(231)@172.17.0.2:32801 (d9632dd1c41e) with id > 773d31e8-383d-4e4b-aa68-f9a3fb9f1fc2-S0 > I0108 01:13:16.179628 1345 registrar.cpp:439] Applied 1 operations in > 71663ns; attempting to update the 'registry' > I0108 01:13:16.180505 1356 log.cpp:683] Attempting to append 343 bytes to > the log > I0108 01:13:16.180711 1352 coordinator.cpp:348] Coordinator attempting to > write APPEND action at position 3 > I0108 01:13:16.181499 1350 replica.cpp:537] Replica received write request > for position 3 from (7103)@172.17.0.2:32801 > I0108 01:13:16.182080 1350 leveldb.cpp:341] Persisting action (362 bytes) to > leveldb took 537757ns > I0108 01:13:16.182112 1350 replica.cpp:712] Persisted action at 3 > I0108 01:13:16.182749 1351 replica.cpp:691] Replica received learned notice > for position 3 from @0.0.0.0:0 > I0108 01:13:16.183120 1351 leveldb.cpp:341] Persisting action (364 bytes) to > leveldb took 340999ns > I0108 01:13:16.183151 1351 replica.cpp:712] Persisted action at 3 > I0108 01:13:16.183177 1351 replica.cpp:697] Replica learned APPEND action at > position 3 > I0108 01:13:16.184787 1348 registrar.cpp:484] Successfully updated the > 'registry' in 0ns > I0108 01:13:16.185287 1348 log.cpp:702] Attempting to truncate the log to 3 > I0108 01:13:16.185484 1349 coordinator.cpp:348] Coordinator attempting to > write TRUNCATE action at position 4 > I0108 01:13:16.186043 1353 slave.cpp:3371] Received ping from > slave-observer(230)@172.17.0.2:32801 > I0108 01:13:16.186074 1345 master.cpp:4222] Registered slave > 773d31e8-383d-4e4b-aa68-f9a3fb9f1fc2-S0 at slave(231)@172.17.0.2:32801 > (d9632dd1c41e) with cpus(*):2; mem(*):1024; disk(role1):2048; > ports(*):[31000-32000] > I0108 01:13:16.186224 1353 slave.cpp:904] Registered with master > master@172.17.0.2:32801; given slave ID > 773d31e8-383d-4e4b-aa68-f9a3fb9f1fc2-S0 > I0108 01:13:16.186441 1353 fetcher.cpp:81] Clearing fetcher cache > I0108 01:13:16.186486 1349 hierarchical.cpp:465] Added slave > 773d31e8-383d-4e4b-aa68-f9a3fb9f1fc2-S0 (d9632dd1c41e) with cpus(*):2; > mem(*):1024; disk(role1):2048; ports(*):[31000-32000] (allocated: ) > I0108 01:13:16.186658 1346 status_update_manager.cpp:181] Resuming sending > status updates > I0108 01:13:16.186885 1353 slave.cpp:927] Checkpointing SlaveInfo to > '/tmp/PersistentVolumeTest_BadACLNoPrincipal_yqJjLY/meta/slaves/773d31e8-383d-4e4b-aa68-f9a3fb9f1fc2-S0/slave.info' > I0108 01:13:16.186905 1350 replica.cpp:537] Replica received write request > for position 4 from (7104)@172.17.0.2:32801 > I0108 01:13:16.187595 1350 leveldb.cpp:341] Persisting action (16 bytes) to > leveldb took 645704ns > I0108 01:13:16.187628 1350 replica.cpp:712] Persisted action at 4 > I0108 01:13:16.188347 1349 hierarchical.cpp:1423] No inverse offers to send > out! > I0108 01:13:16.188475 1349 hierarchical.cpp:1101] Performed allocation for > slave 773d31e8-383d-4e4b-aa68-f9a3fb9f1fc2-S0 in 1.861833ms > I0108 01:13:16.188560 1348 replica.cpp:691] Replica received learned notice > for position 4 from @0.0.0.0:0 > I0108 01:13:16.188385 1353 slave.cpp:963] Forwarding total oversubscribed > resources > I0108 01:13:16.189275 1344 master.cpp:5274] Sending 1 offers to framework > 773d31e8-383d-4e4b-aa68-f9a3fb9f1fc2-0000 (no-principal) at > scheduler-bf0ed267-b4c4-412d-9fb0-84c85cd2fbce@172.17.0.2:32801 > I0108 01:13:16.189792 1344 master.cpp:4564] Received update of slave > 773d31e8-383d-4e4b-aa68-f9a3fb9f1fc2-S0 at slave(231)@172.17.0.2:32801 > (d9632dd1c41e) with total oversubscribed resources > I0108 01:13:16.189851 1348 leveldb.cpp:341] Persisting action (18 bytes) to > leveldb took 1.204958ms > I0108 01:13:16.190150 1348 leveldb.cpp:399] Deleting ~2 keys from leveldb > took 62381ns > I0108 01:13:16.190265 1348 replica.cpp:712] Persisted action at 4 > I0108 01:13:16.190402 1348 replica.cpp:697] Replica learned TRUNCATE action > at position 4 > I0108 01:13:16.191192 1349 sched.cpp:819] Scheduler::resourceOffers took > 126783ns > I0108 01:13:16.191253 1359 hierarchical.cpp:521] Slave > 773d31e8-383d-4e4b-aa68-f9a3fb9f1fc2-S0 (d9632dd1c41e) updated with > oversubscribed resources (total: cpus(*):2; mem(*):1024; disk(role1):2048; > ports(*):[31000-32000], allocated: cpus(*):2; mem(*):1024; > ports(*):[31000-32000]; disk(role1):2048) > I0108 01:13:16.191529 1359 hierarchical.cpp:1329] No resources available to > allocate! > I0108 01:13:16.191591 1359 hierarchical.cpp:1423] No inverse offers to send > out! > I0108 01:13:16.191627 1359 hierarchical.cpp:1101] Performed allocation for > slave 773d31e8-383d-4e4b-aa68-f9a3fb9f1fc2-S0 in 310808ns > I0108 01:13:16.195103 1349 hierarchical.cpp:1329] No resources available to > allocate! > I0108 01:13:16.195171 1349 hierarchical.cpp:1423] No inverse offers to send > out! > I0108 01:13:16.195205 1349 hierarchical.cpp:1079] Performed allocation for 1 > slaves in 368834ns > I0108 01:13:16.205402 1351 master.cpp:3055] Processing ACCEPT call for > offers: [ 773d31e8-383d-4e4b-aa68-f9a3fb9f1fc2-O0 ] on slave > 773d31e8-383d-4e4b-aa68-f9a3fb9f1fc2-S0 at slave(231)@172.17.0.2:32801 > (d9632dd1c41e) for framework 773d31e8-383d-4e4b-aa68-f9a3fb9f1fc2-0000 > (no-principal) at > scheduler-bf0ed267-b4c4-412d-9fb0-84c85cd2fbce@172.17.0.2:32801 > I0108 01:13:16.205471 1351 master.cpp:2843] Authorizing principal 'ANY' to > create volumes > E0108 01:13:16.206641 1351 master.cpp:1737] Dropping CREATE offer operation > from framework 773d31e8-383d-4e4b-aa68-f9a3fb9f1fc2-0000 (no-principal) at > scheduler-bf0ed267-b4c4-412d-9fb0-84c85cd2fbce@172.17.0.2:32801: Not > authorized to create persistent volumes as '' > I0108 01:13:16.207283 1351 hierarchical.cpp:880] Recovered cpus(*):2; > mem(*):1024; ports(*):[31000-32000]; disk(role1):2048 (total: cpus(*):2; > mem(*):1024; disk(role1):2048; ports(*):[31000-32000], allocated: ) on slave > 773d31e8-383d-4e4b-aa68-f9a3fb9f1fc2-S0 from framework > 773d31e8-383d-4e4b-aa68-f9a3fb9f1fc2-0000 > I0108 01:13:16.216485 1348 hierarchical.cpp:1423] No inverse offers to send > out! > I0108 01:13:16.216562 1348 hierarchical.cpp:1079] Performed allocation for 1 > slaves in 983574ns > I0108 01:13:16.216915 1345 master.cpp:5274] Sending 1 offers to framework > 773d31e8-383d-4e4b-aa68-f9a3fb9f1fc2-0000 (no-principal) at > scheduler-bf0ed267-b4c4-412d-9fb0-84c85cd2fbce@172.17.0.2:32801 > I0108 01:13:16.217514 1345 sched.cpp:819] Scheduler::resourceOffers took > 82354ns > I0108 01:13:16.227466 1348 master.cpp:3592] Processing DECLINE call for > offers: [ 773d31e8-383d-4e4b-aa68-f9a3fb9f1fc2-O1 ] for framework > 773d31e8-383d-4e4b-aa68-f9a3fb9f1fc2-0000 (no-principal) at > scheduler-bf0ed267-b4c4-412d-9fb0-84c85cd2fbce@172.17.0.2:32801 > I0108 01:13:16.227843 1325 sched.cpp:164] Version: 0.27.0 > I0108 01:13:16.228489 1344 hierarchical.cpp:880] Recovered cpus(*):2; > mem(*):1024; ports(*):[31000-32000]; disk(role1):2048 (total: cpus(*):2; > mem(*):1024; disk(role1):2048; ports(*):[31000-32000], allocated: ) on slave > 773d31e8-383d-4e4b-aa68-f9a3fb9f1fc2-S0 from framework > 773d31e8-383d-4e4b-aa68-f9a3fb9f1fc2-0000 > I0108 01:13:16.228989 1346 sched.cpp:268] New master detected at > master@172.17.0.2:32801 > I0108 01:13:16.229118 1346 sched.cpp:278] No credentials provided. > Attempting to register without authentication > I0108 01:13:16.229143 1346 sched.cpp:722] Sending SUBSCRIBE call to > master@172.17.0.2:32801 > I0108 01:13:16.229277 1346 sched.cpp:755] Will retry registration in > 1.383902465secs if necessary > I0108 01:13:16.229912 1348 master.cpp:2650] Processing SUPPRESS call for > framework 773d31e8-383d-4e4b-aa68-f9a3fb9f1fc2-0000 (no-principal) at > scheduler-bf0ed267-b4c4-412d-9fb0-84c85cd2fbce@172.17.0.2:32801 > I0108 01:13:16.230171 1346 hierarchical.cpp:953] Suppressed offers for > framework 773d31e8-383d-4e4b-aa68-f9a3fb9f1fc2-0000 > I0108 01:13:16.230262 1348 master.cpp:2197] Received SUBSCRIBE call for > framework 'default' at > scheduler-4eccbeb6-560d-41be-b1d6-1e2971db4bb3@172.17.0.2:32801 > I0108 01:13:16.230370 1348 master.cpp:1668] Authorizing framework principal > 'test-principal' to receive offers for role 'role1' > I0108 01:13:16.230788 1348 master.cpp:2268] Subscribing framework default > with checkpointing disabled and capabilities [ ] > I0108 01:13:16.231477 1346 hierarchical.cpp:260] Added framework > 773d31e8-383d-4e4b-aa68-f9a3fb9f1fc2-0001 > I0108 01:13:16.232698 1346 hierarchical.cpp:1423] No inverse offers to send > out! > I0108 01:13:16.232795 1346 hierarchical.cpp:1079] Performed allocation for 1 > slaves in 1.282992ms > I0108 01:13:16.233512 1348 master.cpp:5274] Sending 1 offers to framework > 773d31e8-383d-4e4b-aa68-f9a3fb9f1fc2-0001 (default) at > scheduler-4eccbeb6-560d-41be-b1d6-1e2971db4bb3@172.17.0.2:32801 > I0108 01:13:16.233728 1351 sched.cpp:649] Framework registered with > 773d31e8-383d-4e4b-aa68-f9a3fb9f1fc2-0001 > I0108 01:13:16.233800 1351 sched.cpp:663] Scheduler::registered took 29498ns > I0108 01:13:16.234381 1359 sched.cpp:819] Scheduler::resourceOffers took > 113212ns > I0108 01:13:16.239941 1348 hierarchical.cpp:1329] No resources available to > allocate! > I0108 01:13:16.240223 1348 hierarchical.cpp:1423] No inverse offers to send > out! > I0108 01:13:16.240275 1348 hierarchical.cpp:1079] Performed allocation for 1 > slaves in 633949ns > I0108 01:13:16.251688 1357 master.cpp:3055] Processing ACCEPT call for > offers: [ 773d31e8-383d-4e4b-aa68-f9a3fb9f1fc2-O2 ] on slave > 773d31e8-383d-4e4b-aa68-f9a3fb9f1fc2-S0 at slave(231)@172.17.0.2:32801 > (d9632dd1c41e) for framework 773d31e8-383d-4e4b-aa68-f9a3fb9f1fc2-0001 > (default) at scheduler-4eccbeb6-560d-41be-b1d6-1e2971db4bb3@172.17.0.2:32801 > I0108 01:13:16.251785 1357 master.cpp:2843] Authorizing principal > 'test-principal' to create volumes > I0108 01:13:16.253445 1352 master.cpp:3384] Applying CREATE operation for > volumes disk(role1)[id1:path1]:128 from framework > 773d31e8-383d-4e4b-aa68-f9a3fb9f1fc2-0001 (default) at > scheduler-4eccbeb6-560d-41be-b1d6-1e2971db4bb3@172.17.0.2:32801 to slave > 773d31e8-383d-4e4b-aa68-f9a3fb9f1fc2-S0 at slave(231)@172.17.0.2:32801 > (d9632dd1c41e) > I0108 01:13:16.253911 1352 master.cpp:6508] Sending checkpointed resources > disk(role1)[id1:path1]:128 to slave 773d31e8-383d-4e4b-aa68-f9a3fb9f1fc2-S0 > at slave(231)@172.17.0.2:32801 (d9632dd1c41e) > I0108 01:13:16.255210 1352 slave.cpp:2277] Updated checkpointed resources > from to disk(role1)[id1:path1]:128 > I0108 01:13:16.257128 1356 hierarchical.cpp:642] Updated allocation of > framework 773d31e8-383d-4e4b-aa68-f9a3fb9f1fc2-0001 on slave > 773d31e8-383d-4e4b-aa68-f9a3fb9f1fc2-S0 from cpus(*):2; mem(*):1024; > ports(*):[31000-32000]; disk(role1):2048 to cpus(*):2; mem(*):1024; > ports(*):[31000-32000]; disk(role1):1920; disk(role1)[id1:path1]:128 > I0108 01:13:16.257844 1356 hierarchical.cpp:880] Recovered cpus(*):2; > mem(*):1024; ports(*):[31000-32000]; disk(role1):1920; > disk(role1)[id1:path1]:128 (total: cpus(*):2; mem(*):1024; disk(role1):1920; > ports(*):[31000-32000]; disk(role1)[id1:path1]:128, allocated: ) on slave > 773d31e8-383d-4e4b-aa68-f9a3fb9f1fc2-S0 from framework > 773d31e8-383d-4e4b-aa68-f9a3fb9f1fc2-0001 > I0108 01:13:16.262976 1344 hierarchical.cpp:1423] No inverse offers to send > out! > I0108 01:13:16.263068 1344 hierarchical.cpp:1079] Performed allocation for 1 > slaves in 1.435723ms > I0108 01:13:16.263535 1353 master.cpp:5274] Sending 1 offers to framework > 773d31e8-383d-4e4b-aa68-f9a3fb9f1fc2-0001 (default) at > scheduler-4eccbeb6-560d-41be-b1d6-1e2971db4bb3@172.17.0.2:32801 > I0108 01:13:16.264181 1356 sched.cpp:819] Scheduler::resourceOffers took > 139353ns > I0108 01:13:16.271931 1355 master.cpp:3671] Processing REVIVE call for > framework 773d31e8-383d-4e4b-aa68-f9a3fb9f1fc2-0000 (no-principal) at > scheduler-bf0ed267-b4c4-412d-9fb0-84c85cd2fbce@172.17.0.2:32801 > I0108 01:13:16.272141 1359 hierarchical.cpp:973] Removed offer filters for > framework 773d31e8-383d-4e4b-aa68-f9a3fb9f1fc2-0000 > I0108 01:13:16.272177 1355 master.cpp:3592] Processing DECLINE call for > offers: [ 773d31e8-383d-4e4b-aa68-f9a3fb9f1fc2-O3 ] for framework > 773d31e8-383d-4e4b-aa68-f9a3fb9f1fc2-0001 (default) at > scheduler-4eccbeb6-560d-41be-b1d6-1e2971db4bb3@172.17.0.2:32801 > I0108 01:13:16.272423 1359 hierarchical.cpp:1329] No resources available to > allocate! > I0108 01:13:16.272483 1359 hierarchical.cpp:1423] No inverse offers to send > out! > I0108 01:13:16.272514 1359 hierarchical.cpp:1079] Performed allocation for 1 > slaves in 344563ns > I0108 01:13:16.272924 1355 master.cpp:2650] Processing SUPPRESS call for > framework 773d31e8-383d-4e4b-aa68-f9a3fb9f1fc2-0001 (default) at > scheduler-4eccbeb6-560d-41be-b1d6-1e2971db4bb3@172.17.0.2:32801 > I0108 01:13:16.272989 1359 hierarchical.cpp:880] Recovered cpus(*):2; > mem(*):1024; ports(*):[31000-32000]; disk(role1):1920; > disk(role1)[id1:path1]:128 (total: cpus(*):2; mem(*):1024; disk(role1):1920; > ports(*):[31000-32000]; disk(role1)[id1:path1]:128, allocated: ) on slave > 773d31e8-383d-4e4b-aa68-f9a3fb9f1fc2-S0 from framework > 773d31e8-383d-4e4b-aa68-f9a3fb9f1fc2-0001 > I0108 01:13:16.273309 1359 hierarchical.cpp:953] Suppressed offers for > framework 773d31e8-383d-4e4b-aa68-f9a3fb9f1fc2-0001 > 2016-01-08 > 01:13:18,959:1325(0x7fb7cd6ae700):ZOO_ERROR@handle_socket_error_msg@1697: > Socket [127.0.0.1:50826] zk retcode=-4, errno=111(Connection refused): server > refused to accept the client > 2016-01-08 > 01:13:22,295:1325(0x7fb7cd6ae700):ZOO_ERROR@handle_socket_error_msg@1697: > Socket [127.0.0.1:50826] zk retcode=-4, errno=111(Connection refused): server > refused to accept the client > 2016-01-08 > 01:13:25,631:1325(0x7fb7cd6ae700):ZOO_ERROR@handle_socket_error_msg@1697: > Socket [127.0.0.1:50826] zk retcode=-4, errno=111(Connection refused): server > refused to accept the client > 2016-01-08 > 01:13:28,968:1325(0x7fb7cd6ae700):ZOO_ERROR@handle_socket_error_msg@1697: > Socket [127.0.0.1:50826] zk retcode=-4, errno=111(Connection refused): server > refused to accept the client > ../../src/tests/persistent_volume_tests.cpp:1211: Failure > Failed to wait 15secs for offers > I0108 01:13:31.277577 1354 master.cpp:1130] Framework > 773d31e8-383d-4e4b-aa68-f9a3fb9f1fc2-0001 (default) at > scheduler-4eccbeb6-560d-41be-b1d6-1e2971db4bb3@172.17.0.2:32801 disconnected > I../../src/tests/persistent_volume_tests.cpp:1204: Failure > Actual function call count doesn't match EXPECT_CALL(sched1, > resourceOffers(&driver1, _))... > Expected: to be called once > Actual: never called - unsatisfied and active > 0108 01:13:31.277909 1354 master.cpp:2493] Disconnecting framework > 773d31e8-383d-4e4b-aa68-f9a3fb9f1fc2-0001 (default) at > scheduler-4eccbeb6-560d-41be-b1d6-1e2971db4bb3@172.17.0.2:32801 > I0108 01:13:31.279088 1354 master.cpp:2517] Deactivating framework > 773d31e8-383d-4e4b-aa68-f9a3fb9f1fc2-0001 (default) at > scheduler-4eccbeb6-560d-41be-b1d6-1e2971db4bb3@172.17.0.2:32801 > I0108 01:13:31.279496 1354 master.cpp:1154] Giving framework > 773d31e8-383d-4e4b-aa68-f9a3fb9f1fc2-0001 (default) at > scheduler-4eccbeb6-560d-41be-b1d6-1e2971db4bb3@172.17.0.2:32801 0ns to > failover > I0108 01:13:31.280046 1354 master.cpp:1130] Framework > 773d31e8-383d-4e4b-aa68-f9a3fb9f1fc2-0000 (no-principal) at > scheduler-bf0ed267-b4c4-412d-9fb0-84c85cd2fbce@172.17.0.2:32801 disconnected > I0108 01:13:31.280603 1354 master.cpp:2493] Disconnecting framework > 773d31e8-383d-4e4b-aa68-f9a3fb9f1fc2-0000 (no-principal) at > scheduler-bf0ed267-b4c4-412d-9fb0-84c85cd2fbce@172.17.0.2:32801 > I0108 01:13:31.280644 1354 master.cpp:2517] Deactivating framework > 773d31e8-383d-4e4b-aa68-f9a3fb9f1fc2-0000 (no-principal) at > scheduler-bf0ed267-b4c4-412d-9fb0-84c85cd2fbce@172.17.0.2:32801 > I0108 01:13:31.280863 1354 master.cpp:1154] Giving framework > 773d31e8-383d-4e4b-aa68-f9a3fb9f1fc2-0000 (no-principal) at > scheduler-bf0ed267-b4c4-412d-9fb0-84c85cd2fbce@172.17.0.2:32801 0ns to > failover > I0108 01:13:31.280563 1348 hierarchical.cpp:366] Deactivated framework > 773d31e8-383d-4e4b-aa68-f9a3fb9f1fc2-0001 > I0108 01:13:31.281056 1348 hierarchical.cpp:366] Deactivated framework > 773d31e8-383d-4e4b-aa68-f9a3fb9f1fc2-0000 > I0108 01:13:31.281097 1354 master.cpp:930] Master terminating > I0108 01:13:31.281910 1355 hierarchical.cpp:496] Removed slave > 773d31e8-383d-4e4b-aa68-f9a3fb9f1fc2-S0 > I0108 01:13:31.282516 1352 hierarchical.cpp:321] Removed framework > 773d31e8-383d-4e4b-aa68-f9a3fb9f1fc2-0001 > I0108 01:13:31.282817 1352 hierarchical.cpp:321] Removed framework > 773d31e8-383d-4e4b-aa68-f9a3fb9f1fc2-0000 > I0108 01:13:31.282985 1352 slave.cpp:3417] master@172.17.0.2:32801 exited > W0108 01:13:31.283144 1352 slave.cpp:3420] Master disconnected! Waiting for > a new master to be elected > I0108 01:13:31.313812 1346 slave.cpp:601] Slave terminating > [ FAILED ] PersistentVolumeTest.BadACLNoPrincipal (15203 ms) > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)