Cody, your stack trace looks unrelated to the original issue here? It seems related to checkpointing in the slave.
On Thu, Apr 16, 2015 at 5:02 PM, Benjamin Mahler <benjamin.mah...@gmail.com> wrote: > Hey Cody, the 'F' failure log line was far up from the stack trace, I > edited the stack trace to show it. > > On Thu, Apr 16, 2015 at 2:48 PM, Cody Maloney (JIRA) <j...@apache.org> > wrote: > > > > > [ > > > https://issues.apache.org/jira/browse/MESOS-2144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14498789#comment-14498789 > > ] > > > > Cody Maloney commented on MESOS-2144: > > ------------------------------------- > > > > Just got one of these with full backtrace: > > {code} > > I0416 12:21:01.673476 367333376 authenticatee.hpp:115] Initializing > client > > SASL > > @ 0x110e9284a google::LogMessage::Fail() > > @ 0x110e917dd google::LogMessage::SendToLog() > > @ 0x110e924ea google::LogMessage::Flush() > > @ 0x110e99348 google::LogMessageFatal::~LogMessageFatal() > > I0416 12:21:01.747539 308416512 process.cpp:2091] Resuming reaper(1)@ > > 127.0.0.1:52842 at 2015-04-16 19:21:33.747597056+00:00 > > @ 0x110e92ca5 google::LogMessageFatal::~LogMessageFatal() > > @ 0x10f3d33d3 _CheckFatal::~_CheckFatal() > > @ 0x10f3d3025 _CheckFatal::~_CheckFatal() > > @ 0x10fd94da6 mesos::internal::slave::Slave::__recover() > > @ 0x10fe7f09d > > > _ZZN7process8dispatchIN5mesos8internal5slave5SlaveERKNS_6FutureI7NothingEES7_EEvRKNS_3PIDIT_EEMSB_FvT0_ET1_ENKUlPNS_11ProcessBaseEE_clESK_ > > @ 0x10fe7ee7f > > > _ZNSt3__110__function6__funcIZN7process8dispatchIN5mesos8internal5slave5SlaveERKNS2_6FutureI7NothingEESA_EEvRKNS2_3PIDIT_EEMSE_FvT0_ET1_EUlPNS2_11ProcessBaseEE_NS_9allocatorISO_EEFvSN_EEclEOSN_ > > @ 0x110d74e7b std::__1::function<>::operator()() > > @ 0x110d5c5bf process::ProcessBase::visit() > > @ 0x110de6c0e process::DispatchEvent::visit() > > @ 0x10f3d0841 process::ProcessBase::serve() > > @ 0x110d45abe process::ProcessManager::resume() > > @ 0x110d451de process::schedule() > > @ 0x7fff8f1eb268 _pthread_body > > @ 0x7fff8f1eb1e5 _pthread_start > > @ 0x7fff8f1e941d thread_start > > {code} > > > > The full log from the test (MESOS_VERBOSE, GLOG_v=2) > > {code} > > [ RUN ] ExamplesTest.LowLevelSchedulerPthread > > Using temporary directory > > '/tmp/ExamplesTest_LowLevelSchedulerPthread_vVqryS' > > I0416 12:21:01.637110 2105078528 logging.cpp:177] Logging to STDERR > > Enabling authentication for the scheduler > > I0416 12:21:01.639566 2105078528 process.cpp:2081] Spawned process __ > > gc__@127.0.0.1:52945 > > I0416 12:21:01.639770 2105078528 process.cpp:2081] Spawned process > > help@127.0.0.1:52945 > > I0416 12:21:01.639583 365723648 process.cpp:2091] Resuming __ > > gc__@127.0.0.1:52945 at 2015-04-16 19:21:01.639622912+00:00 > > I0416 12:21:01.639777 367869952 process.cpp:2091] Resuming > > help@127.0.0.1:52945 at 2015-04-16 19:21:01.639796992+00:00 > > I0416 12:21:01.639875 366260224 process.cpp:2091] Resuming > > logging@127.0.0.1:52945 at 2015-04-16 19:21:01.639906816+00:00 > > I0416 12:21:01.639909 2105078528 process.cpp:2081] Spawned process > > logging@127.0.0.1:52945 > > I0416 12:21:01.639978 367869952 process.cpp:2091] Resuming > > profiler@127.0.0.1:52945 at 2015-04-16 19:21:01.640003840+00:00 > > I0416 12:21:01.640033 368943104 process.cpp:2091] Resuming > > help@127.0.0.1:52945 at 2015-04-16 19:21:01.640058880+00:00 > > I0416 12:21:01.640051 2105078528 process.cpp:2081] Spawned process > > profiler@127.0.0.1:52945 > > I0416 12:21:01.640246 368406528 process.cpp:2091] Resuming > > system@127.0.0.1:52945 at 2015-04-16 19:21:01.640268032+00:00 > > I0416 12:21:01.640236 368943104 process.cpp:2091] Resuming __ > > gc__@127.0.0.1:52945 at 2015-04-16 19:21:01.640258048+00:00 > > I0416 12:21:01.640318 2105078528 process.cpp:2081] Spawned process > > system@127.0.0.1:52945 > > I0416 12:21:01.640321 368943104 process.cpp:2091] Resuming > __limiter__(1)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.640336128+00:00 > > I0416 12:21:01.640390 368406528 process.cpp:2081] Spawned process > > __limiter__(1)@127.0.0.1:52945 > > I0416 12:21:01.640425 365723648 process.cpp:2091] Resuming > > metrics@127.0.0.1:52945 at 2015-04-16 19:21:01.640440064+00:00 > > I0416 12:21:01.640472 368406528 process.cpp:2081] Spawned process > > metrics@127.0.0.1:52945 > > I0416 12:21:01.640521 367869952 process.cpp:2091] Resuming > > help@127.0.0.1:52945 at 2015-04-16 19:21:01.640538880+00:00 > > I0416 12:21:01.640733 366796800 process.cpp:2091] Resuming > > help@127.0.0.1:52945 at 2015-04-16 19:21:01.640760064+00:00 > > I0416 12:21:01.640913 2105078528 process.cpp:2081] Spawned process __ > > processes__@127.0.0.1:52945 > > I0416 12:21:01.640919 366796800 process.cpp:2091] Resuming __ > > processes__@127.0.0.1:52945 at 2015-04-16 19:21:01.640937984+00:00 > > I0416 12:21:01.640949 2105078528 process.cpp:912] libprocess is > > initialized on 127.0.0.1:52945 for 8 cpus > > I0416 12:21:01.640971 365723648 process.cpp:2091] Resuming > > help@127.0.0.1:52945 at 2015-04-16 19:21:01.640985856+00:00 > > W0416 12:21:01.641326 2105078528 scheduler.cpp:134] > > ************************************************** > > Scheduler driver bound to loopback interface! Cannot communicate with > > remote master(s). You might want to set 'LIBPROCESS_IP' environment > > variable to use a routable IP address. > > ************************************************** > > I0416 12:21:01.641348 2105078528 scheduler.cpp:149] Version: 0.23.0 > > I0416 12:21:01.642009 2105078528 process.cpp:2081] Spawned process > > hierarchical-allocator(1)@127.0.0.1:52945 > > I0416 12:21:01.642019 368943104 process.cpp:2091] Resuming > > hierarchical-allocator(1)@127.0.0.1:52945 at 2015-04-16 > > 19:21:01.642038016+00:00 > > I0416 12:21:01.642069 365723648 process.cpp:2091] Resuming > > files@127.0.0.1:52945 at 2015-04-16 19:21:01.642086912+00:00 > > I0416 12:21:01.642115 2105078528 process.cpp:2081] Spawned process > > files@127.0.0.1:52945 > > I0416 12:21:01.642194 366260224 process.cpp:2091] Resuming > > help@127.0.0.1:52945 at 2015-04-16 19:21:01.642208000+00:00 > > I0416 12:21:01.647266 310026240 process.cpp:2091] Resuming reaper(1)@ > > 127.0.0.1:52842 at 2015-04-16 19:21:33.647293184+00:00 > > I0416 12:21:01.648769 2105078528 leveldb.cpp:176] Opened db in 2269us > > I0416 12:21:01.649345 2105078528 leveldb.cpp:183] Compacted db in 555us > > I0416 12:21:01.649394 2105078528 leveldb.cpp:198] Created db iterator in > > 27us > > I0416 12:21:01.649420 2105078528 leveldb.cpp:204] Seeked to beginning of > > db in 14us > > I0416 12:21:01.649440 2105078528 leveldb.cpp:273] Iterated through 0 keys > > in the db in 15us > > I0416 12:21:01.649510 2105078528 replica.cpp:744] Replica recovered with > > log positions 0 -> 0 with 1 holes and 0 unlearned > > I0416 12:21:01.650113 2105078528 process.cpp:2081] Spawned process > > log-replica(1)@127.0.0.1:52945 > > I0416 12:21:01.650157 366796800 process.cpp:2091] Resuming > log-replica(1)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.650183936+00:00 > > I0416 12:21:01.650243 368943104 process.cpp:2091] Resuming (1)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.650263040+00:00 > > I0416 12:21:01.650296 2105078528 process.cpp:2081] Spawned process (1)@ > > 127.0.0.1:52945 > > I0416 12:21:01.650413 2105078528 process.cpp:2081] Spawned process > log(1)@ > > 127.0.0.1:52945 > > I0416 12:21:01.650424 365723648 process.cpp:2091] Resuming log(1)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.650441984+00:00 > > I0416 12:21:01.650574 2105078528 process.cpp:2081] Spawned process > > log-reader(1)@127.0.0.1:52945 > > I0416 12:21:01.650580 366260224 process.cpp:2091] Resuming log-reader(1)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.650599936+00:00 > > I0416 12:21:01.650606 368943104 process.cpp:2091] Resuming __ > > gc__@127.0.0.1:52945 at 2015-04-16 19:21:01.650621952+00:00 > > I0416 12:21:01.650607 367333376 process.cpp:2091] Resuming > log-recover(1)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.650633984+00:00 > > I0416 12:21:01.650658 367333376 recover.cpp:449] Starting replica > recovery > > I0416 12:21:01.650636 369479680 process.cpp:2091] Resuming log-writer(1)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.650664960+00:00 > > I0416 12:21:01.650686 2105078528 process.cpp:2081] Spawned process > > log-writer(1)@127.0.0.1:52945 > > I0416 12:21:01.650688 365723648 process.cpp:2081] Spawned process > > log-recover(1)@127.0.0.1:52945 > > I0416 12:21:01.650837 365723648 process.cpp:2091] Resuming > > metrics@127.0.0.1:52945 at 2015-04-16 19:21:01.650851072+00:00 > > I0416 12:21:01.650836 369479680 process.cpp:2091] Resuming > log-replica(1)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.650855936+00:00 > > I0416 12:21:01.650857 366260224 process.cpp:2091] Resuming (2)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.650881024+00:00 > > I0416 12:21:01.650900 2105078528 process.cpp:2081] Spawned process (2)@ > > 127.0.0.1:52945 > > I0416 12:21:01.650966 367333376 recover.cpp:475] Replica is in EMPTY > status > > I0416 12:21:01.651100 367869952 process.cpp:2091] Resuming > > metrics@127.0.0.1:52945 at 2015-04-16 19:21:01.651115008+00:00 > > I0416 12:21:01.651093 368943104 process.cpp:2091] Resuming __ > > gc__@127.0.0.1:52945 at 2015-04-16 19:21:01.651112192+00:00 > > I0416 12:21:01.651120 366260224 process.cpp:2091] Resuming > > log-recover-protocol(1)@127.0.0.1:52945 at 2015-04-16 > > 19:21:01.651134976+00:00 > > I0416 12:21:01.651160 367333376 process.cpp:2081] Spawned process > > log-recover-protocol(1)@127.0.0.1:52945 > > I0416 12:21:01.651211 366260224 recover.cpp:131] Starting to wait for > > enough quorum of replicas before running recovery protocol, expected > quroum > > size: 1 > > I0416 12:21:01.651298 2105078528 process.cpp:2081] Spawned process > > registrar(1)@127.0.0.1:52945 > > I0416 12:21:01.651301 365723648 process.cpp:2091] Resuming registrar(1)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.651323136+00:00 > > I0416 12:21:01.651314 366796800 process.cpp:2091] Resuming (1)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.651334144+00:00 > > I0416 12:21:01.651384 367333376 process.cpp:2091] Resuming > > standalone-master-detector(1)@127.0.0.1:52945 at 2015-04-16 > > 19:21:01.651405056+00:00 > > I0416 12:21:01.651434 2105078528 process.cpp:2081] Spawned process > > standalone-master-detector(1)@127.0.0.1:52945 > > I0416 12:21:01.651576 2105078528 process.cpp:2081] Spawned process > > authorizer(1)@127.0.0.1:52945 > > I0416 12:21:01.651594 365723648 process.cpp:2091] Resuming __ > > gc__@127.0.0.1:52945 at 2015-04-16 19:21:01.651608064+00:00 > > I0416 12:21:01.651600 368406528 process.cpp:2091] Resuming > > help@127.0.0.1:52945 at 2015-04-16 19:21:01.651613952+00:00 > > I0416 12:21:01.651583 367869952 process.cpp:2091] Resuming authorizer(1)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.651602944+00:00 > > I0416 12:21:01.651657 369479680 process.cpp:2091] Resuming __latch__(1)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.651691008+00:00 > > I0416 12:21:01.651705 366260224 process.cpp:2081] Spawned process > > __latch__(1)@127.0.0.1:52945 > > I0416 12:21:01.651888 366260224 recover.cpp:145] Broadcasting recover > > request to all replicas > > I0416 12:21:01.651996 368406528 process.cpp:2091] Resuming (1)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.652011008+00:00 > > I0416 12:21:01.652019 367333376 process.cpp:2091] Resuming > > metrics@127.0.0.1:52945 at 2015-04-16 19:21:01.652039168+00:00 > > I0416 12:21:01.652148 366796800 process.cpp:2091] Resuming __ > > gc__@127.0.0.1:52945 at 2015-04-16 19:21:01.652165888+00:00 > > I0416 12:21:01.652154 367869952 process.cpp:2091] Resuming (3)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.652169984+00:00 > > I0416 12:21:01.652217 368406528 process.cpp:2081] Spawned process (3)@ > > 127.0.0.1:52945 > > I0416 12:21:01.652369 366260224 process.cpp:2091] Resuming (3)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.652391936+00:00 > > I0416 12:21:01.652400 368943104 process.cpp:2091] Resuming > > log-recover-protocol(1)@127.0.0.1:52945 at 2015-04-16 > > 19:21:01.652436992+00:00 > > I0416 12:21:01.652465 368943104 recover.cpp:154] Broadcast request > > completed > > I0416 12:21:01.652479 368406528 process.cpp:2091] Resuming > log-replica(1)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.652500992+00:00 > > I0416 12:21:01.652529 368406528 replica.cpp:641] Replica in EMPTY status > > received a broadcasted recover request > > I0416 12:21:01.652642 366260224 process.cpp:2091] Resuming (3)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.652664064+00:00 > > I0416 12:21:01.652740 366260224 process.cpp:2198] Cleaning up (3)@ > > 127.0.0.1:52945 > > I0416 12:21:01.652799 368943104 recover.cpp:195] Received a recover > > response from a replica in EMPTY status > > I0416 12:21:01.652814 368406528 process.cpp:2091] Resuming __ > > gc__@127.0.0.1:52945 at 2015-04-16 19:21:01.652847104+00:00 > > I0416 12:21:01.652892 368406528 process.cpp:2091] Resuming __latch__(1)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.652925952+00:00 > > I0416 12:21:01.652953 368406528 process.cpp:2198] Cleaning up > __latch__(1)@ > > 127.0.0.1:52945 > > I0416 12:21:01.653031 366796800 process.cpp:2091] Resuming __ > > gc__@127.0.0.1:52945 at 2015-04-16 19:21:01.653053952+00:00 > > I0416 12:21:01.653064 367869952 process.cpp:2091] Resuming > log-recover(1)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.653084160+00:00 > > I0416 12:21:01.653147 367869952 recover.cpp:566] Updating replica status > > to STARTING > > I0416 12:21:01.653162 368943104 process.cpp:2198] Cleaning up > > log-recover-protocol(1)@127.0.0.1:52945 > > I0416 12:21:01.653259 366796800 process.cpp:2091] Resuming __ > > gc__@127.0.0.1:52945 at 2015-04-16 19:21:01.653281024+00:00 > > I0416 12:21:01.653301 368406528 process.cpp:2091] Resuming > log-replica(1)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.653321984+00:00 > > I0416 12:21:01.653651 368406528 leveldb.cpp:306] Persisting metadata (8 > > bytes) to leveldb took 295us > > I0416 12:21:01.653693 368406528 replica.cpp:323] Persisted replica status > > to STARTING > > I0416 12:21:01.653681 365723648 process.cpp:2091] Resuming > > standalone-master-detector(1)@127.0.0.1:52945 at 2015-04-16 > > 19:21:01.653703936+00:00 > > I0416 12:21:01.653806 365723648 process.cpp:2091] Resuming > log-recover(1)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.653827072+00:00 > > I0416 12:21:01.653833 366260224 process.cpp:2091] Resuming > > master@127.0.0.1:52945 at 2015-04-16 19:21:01.653851136+00:00 > > I0416 12:21:01.653863 2105078528 process.cpp:2081] Spawned process > > master@127.0.0.1:52945 > > I0416 12:21:01.653874 366260224 master.cpp:361] Master > > 20150416-122101-16777343-52945-98969 (localhost) started on > > 127.0.0.1:52945 > > I0416 12:21:01.653918 365723648 recover.cpp:475] Replica is in STARTING > > status > > I0416 12:21:01.654013 365723648 process.cpp:2081] Spawned process > > log-recover-protocol(2)@127.0.0.1:52945 > > I0416 12:21:01.654022 367333376 process.cpp:2091] Resuming __ > > gc__@127.0.0.1:52945 at 2015-04-16 19:21:01.654042880+00:00 > > I0416 12:21:01.654026 368406528 process.cpp:2091] Resuming > > log-recover-protocol(2)@127.0.0.1:52945 at 2015-04-16 > > 19:21:01.654049024+00:00 > > I0416 12:21:01.654101 368406528 recover.cpp:131] Starting to wait for > > enough quorum of replicas before running recovery protocol, expected > quroum > > size: 1 > > I0416 12:21:01.654170 367869952 process.cpp:2091] Resuming (1)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.654191872+00:00 > > I0416 12:21:01.653894 366260224 master.cpp:363] Flags at startup: > > --acls="permissive: false > > register_frameworks { > > principals { > > type: SOME > > values: "test-principal" > > } > > roles { > > type: SOME > > values: "*" > > } > > } > > run_tasks { > > principals { > > type: SOME > > values: "test-principal" > > } > > users { > > type: SOME > > values: "osxbuildbot" > > } > > } > > " --allocation_interval="1secs" --authenticate="true" > > --authenticate_slaves="false" --authenticators="crammd5" > > > --credentials="/tmp/ExamplesTest_LowLevelSchedulerPthread_vVqryS/credentials" > > --framework_sorter="drf" --initialize_driver_logging="true" > > --log_auto_initialize="true" --logbufsecs="0" --logging_level="INFO" > > --quiet="false" --recovery_slave_removal_limit="100%" > > --registry="replicated_log" --registry_fetch_timeout="1mins" > > --registry_store_timeout="5secs" --registry_strict="false" > > --root_submissions="true" --slave_reregister_timeout="10mins" > > --user_sorter="drf" --version="false" > > > --webui_dir="/Users/osxbuildbot/buildslave/slave/dev_test_osx/dev_distcheck_build_dist/mesos-0.23.0/src/webui" > > > --work_dir="/var/folders/lx/kp985px11jg_0mpdwnd3ty8r0000gn/T/mesos-XXXXXX.RtaNR5cQ" > > --zk_session_timeout="10secs" > > W0416 12:21:01.654228 366260224 master.cpp:366] > > ************************************************** > > Master bound to loopback interface! Cannot communicate with remote > > schedulers or slaves. You might want to set '--ip' flag to a routable IP > > address. > > ************************************************** > > I0416 12:21:01.654275 368406528 process.cpp:2081] Spawned process > > __latch__(2)@127.0.0.1:52945 > > I0416 12:21:01.654292 366260224 master.cpp:408] Master only allowing > > authenticated frameworks to register > > I0416 12:21:01.654278 369479680 process.cpp:2091] Resuming __ > > gc__@127.0.0.1:52945 at 2015-04-16 19:21:01.654299136+00:00 > > I0416 12:21:01.654280 366796800 process.cpp:2091] Resuming __latch__(2)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.654301952+00:00 > > I0416 12:21:01.654309 366260224 master.cpp:415] Master allowing > > unauthenticated slaves to register > > I0416 12:21:01.654340 366260224 credentials.hpp:37] Loading credentials > > for authentication from > > '/tmp/ExamplesTest_LowLevelSchedulerPthread_vVqryS/credentials' > > I0416 12:21:01.654362 368406528 recover.cpp:145] Broadcasting recover > > request to all replicas > > I0416 12:21:01.654424 365723648 process.cpp:2091] Resuming (1)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.654445056+00:00 > > I0416 12:21:01.654500 366260224 master.cpp:452] Using default 'crammd5' > > authenticator > > I0416 12:21:01.654549 366260224 authenticator.cpp:511] Initializing > server > > SASL > > I0416 12:21:01.654570 368406528 process.cpp:2091] Resuming __ > > gc__@127.0.0.1:52945 at 2015-04-16 19:21:01.654592000+00:00 > > I0416 12:21:01.654582 368943104 process.cpp:2091] Resuming (4)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.654604032+00:00 > > I0416 12:21:01.654599 365723648 process.cpp:2081] Spawned process (4)@ > > 127.0.0.1:52945 > > I0416 12:21:01.654757 368943104 process.cpp:2091] Resuming (4)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.654782976+00:00 > > I0416 12:21:01.654768 367869952 process.cpp:2091] Resuming > > log-recover-protocol(2)@127.0.0.1:52945 at 2015-04-16 > > 19:21:01.654791168+00:00 > > I0416 12:21:01.654835 367869952 recover.cpp:154] Broadcast request > > completed > > I0416 12:21:01.654892 368406528 process.cpp:2091] Resuming > log-replica(1)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.654915072+00:00 > > I0416 12:21:01.654942 368406528 replica.cpp:641] Replica in STARTING > > status received a broadcasted recover request > > I0416 12:21:01.655048 367333376 process.cpp:2091] Resuming (4)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.655097088+00:00 > > I0416 12:21:01.655110 366796800 process.cpp:2091] Resuming (5)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.655124992+00:00 > > I0416 12:21:01.655139 366796800 process.cpp:2091] Resuming > > log-recover-protocol(2)@127.0.0.1:52945 at 2015-04-16 > > 19:21:01.655148032+00:00 > > I0416 12:21:01.655169 2105078528 process.cpp:2081] Spawned process (5)@ > > 127.0.0.1:52945 > > I0416 12:21:01.655215 366796800 recover.cpp:195] Received a recover > > response from a replica in STARTING status > > I0416 12:21:01.655246 367333376 process.cpp:2198] Cleaning up (4)@ > > 127.0.0.1:52945 > > I0416 12:21:01.655344 368406528 process.cpp:2091] Resuming __ > > gc__@127.0.0.1:52945 at 2015-04-16 19:21:01.655368192+00:00 > > I0416 12:21:01.655405 365723648 process.cpp:2091] Resuming __latch__(2)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.655430144+00:00 > > I0416 12:21:01.655441 2105078528 process.cpp:2081] Spawned process (6)@ > > 127.0.0.1:52945 > > I0416 12:21:01.655452 365723648 process.cpp:2198] Cleaning up > __latch__(2)@ > > 127.0.0.1:52945 > > I0416 12:21:01.655447 367333376 process.cpp:2091] Resuming (6)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.655467008+00:00 > > I0416 12:21:01.655568 369479680 process.cpp:2091] Resuming __ > > gc__@127.0.0.1:52945 at 2015-04-16 19:21:01.655589120+00:00 > > I0416 12:21:01.655596 367333376 process.cpp:2091] Resuming > log-recover(1)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.655617024+00:00 > > I0416 12:21:01.655603 367869952 process.cpp:2091] Resuming __ > > fetcher__@127.0.0.1:52945 at 2015-04-16 19:21:01.655620096+00:00 > > I0416 12:21:01.655654 2105078528 process.cpp:2081] Spawned process __ > > fetcher__@127.0.0.1:52945 > > I0416 12:21:01.655663 367333376 recover.cpp:566] Updating replica status > > to VOTING > > I0416 12:21:01.655693 366796800 process.cpp:2198] Cleaning up > > log-recover-protocol(2)@127.0.0.1:52945 > > I0416 12:21:01.655778 365723648 process.cpp:2091] Resuming > log-replica(1)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.655799040+00:00 > > I0416 12:21:01.655797 368943104 process.cpp:2091] Resuming __ > > gc__@127.0.0.1:52945 at 2015-04-16 19:21:01.655819776+00:00 > > I0416 12:21:01.655854 2105078528 containerizer.cpp:113] Using isolation: > > posix/cpu,posix/mem > > I0416 12:21:01.656002 2105078528 process.cpp:2081] Spawned process (7)@ > > 127.0.0.1:52945 > > I0416 12:21:01.655997 365723648 leveldb.cpp:306] Persisting metadata (8 > > bytes) to leveldb took 193us > > I0416 12:21:01.656008 368406528 process.cpp:2091] Resuming (7)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.656030976+00:00 > > I0416 12:21:01.656043 365723648 replica.cpp:323] Persisted replica status > > to VOTING > > I0416 12:21:01.656143 2105078528 process.cpp:2081] Spawned process (8)@ > > 127.0.0.1:52945 > > I0416 12:21:01.656160 368943104 process.cpp:2091] Resuming (8)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.656179968+00:00 > > I0416 12:21:01.656152 368406528 process.cpp:2091] Resuming > log-recover(1)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.656173056+00:00 > > I0416 12:21:01.656216 368406528 recover.cpp:580] Successfully joined the > > Paxos group > > I0416 12:21:01.656354 368406528 recover.cpp:464] Recover process > terminated > > I0416 12:21:01.656347 365723648 process.cpp:2091] Resuming log(1)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.656369920+00:00 > > I0416 12:21:01.656373 368406528 process.cpp:2198] Cleaning up > > log-recover(1)@127.0.0.1:52945 > > I0416 12:21:01.656404 365723648 log.cpp:358] Log recovery completed > > I0416 12:21:01.656476 367869952 process.cpp:2091] Resuming __ > > gc__@127.0.0.1:52945 at 2015-04-16 19:21:01.656493824+00:00 > > I0416 12:21:01.656512 369479680 process.cpp:2091] Resuming > > metrics@127.0.0.1:52945 at 2015-04-16 19:21:01.656531200+00:00 > > I0416 12:21:01.656528 367869952 process.cpp:2091] Resuming log-reader(1)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.656544000+00:00 > > I0416 12:21:01.656625 366796800 process.cpp:2091] Resuming log-writer(1)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.656647936+00:00 > > I0416 12:21:01.656646 2105078528 process.cpp:2081] Spawned process (9)@ > > 127.0.0.1:52945 > > I0416 12:21:01.656658 367333376 process.cpp:2091] Resuming (9)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.656680192+00:00 > > I0416 12:21:01.657052 368943104 process.cpp:2091] Resuming > > metrics@127.0.0.1:52945 at 2015-04-16 19:21:01.657074944+00:00 > > I0416 12:21:01.657438 2105078528 process.cpp:2081] Spawned process > > __limiter__(2)@127.0.0.1:52945 > > I0416 12:21:01.657449 367869952 process.cpp:2091] Resuming > __limiter__(2)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.657474048+00:00 > > I0416 12:21:01.657488 366796800 process.cpp:2091] Resuming > > monitor@127.0.0.1:52945 at 2015-04-16 19:21:01.657510144+00:00 > > I0416 12:21:01.657542 2105078528 process.cpp:2081] Spawned process > > monitor@127.0.0.1:52945 > > I0416 12:21:01.657567 365723648 process.cpp:2091] Resuming > > help@127.0.0.1:52945 at 2015-04-16 19:21:01.657587968+00:00 > > I0416 12:21:01.657670 2105078528 process.cpp:2081] Spawned process > > slave(1)@127.0.0.1:52945 > > I0416 12:21:01.657703 367333376 process.cpp:2091] Resuming slave(1)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.657718784+00:00 > > I0416 12:21:01.657738 367333376 slave.cpp:176] Slave started on 1)@ > > 127.0.0.1:52945 > > I0416 12:21:01.657760 367333376 slave.cpp:177] Flags at startup: > > --authenticatee="crammd5" --container_disk_watch_interval="15secs" > > --containerizers="mesos" --default_role="*" --disk_watch_interval="1mins" > > --docker="docker" --docker_remove_delay="6hrs" > > --docker_sandbox_directory="/mnt/mesos/sandbox" > --docker_stop_timeout="0ns" > > --enforce_container_disk_quota="false" > > --executor_registration_timeout="1mins" > > --executor_shutdown_grace_period="5secs" --frameworks_home="" > > --gc_delay="1weeks" --gc_disk_headroom="0.1" --hadoop_home="" > > --initialize_driver_logging="true" --isolation="posix/cpu,posix/mem" > > > --launcher_dir="/Users/osxbuildbot/buildslave/slave/dev_test_osx/dev_distcheck_build_dist/mesos-0.23.0/_build/src" > > --logbufsecs="0" --logging_level="INFO" --quiet="false" > > --recover="reconnect" --recovery_timeout="15mins" > > --registration_backoff_factor="1secs" > > --resource_monitoring_interval="1secs" --resources="cpus:2;mem:10240" > > --strict="true" --switch_user="true" --version="false" > > > --work_dir="/var/folders/lx/kp985px11jg_0mpdwnd3ty8r0000gn/T/mesos-XXXXXX.RtaNR5cQ/0" > > W0416 12:21:01.657997 367333376 slave.cpp:180] > > ************************************************** > > Slave bound to loopback interface! Cannot communicate with remote > > master(s). You might want to set '--ip' flag to a routable IP address. > > ************************************************** > > I0416 12:21:01.658305 367333376 slave.cpp:316] Slave resources: > cpus(*):2; > > mem(*):10240; disk(*):233432; ports(*):[31000-32000] > > I0416 12:21:01.658526 2105078528 process.cpp:2081] Spawned process (10)@ > > 127.0.0.1:52945 > > I0416 12:21:01.658538 366796800 process.cpp:2091] Resuming (10)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.658558976+00:00 > > I0416 12:21:01.658727 2105078528 process.cpp:2081] Spawned process (11)@ > > 127.0.0.1:52945 > > I0416 12:21:01.658721 367333376 slave.cpp:346] Slave hostname: localhost > > I0416 12:21:01.658736 368943104 process.cpp:2091] Resuming (11)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.658759168+00:00 > > I0416 12:21:01.658768 367333376 slave.cpp:351] Slave checkpoint: true > > I0416 12:21:01.658927 2105078528 containerizer.cpp:113] Using isolation: > > posix/cpu,posix/mem > > I0416 12:21:01.658936 367869952 process.cpp:2091] Resuming (6)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.658972928+00:00 > > I0416 12:21:01.659078 2105078528 process.cpp:2081] Spawned process (12)@ > > 127.0.0.1:52945 > > I0416 12:21:01.659090 365723648 process.cpp:2091] Resuming (12)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.659112960+00:00 > > I0416 12:21:01.659210 2105078528 process.cpp:2081] Spawned process (13)@ > > 127.0.0.1:52945 > > I0416 12:21:01.659231 366796800 process.cpp:2091] Resuming (13)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.659246848+00:00 > > I0416 12:21:01.659433 369479680 process.cpp:2091] Resuming > > metrics@127.0.0.1:52945 at 2015-04-16 19:21:01.659454976+00:00 > > I0416 12:21:01.659456 365723648 process.cpp:2091] Resuming (14)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.659475968+00:00 > > I0416 12:21:01.659478 2105078528 process.cpp:2081] Spawned process (14)@ > > 127.0.0.1:52945 > > I0416 12:21:01.659782 367869952 process.cpp:2091] Resuming > > metrics@127.0.0.1:52945 at 2015-04-16 19:21:01.659796992+00:00 > > I0416 12:21:01.660182 2105078528 process.cpp:2081] Spawned process > > __limiter__(3)@127.0.0.1:52945 > > I0416 12:21:01.660195 365723648 process.cpp:2091] Resuming > __limiter__(3)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.660218112+00:00 > > I0416 12:21:01.660320 2105078528 process.cpp:2081] Spawned process > > slave(2)@127.0.0.1:52945 > > I0416 12:21:01.660328 365723648 process.cpp:2091] Resuming slave(2)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.660350976+00:00 > > I0416 12:21:01.660370 365723648 slave.cpp:176] Slave started on 2)@ > > 127.0.0.1:52945 > > I0416 12:21:01.660398 365723648 slave.cpp:177] Flags at startup: > > --authenticatee="crammd5" --container_disk_watch_interval="15secs" > > --containerizers="mesos" --default_role="*" --disk_watch_interval="1mins" > > --docker="docker" --docker_remove_delay="6hrs" > > --docker_sandbox_directory="/mnt/mesos/sandbox" > --docker_stop_timeout="0ns" > > --enforce_container_disk_quota="false" > > --executor_registration_timeout="1mins" > > --executor_shutdown_grace_period="5secs" --frameworks_home="" > > --gc_delay="1weeks" --gc_disk_headroom="0.1" --hadoop_home="" > > --initialize_driver_logging="true" --isolation="posix/cpu,posix/mem" > > > --launcher_dir="/Users/osxbuildbot/buildslave/slave/dev_test_osx/dev_distcheck_build_dist/mesos-0.23.0/_build/src" > > --logbufsecs="0" --logging_level="INFO" --quiet="false" > > --recover="reconnect" --recovery_timeout="15mins" > > --registration_backoff_factor="1secs" > > --resource_monitoring_interval="1secs" --resources="cpus:2;mem:10240" > > --strict="true" --switch_user="true" --version="false" > > > --work_dir="/var/folders/lx/kp985px11jg_0mpdwnd3ty8r0000gn/T/mesos-XXXXXX.RtaNR5cQ/1" > > W0416 12:21:01.660644 365723648 slave.cpp:180] > > ************************************************** > > Slave bound to loopback interface! Cannot communicate with remote > > master(s). You might want to set '--ip' flag to a routable IP address. > > ************************************************** > > I0416 12:21:01.660853 365723648 slave.cpp:316] Slave resources: > cpus(*):2; > > mem(*):10240; disk(*):233432; ports(*):[31000-32000] > > I0416 12:21:01.661152 365723648 slave.cpp:346] Slave hostname: localhost > > I0416 12:21:01.661175 365723648 slave.cpp:351] Slave checkpoint: true > > I0416 12:21:01.661226 368406528 process.cpp:2091] Resuming > > help@127.0.0.1:52945 at 2015-04-16 19:21:01.661250048+00:00 > > I0416 12:21:01.661245 368943104 process.cpp:2091] Resuming (11)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.661270016+00:00 > > I0416 12:21:01.661363 368406528 process.cpp:2091] Resuming __ > > gc__@127.0.0.1:52945 at 2015-04-16 19:21:01.661381888+00:00 > > I0416 12:21:01.661384 2105078528 process.cpp:2081] Spawned process (15)@ > > 127.0.0.1:52945 > > I0416 12:21:01.661396 367333376 process.cpp:2081] Spawned process > > __async_executor__(1)@127.0.0.1:52945 > > I0416 12:21:01.661409 367869952 process.cpp:2091] Resuming (15)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.661426176+00:00 > > I0416 12:21:01.661424 368943104 process.cpp:2091] Resuming > > __async_executor__(1)@127.0.0.1:52945 at 2015-04-16 > > 19:21:01.661444864+00:00 > > I0416 12:21:01.661437 369479680 process.cpp:2091] Resuming > > help@127.0.0.1:52945 at 2015-04-16 19:21:01.661457920+00:00 > > I0416 12:21:01.661505 368943104 state.cpp:35] Recovering state from > > > '/var/folders/lx/kp985px11jg_0mpdwnd3ty8r0000gn/T/mesos-XXXXXX.RtaNR5cQ/0/meta' > > I0416 12:21:01.661540 365723648 process.cpp:2081] Spawned process > > __async_executor__(2)@127.0.0.1:52945 > > I0416 12:21:01.661530 369479680 process.cpp:2091] Resuming __ > > gc__@127.0.0.1:52945 at 2015-04-16 19:21:01.661549056+00:00 > > I0416 12:21:01.661551 367869952 process.cpp:2091] Resuming > > __async_executor__(2)@127.0.0.1:52945 at 2015-04-16 > > 19:21:01.661587968+00:00 > > I0416 12:21:01.661619 2105078528 process.cpp:2081] Spawned process (16)@ > > 127.0.0.1:52945 > > I0416 12:21:01.661615 366796800 process.cpp:2091] Resuming (16)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.661634816+00:00 > > I0416 12:21:01.661654 367869952 state.cpp:35] Recovering state from > > > '/var/folders/lx/kp985px11jg_0mpdwnd3ty8r0000gn/T/mesos-XXXXXX.RtaNR5cQ/1/meta' > > I0416 12:21:01.661659 368943104 process.cpp:2198] Cleaning up > > __async_executor__(1)@127.0.0.1:52945 > > I0416 12:21:01.661775 368406528 process.cpp:2091] Resuming __ > > gc__@127.0.0.1:52945 at 2015-04-16 19:21:01.661796096+00:00 > > I0416 12:21:01.661835 2105078528 containerizer.cpp:113] Using isolation: > > posix/cpu,posix/mem > > I0416 12:21:01.661828 365723648 process.cpp:2091] Resuming slave(2)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.661849856+00:00 > > I0416 12:21:01.661828 368406528 process.cpp:2091] Resuming (6)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.661850880+00:00 > > I0416 12:21:01.661852 367869952 process.cpp:2198] Cleaning up > > __async_executor__(2)@127.0.0.1:52945 > > I0416 12:21:01.661906 2105078528 process.cpp:2081] Spawned process (17)@ > > 127.0.0.1:52945 > > I0416 12:21:01.661903 368406528 status_update_manager.cpp:197] Recovering > > status update manager > > I0416 12:21:01.661912 367333376 process.cpp:2091] Resuming (17)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.661935104+00:00 > > I0416 12:21:01.661911 366796800 process.cpp:2091] Resuming (11)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.661954048+00:00 > > I0416 12:21:01.661938 369479680 process.cpp:2091] Resuming __ > > gc__@127.0.0.1:52945 at 2015-04-16 19:21:01.661967872+00:00 > > I0416 12:21:01.661979 366796800 status_update_manager.cpp:197] Recovering > > status update manager > > I0416 12:21:01.661993 2105078528 process.cpp:2081] Spawned process (18)@ > > 127.0.0.1:52945 > > I0416 12:21:01.662013 365723648 process.cpp:2091] Resuming (18)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.662036992+00:00 > > I0416 12:21:01.662022 367869952 process.cpp:2091] Resuming slave(1)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.662044928+00:00 > > I0416 12:21:01.662078 368406528 process.cpp:2091] Resuming slave(2)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.662102016+00:00 > > I0416 12:21:01.662204 369479680 process.cpp:2091] Resuming (9)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.662226944+00:00 > > I0416 12:21:01.662245 366796800 process.cpp:2091] Resuming (14)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.662259200+00:00 > > I0416 12:21:01.662256 369479680 containerizer.cpp:310] Recovering > > containerizer > > I0416 12:21:01.662277 366796800 containerizer.cpp:310] Recovering > > containerizer > > I0416 12:21:01.662314 367869952 process.cpp:2091] Resuming > > metrics@127.0.0.1:52945 at 2015-04-16 19:21:01.662337024+00:00 > > I0416 12:21:01.662366 2105078528 process.cpp:2081] Spawned process (19)@ > > 127.0.0.1:52945 > > I0416 12:21:01.662379 368406528 process.cpp:2091] Resuming (19)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.662401024+00:00 > > I0416 12:21:01.662529 365723648 process.cpp:2091] Resuming (12)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.662550016+00:00 > > I0416 12:21:01.662535 367869952 process.cpp:2091] Resuming (7)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.662552064+00:00 > > I0416 12:21:01.662574 368943104 process.cpp:2091] Resuming (13)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.662606080+00:00 > > I0416 12:21:01.662621 367869952 process.cpp:2091] Resuming (8)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.662644992+00:00 > > I0416 12:21:01.662680 368406528 process.cpp:2091] Resuming __ > > gc__@127.0.0.1:52945 at 2015-04-16 19:21:01.662700800+00:00 > > I0416 12:21:01.662730 368943104 process.cpp:2091] Resuming (20)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.662752000+00:00 > > I0416 12:21:01.662750 367333376 process.cpp:2091] Resuming (21)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.662771968+00:00 > > I0416 12:21:01.662768 366796800 process.cpp:2081] Spawned process (20)@ > > 127.0.0.1:52945 > > I0416 12:21:01.662792 369479680 process.cpp:2081] Spawned process (21)@ > > 127.0.0.1:52945 > > I0416 12:21:01.662801 367869952 process.cpp:2091] Resuming > > metrics@127.0.0.1:52945 at 2015-04-16 19:21:01.662818048+00:00 > > I0416 12:21:01.662976 365723648 process.cpp:2091] Resuming (9)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.662996992+00:00 > > I0416 12:21:01.663003 367333376 process.cpp:2198] Cleaning up (21)@ > > 127.0.0.1:52945 > > I0416 12:21:01.662993 368406528 process.cpp:2091] Resuming (14)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.663014912+00:00 > > I0416 12:21:01.663022 368943104 process.cpp:2198] Cleaning up (20)@ > > 127.0.0.1:52945 > > I0416 12:21:01.663079 366796800 process.cpp:2091] Resuming __ > > gc__@127.0.0.1:52945 at 2015-04-16 19:21:01.663098112+00:00 > > I0416 12:21:01.663130 366796800 process.cpp:2091] Resuming slave(1)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.663148032+00:00 > > I0416 12:21:01.663162 369479680 process.cpp:2091] Resuming __ > > gc__@127.0.0.1:52945 at 2015-04-16 19:21:01.663183872+00:00 > > I0416 12:21:01.663200 365723648 process.cpp:2091] Resuming slave(2)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.663222016+00:00 > > I0416 12:21:01.663219 367333376 process.cpp:2091] Resuming > > metrics@127.0.0.1:52945 at 2015-04-16 19:21:01.663239936+00:00 > > I0416 12:21:01.663233 366796800 slave.cpp:3806] Finished recovery > > I0416 12:21:01.663290 365723648 slave.cpp:3806] Finished recovery > > I0416 12:21:01.663611 368943104 process.cpp:2091] Resuming > > metrics@127.0.0.1:52945 at 2015-04-16 19:21:01.663631872+00:00 > > I0416 12:21:01.663646 2105078528 process.cpp:2081] Spawned process > > __limiter__(4)@127.0.0.1:52945 > > I0416 12:21:01.663648 369479680 process.cpp:2091] Resuming > __limiter__(4)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.663665920+00:00 > > I0416 12:21:01.663733 2105078528 process.cpp:2081] Spawned process > > slave(3)@127.0.0.1:52945 > > I0416 12:21:01.663791 367869952 process.cpp:2091] Resuming slave(3)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.663815936+00:00 > > I0416 12:21:01.663836 367869952 slave.cpp:176] Slave started on 3)@ > > 127.0.0.1:52945 > > I0416 12:21:01.663892 2105078528 pid.cpp:82] Attempting to parse ' > > master@127.0.0.1:52945' into a PID > > F0416 12:21:01.664001 365723648 slave.cpp:3816] > > CHECK_SOME(state::checkpoint(path, bootId.get())): Failed to rename > > > '/var/folders/lx/kp985px11jg_0mpdwnd3ty8r0000gn/T/mesos-XXXXXX.RtaNR5cQ/0/meta/EntzLz' > > to > > > '/var/folders/lx/kp985px11jg_0mpdwnd3ty8r0000gn/T/mesos-XXXXXX.RtaNR5cQ/1/meta/boot_id': > > No such file or directory > > *** Check failure stack trace: *** > > I0416 12:21:01.663998 368406528 process.cpp:2091] Resuming > > standalone-master-detector(1)@127.0.0.1:52945 at 2015-04-16 > > 19:21:01.664019968+00:00 > > I0416 12:21:01.664113 367333376 process.cpp:2091] Resuming slave(1)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.664136960+00:00 > > I0416 12:21:01.664203 367333376 pid.cpp:82] Attempting to parse ' > > master@127.0.0.1:52945' into a PID > > I0416 12:21:01.664191 369479680 process.cpp:2091] Resuming (6)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.664210944+00:00 > > I0416 12:21:01.663877 367869952 slave.cpp:177] Flags at startup: > > --authenticatee="crammd5" --container_disk_watch_interval="15secs" > > --containerizers="mesos" --default_role="*" --disk_watch_interval="1mins" > > --docker="docker" --docker_remove_delay="6hrs" > > --docker_sandbox_directory="/mnt/mesos/sandbox" > --docker_stop_timeout="0ns" > > --enforce_container_disk_quota="false" > > --executor_registration_timeout="1mins" > > --executor_shutdown_grace_period="5secs" --frameworks_home="" > > --gc_delay="1weeks" --gc_disk_headroom="0.1" --hadoop_home="" > > --initialize_driver_logging="true" --isolation="posix/cpu,posix/mem" > > > --launcher_dir="/Users/osxbuildbot/buildslave/slave/dev_test_osx/dev_distcheck_build_dist/mesos-0.23.0/_build/src" > > --logbufsecs="0" --logging_level="INFO" --quiet="false" > > --recover="reconnect" --recovery_timeout="15mins" > > --registration_backoff_factor="1secs" > > --resource_monitoring_interval="1secs" --resources="cpus:2;mem:10240" > > --strict="true" --switch_user="true" --version="false" > > > --work_dir="/var/folders/lx/kp985px11jg_0mpdwnd3ty8r0000gn/T/mesos-XXXXXX.RtaNR5cQ/2" > > I0416 12:21:01.664229 369479680 status_update_manager.cpp:171] Pausing > > sending status updates > > W0416 12:21:01.664244 367869952 slave.cpp:180] > > ************************************************** > > Slave bound to loopback interface! Cannot communicate with remote > > master(s). You might want to set '--ip' flag to a routable IP address. > > ************************************************** > > I0416 12:21:01.664242 367333376 slave.cpp:627] New master detected at > > master@127.0.0.1:52945 > > I0416 12:21:01.664312 367333376 slave.cpp:652] No credentials provided. > > Attempting to register without authentication > > I0416 12:21:01.664378 367333376 slave.cpp:663] Detecting new master > > I0416 12:21:01.664428 366796800 process.cpp:2091] Resuming > > standalone-master-detector(1)@127.0.0.1:52945 at 2015-04-16 > > 19:21:01.664446208+00:00 > > I0416 12:21:01.664546 367869952 slave.cpp:316] Slave resources: > cpus(*):2; > > mem(*):10240; disk(*):233432; ports(*):[31000-32000] > > I0416 12:21:01.664926 367869952 slave.cpp:346] Slave hostname: localhost > > I0416 12:21:01.664950 367869952 slave.cpp:351] Slave checkpoint: true > > I0416 12:21:01.665007 368406528 process.cpp:2091] Resuming (16)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.665029888+00:00 > > I0416 12:21:01.665225 369479680 process.cpp:2091] Resuming > > help@127.0.0.1:52945 at 2015-04-16 19:21:01.665245184+00:00 > > I0416 12:21:01.665330 367333376 process.cpp:2091] Resuming __ > > gc__@127.0.0.1:52945 at 2015-04-16 19:21:01.665344000+00:00 > > I0416 12:21:01.665340 366796800 process.cpp:2091] Resuming > > __async_executor__(3)@127.0.0.1:52945 at 2015-04-16 > > 19:21:01.665360896+00:00 > > I0416 12:21:01.665375 367869952 process.cpp:2081] Spawned process > > __async_executor__(3)@127.0.0.1:52945 > > I0416 12:21:01.665475 367333376 process.cpp:2091] Resuming > > __async_executor__(3)@127.0.0.1:52945 at 2015-04-16 > > 19:21:01.665491968+00:00 > > I0416 12:21:01.665511 367333376 state.cpp:35] Recovering state from > > > '/var/folders/lx/kp985px11jg_0mpdwnd3ty8r0000gn/T/mesos-XXXXXX.RtaNR5cQ/2/meta' > > I0416 12:21:01.665587 367333376 process.cpp:2198] Cleaning up > > __async_executor__(3)@127.0.0.1:52945 > > I0416 12:21:01.665627 369479680 process.cpp:2091] Resuming (16)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.665649920+00:00 > > I0416 12:21:01.665645 368943104 process.cpp:2091] Resuming __ > > gc__@127.0.0.1:52945 at 2015-04-16 19:21:01.665666048+00:00 > > I0416 12:21:01.665680 369479680 status_update_manager.cpp:197] Recovering > > status update manager > > I0416 12:21:01.665772 368406528 process.cpp:2091] Resuming slave(3)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.665793792+00:00 > > I0416 12:21:01.665886 367333376 process.cpp:2091] Resuming (19)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.665904896+00:00 > > I0416 12:21:01.665922 367333376 containerizer.cpp:310] Recovering > > containerizer > > I0416 12:21:01.666064 369479680 process.cpp:2091] Resuming (18)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.666081792+00:00 > > I0416 12:21:01.666057 368943104 process.cpp:2091] Resuming (17)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.666080000+00:00 > > I0416 12:21:01.666128 369479680 process.cpp:2091] Resuming __ > > gc__@127.0.0.1:52945 at 2015-04-16 19:21:01.666151168+00:00 > > I0416 12:21:01.666173 367333376 process.cpp:2081] Spawned process (22)@ > > 127.0.0.1:52945 > > I0416 12:21:01.666196 368943104 process.cpp:2091] Resuming (22)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.666230016+00:00 > > I0416 12:21:01.666328 366796800 process.cpp:2091] Resuming (19)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.666349824+00:00 > > I0416 12:21:01.666349 368943104 process.cpp:2198] Cleaning up (22)@ > > 127.0.0.1:52945 > > I0416 12:21:01.666451 367869952 process.cpp:2091] Resuming __ > > gc__@127.0.0.1:52945 at 2015-04-16 19:21:01.666472960+00:00 > > I0416 12:21:01.666476 368406528 process.cpp:2091] Resuming slave(3)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.666493952+00:00 > > I0416 12:21:01.666559 368406528 slave.cpp:3806] Finished recovery > > I0416 12:21:01.668668 368943104 process.cpp:2091] Resuming > > standalone-master-detector(1)@127.0.0.1:52945 at 2015-04-16 > > 19:21:01.668697088+00:00 > > I0416 12:21:01.668804 367869952 process.cpp:2091] Resuming slave(3)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.668827904+00:00 > > I0416 12:21:01.668913 367869952 pid.cpp:82] Attempting to parse ' > > master@127.0.0.1:52945' into a PID > > I0416 12:21:01.668900 368406528 process.cpp:2091] Resuming (16)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.668921856+00:00 > > I0416 12:21:01.668959 368406528 status_update_manager.cpp:171] Pausing > > sending status updates > > I0416 12:21:01.668977 367869952 slave.cpp:627] New master detected at > > master@127.0.0.1:52945 > > I0416 12:21:01.669018 367869952 slave.cpp:652] No credentials provided. > > Attempting to register without authentication > > I0416 12:21:01.669067 367869952 slave.cpp:663] Detecting new master > > I0416 12:21:01.669113 368943104 process.cpp:2091] Resuming > > standalone-master-detector(1)@127.0.0.1:52945 at 2015-04-16 > > 19:21:01.669146112+00:00 > > I0416 12:21:01.672554 2105078528 process.cpp:2081] Spawned process > > standalone-master-detector(2)@127.0.0.1:52945 > > I0416 12:21:01.672559 366796800 process.cpp:2091] Resuming > > standalone-master-detector(2)@127.0.0.1:52945 at 2015-04-16 > > 19:21:01.672573184+00:00 > > I0416 12:21:01.672622 2105078528 process.cpp:2081] Spawned process > > scheduler(1)@127.0.0.1:52945 > > I0416 12:21:01.672631 369479680 process.cpp:2091] Resuming scheduler(1)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.672656128+00:00 > > I0416 12:21:01.672837 368406528 process.cpp:2091] Resuming > > standalone-master-detector(2)@127.0.0.1:52945 at 2015-04-16 > > 19:21:01.672858880+00:00 > > I0416 12:21:01.672935 369479680 process.cpp:2091] Resuming scheduler(1)@ > > 127.0.0.1:52945 at 2015-04-16 19:21:01.672957952+00:00 > > I0416 12:21:01.672996 369479680 pid.cpp:82] Attempting to parse ' > > master@127.0.0.1:52945' into a PID > > I0416 12:21:01.673032 369479680 scheduler.cpp:420] New master detected at > > master@127.0.0.1:52945 > > I0416 12:21:01.673056 369479680 scheduler.cpp:469] Authenticating with > > master master@127.0.0.1:52945 > > I0416 12:21:01.673131 369479680 process.cpp:2081] Spawned process > > crammd5_authenticatee(1)@127.0.0.1:52945 > > I0416 12:21:01.673141 367333376 process.cpp:2091] Resuming > > crammd5_authenticatee(1)@127.0.0.1:52945 at 2015-04-16 > > 19:21:01.673160192+00:00 > > I0416 12:21:01.673285 366796800 process.cpp:2091] Resuming > > standalone-master-detector(2)@127.0.0.1:52945 at 2015-04-16 > > 19:21:01.673306880+00:00 > > I0416 12:21:01.673476 367333376 authenticatee.hpp:115] Initializing > client > > SASL > > @ 0x110e9284a google::LogMessage::Fail() > > @ 0x110e917dd google::LogMessage::SendToLog() > > @ 0x110e924ea google::LogMessage::Flush() > > @ 0x110e99348 google::LogMessageFatal::~LogMessageFatal() > > I0416 12:21:01.747539 308416512 process.cpp:2091] Resuming reaper(1)@ > > 127.0.0.1:52842 at 2015-04-16 19:21:33.747597056+00:00 > > @ 0x110e92ca5 google::LogMessageFatal::~LogMessageFatal() > > @ 0x10f3d33d3 _CheckFatal::~_CheckFatal() > > @ 0x10f3d3025 _CheckFatal::~_CheckFatal() > > @ 0x10fd94da6 mesos::internal::slave::Slave::__recover() > > @ 0x10fe7f09d > > > _ZZN7process8dispatchIN5mesos8internal5slave5SlaveERKNS_6FutureI7NothingEES7_EEvRKNS_3PIDIT_EEMSB_FvT0_ET1_ENKUlPNS_11ProcessBaseEE_clESK_ > > @ 0x10fe7ee7f > > > _ZNSt3__110__function6__funcIZN7process8dispatchIN5mesos8internal5slave5SlaveERKNS2_6FutureI7NothingEESA_EEvRKNS2_3PIDIT_EEMSE_FvT0_ET1_EUlPNS2_11ProcessBaseEE_NS_9allocatorISO_EEFvSN_EEclEOSN_ > > @ 0x110d74e7b std::__1::function<>::operator()() > > @ 0x110d5c5bf process::ProcessBase::visit() > > @ 0x110de6c0e process::DispatchEvent::visit() > > @ 0x10f3d0841 process::ProcessBase::serve() > > @ 0x110d45abe process::ProcessManager::resume() > > @ 0x110d451de process::schedule() > > @ 0x7fff8f1eb268 _pthread_body > > @ 0x7fff8f1eb1e5 _pthread_start > > @ 0x7fff8f1e941d thread_start > > I0416 12:21:01.848065 307879936 process.cpp:2091] Resuming reaper(1)@ > > 127.0.0.1:52842 at 2015-04-16 19:21:33.848114944+00:00 > > I0416 12:21:01.948398 306806784 process.cpp:2091] Resuming reaper(1)@ > > 127.0.0.1:52842 at 2015-04-16 19:21:33.948432896+00:00 > > I0416 12:21:02.048640 310562816 process.cpp:2091] Resuming reaper(1)@ > > 127.0.0.1:52842 at 2015-04-16 19:21:34.048694016+00:00 > > I0416 12:21:02.148989 308953088 process.cpp:2091] Resuming reaper(1)@ > > 127.0.0.1:52842 at 2015-04-16 19:21:34.149024000+00:00 > > I0416 12:21:02.252266 310026240 process.cpp:2091] Resuming reaper(1)@ > > 127.0.0.1:52842 at 2015-04-16 19:21:34.252296960+00:00 > > I0416 12:21:02.352577 310562816 process.cpp:2091] Resuming reaper(1)@ > > 127.0.0.1:52842 at 2015-04-16 19:21:34.352615168+00:00 > > I0416 12:21:02.452965 308416512 process.cpp:2091] Resuming reaper(1)@ > > 127.0.0.1:52842 at 2015-04-16 19:21:34.452996864+00:00 > > I0416 12:21:02.553390 308953088 process.cpp:2091] Resuming reaper(1)@ > > 127.0.0.1:52842 at 2015-04-16 19:21:34.553423872+00:00 > > I0416 12:21:02.653735 306806784 process.cpp:2091] Resuming reaper(1)@ > > 127.0.0.1:52842 at 2015-04-16 19:21:34.653764864+00:00 > > I0416 12:21:02.756922 310026240 process.cpp:2091] Resuming reaper(1)@ > > 127.0.0.1:52842 at 2015-04-16 19:21:34.756952064+00:00 > > I0416 12:21:02.857390 309489664 process.cpp:2091] Resuming reaper(1)@ > > 127.0.0.1:52842 at 2015-04-16 19:21:34.857434880+00:00 > > I0416 12:21:02.961252 307343360 process.cpp:2091] Resuming reaper(1)@ > > 127.0.0.1:52842 at 2015-04-16 19:21:34.961291008+00:00 > > I0416 12:21:03.061522 306806784 process.cpp:2091] Resuming reaper(1)@ > > 127.0.0.1:52842 at 2015-04-16 19:21:35.061554944+00:00 > > I0416 12:21:03.166790 310562816 process.cpp:2091] Resuming reaper(1)@ > > 127.0.0.1:52842 at 2015-04-16 19:21:35.166821120+00:00 > > I0416 12:21:03.267645 307879936 process.cpp:2091] Resuming reaper(1)@ > > 127.0.0.1:52842 at 2015-04-16 19:21:35.267698176+00:00 > > I0416 12:21:03.367985 310026240 process.cpp:2091] Resuming reaper(1)@ > > 127.0.0.1:52842 at 2015-04-16 19:21:35.368038912+00:00 > > I0416 12:21:03.468283 309489664 process.cpp:2091] Resuming reaper(1)@ > > 127.0.0.1:52842 at 2015-04-16 19:21:35.468316928+00:00 > > I0416 12:21:03.568531 306806784 process.cpp:2091] Resuming reaper(1)@ > > 127.0.0.1:52842 at 2015-04-16 19:21:35.568573952+00:00 > > I0416 12:21:03.668767 307879936 process.cpp:2091] Resuming reaper(1)@ > > 127.0.0.1:52842 at 2015-04-16 19:21:35.668807936+00:00 > > I0416 12:21:03.769104 308953088 process.cpp:2091] Resuming reaper(1)@ > > 127.0.0.1:52842 at 2015-04-16 19:21:35.769152000+00:00 > > I0416 12:21:03.869385 308416512 process.cpp:2091] Resuming reaper(1)@ > > 127.0.0.1:52842 at 2015-04-16 19:21:35.869432832+00:00 > > I0416 12:21:03.969768 307343360 process.cpp:2091] Resuming reaper(1)@ > > 127.0.0.1:52842 at 2015-04-16 19:21:35.969798144+00:00 > > I0416 12:21:04.074630 307879936 process.cpp:2091] Resuming reaper(1)@ > > 127.0.0.1:52842 at 2015-04-16 19:21:36.074662912+00:00 > > I0416 12:21:04.174993 310562816 process.cpp:2091] Resuming reaper(1)@ > > 127.0.0.1:52842 at 2015-04-16 19:21:36.175024128+00:00 > > I0416 12:21:04.275383 309489664 process.cpp:2091] Resuming reaper(1)@ > > 127.0.0.1:52842 at 2015-04-16 19:21:36.275419904+00:00 > > I0416 12:21:04.375689 310026240 process.cpp:2091] Resuming reaper(1)@ > > 127.0.0.1:52842 at 2015-04-16 19:21:36.375726848+00:00 > > I0416 12:21:04.476035 306806784 process.cpp:2091] Resuming reaper(1)@ > > 127.0.0.1:52842 at 2015-04-16 19:21:36.476072960+00:00 > > I0416 12:21:04.576369 308953088 process.cpp:2091] Resuming reaper(1)@ > > 127.0.0.1:52842 at 2015-04-16 19:21:36.576424960+00:00 > > I0416 12:21:04.676823 307879936 process.cpp:2091] Resuming reaper(1)@ > > 127.0.0.1:52842 at 2015-04-16 19:21:36.676861952+00:00 > > I0416 12:21:04.777240 308416512 process.cpp:2091] Resuming reaper(1)@ > > 127.0.0.1:52842 at 2015-04-16 19:21:36.777278976+00:00 > > I0416 12:21:04.877797 310026240 process.cpp:2091] Resuming reaper(1)@ > > 127.0.0.1:52842 at 2015-04-16 19:21:36.877843968+00:00 > > I0416 12:21:04.978090 307343360 process.cpp:2091] Resuming reaper(1)@ > > 127.0.0.1:52842 at 2015-04-16 19:21:36.978121984+00:00 > > I0416 12:21:05.079474 309489664 process.cpp:2091] Resuming reaper(1)@ > > 127.0.0.1:52842 at 2015-04-16 19:21:37.079504896+00:00 > > I0416 12:21:05.181157 308416512 process.cpp:2091] Resuming reaper(1)@ > > 127.0.0.1:52842 at 2015-04-16 19:21:37.181199104+00:00 > > I0416 12:21:05.281492 308953088 process.cpp:2091] Resuming reaper(1)@ > > 127.0.0.1:52842 at 2015-04-16 19:21:37.281528064+00:00 > > I0416 12:21:05.381813 310562816 process.cpp:2091] Resuming reaper(1)@ > > 127.0.0.1:52842 at 2015-04-16 19:21:37.381846016+00:00 > > I0416 12:21:05.482925 307879936 process.cpp:2091] Resuming reaper(1)@ > > 127.0.0.1:52842 at 2015-04-16 19:21:37.482966784+00:00 > > I0416 12:21:05.583298 308953088 process.cpp:2091] Resuming reaper(1)@ > > 127.0.0.1:52842 at 2015-04-16 19:21:37.583326208+00:00 > > I0416 12:21:05.683570 308416512 process.cpp:2091] Resuming reaper(1)@ > > 127.0.0.1:52842 at 2015-04-16 19:21:37.683599872+00:00 > > I0416 12:21:05.785339 307879936 process.cpp:2091] Resuming reaper(1)@ > > 127.0.0.1:52842 at 2015-04-16 19:21:37.785371136+00:00 > > I0416 12:21:05.885790 310562816 process.cpp:2091] Resuming reaper(1)@ > > 127.0.0.1:52842 at 2015-04-16 19:21:37.885846016+00:00 > > I0416 12:21:05.986150 310026240 process.cpp:2091] Resuming reaper(1)@ > > 127.0.0.1:52842 at 2015-04-16 19:21:37.986181120+00:00 > > I0416 12:21:06.086530 307879936 process.cpp:2091] Resuming reaper(1)@ > > 127.0.0.1:52842 at 2015-04-16 19:21:38.086565120+00:00 > > I0416 12:21:06.187046 308953088 process.cpp:2091] Resuming reaper(1)@ > > 127.0.0.1:52842 at 2015-04-16 19:21:38.187074816+00:00 > > I0416 12:21:06.287353 309489664 process.cpp:2091] Resuming reaper(1)@ > > 127.0.0.1:52842 at 2015-04-16 19:21:38.287381760+00:00 > > I0416 12:21:06.390130 310026240 process.cpp:2091] Resuming reaper(1)@ > > 127.0.0.1:52842 at 2015-04-16 19:21:38.390161920+00:00 > > I0416 12:21:06.490376 306806784 process.cpp:2091] Resuming reaper(1)@ > > 127.0.0.1:52842 at 2015-04-16 19:21:38.490406912+00:00 > > I0416 12:21:06.564841 143380480 process.cpp:2091] Resuming (1)@ > > 127.0.0.1:52923 at 2015-04-16 19:21:06.564875008+00:00 > > I0416 12:21:06.564909 143380480 exec.cpp:86] Committing suicide by > killing > > the process group > > I0416 12:21:06.566997 344678400 process.cpp:2091] Resuming (1)@ > > 127.0.0.1:52922 at 2015-04-16 19:21:06.567048192+00:00 > > I0416 12:21:06.567098 344678400 exec.cpp:86] Committing suicide by > killing > > the process group > > I0416 12:21:06.568764 136261632 process.cpp:2091] Resuming (1)@ > > 127.0.0.1:52921 at 2015-04-16 19:21:06.568818176+00:00 > > I0416 12:21:06.568856 136261632 exec.cpp:86] Committing suicide by > killing > > the process group > > I0416 12:21:06.590653 307343360 process.cpp:2091] Resuming reaper(1)@ > > 127.0.0.1:52842 at 2015-04-16 19:21:38.590689792+00:00 > > I0416 12:21:06.694011 308416512 process.cpp:2091] Resuming reaper(1)@ > > 127.0.0.1:52842 at 2015-04-16 19:21:38.694044160+00:00 > > I0416 12:21:06.794332 307879936 process.cpp:2091] Resuming reaper(1)@ > > 127.0.0.1:52842 at 2015-04-16 19:21:38.794361088+00:00 > > I0416 12:21:06.894798 310026240 process.cpp:2091] Resuming reaper(1)@ > > 127.0.0.1:52842 at 2015-04-16 19:21:38.894850048+00:00 > > I0416 12:21:06.995139 307343360 process.cpp:2091] Resuming reaper(1)@ > > 127.0.0.1:52842 at 2015-04-16 19:21:38.995166976+00:00 > > I0416 12:21:07.095384 308416512 process.cpp:2091] Resuming reaper(1)@ > > 127.0.0.1:52842 at 2015-04-16 19:21:39.095429888+00:00 > > I0416 12:21:07.195736 308953088 process.cpp:2091] Resuming reaper(1)@ > > 127.0.0.1:52842 at 2015-04-16 19:21:39.195770112+00:00 > > I0416 12:21:07.296052 307879936 process.cpp:2091] Resuming reaper(1)@ > > 127.0.0.1:52842 at 2015-04-16 19:21:39.296078848+00:00 > > I0416 12:21:07.396627 310026240 process.cpp:2091] Resuming reaper(1)@ > > 127.0.0.1:52842 at 2015-04-16 19:21:39.396659968+00:00 > > I0416 12:21:07.497102 308416512 process.cpp:2091] Resuming reaper(1)@ > > 127.0.0.1:52842 at 2015-04-16 19:21:39.497129984+00:00 > > I0416 12:21:07.597435 308953088 process.cpp:2091] Resuming reaper(1)@ > > 127.0.0.1:52842 at 2015-04-16 19:21:39.597464064+00:00 > > I0416 12:21:07.697954 307879936 process.cpp:2091] Resuming reaper(1)@ > > 127.0.0.1:52842 at 2015-04-16 19:21:39.697988864+00:00 > > I0416 12:21:07.798230 307343360 process.cpp:2091] Resuming reaper(1)@ > > 127.0.0.1:52842 at 2015-04-16 19:21:39.798292992+00:00 > > I0416 12:21:07.898574 309489664 process.cpp:2091] Resuming reaper(1)@ > > 127.0.0.1:52842 at 2015-04-16 19:21:39.898610176+00:00 > > I0416 12:21:08.003020 310026240 process.cpp:2091] Resuming reaper(1)@ > > 127.0.0.1:52842 at 2015-04-16 19:21:40.003076096+00:00 > > I0416 12:21:08.103488 308416512 process.cpp:2091] Resuming reaper(1)@ > > 127.0.0.1:52842 at 2015-04-16 19:21:40.103524864+00:00 > > I0416 12:21:08.203766 307879936 process.cpp:2091] Resuming reaper(1)@ > > 127.0.0.1:52842 at 2015-04-16 19:21:40.203793152+00:00 > > I0416 12:21:08.304029 308953088 process.cpp:2091] Resuming reaper(1)@ > > 127.0.0.1:52842 at 2015-04-16 19:21:40.304101888+00:00 > > I0416 12:21:08.355402 311099392 process.cpp:2048] Dropping event for > > process slave(1)@127.0.0.1:52842 > > I0416 12:21:08.404270 309489664 process.cpp:2091] Resuming reaper(1)@ > > 127.0.0.1:52842 at 2015-04-16 19:21:40.404307968+00:00 > > I0416 12:21:08.508119 310026240 process.cpp:2091] Resuming reaper(1)@ > > 127.0.0.1:52842 at 2015-04-16 19:21:40.508163072+00:00 > > I0416 12:21:08.520028 311099392 process.cpp:2048] Dropping event for > > process slave(2)@127.0.0.1:52842 > > I0416 12:21:08.608477 307343360 process.cpp:2091] Resuming reaper(1)@ > > 127.0.0.1:52842 at 2015-04-16 19:21:40.608509952+00:00 > > I0416 12:21:08.713762 308416512 process.cpp:2091] Resuming reaper(1)@ > > 127.0.0.1:52842 at 2015-04-16 19:21:40.713812224+00:00 > > I0416 12:21:08.814121 307879936 process.cpp:2091] Resuming reaper(1)@ > > 127.0.0.1:52842 at 2015-04-16 19:21:40.814168064+00:00 > > I0416 12:21:08.841398 311099392 process.cpp:2048] Dropping event for > > process slave(3)@127.0.0.1:52842 > > I0416 12:21:08.914386 310562816 process.cpp:2091] Resuming reaper(1)@ > > 127.0.0.1:52842 at 2015-04-16 19:21:40.914430976+00:00 > > I0416 12:21:09.019639 309489664 process.cpp:2091] Resuming reaper(1)@ > > 127.0.0.1:52842 at 2015-04-16 19:21:41.019673088+00:00 > > I0416 12:21:09.119954 308953088 process.cpp:2091] Resuming reaper(1)@ > > 127.0.0.1:52842 at 2015-04-16 19:21:41.120000000+00:00 > > I0416 12:21:09.224282 306806784 process.cpp:2091] Resuming reaper(1)@ > > 127.0.0.1:52842 at 2015-04-16 19:21:41.224335872+00:00 > > I0416 12:21:09.326063 310026240 process.cpp:2091] Resuming reaper(1)@ > > 127.0.0.1:52842 at 2015-04-16 19:21:41.326106112+00:00 > > ../../src/tests/script.cpp:83: Failure > > Failed > > low_level_scheduler_pthread_test.sh terminated with signal Abort trap: 6 > > [ FAILED ] ExamplesTest.LowLevelSchedulerPthread (7774 ms) > > {code} > > > > > Segmentation Fault in ExamplesTest.LowLevelSchedulerPthread > > > ----------------------------------------------------------- > > > > > > Key: MESOS-2144 > > > URL: https://issues.apache.org/jira/browse/MESOS-2144 > > > Project: Mesos > > > Issue Type: Bug > > > Components: test > > > Affects Versions: 0.21.0 > > > Reporter: Cody Maloney > > > Assignee: Yan Xu > > > Priority: Minor > > > Labels: flaky, twitter > > > > > > Occured on review bot review of: > > https://reviews.apache.org/r/28262/#review62333 > > > The review doesn't touch code related to the test (And doesn't break > > libprocess in general) > > > [ RUN ] ExamplesTest.LowLevelSchedulerPthread > > > ../../src/tests/script.cpp:83: Failure > > > Failed > > > low_level_scheduler_pthread_test.sh terminated with signal Segmentation > > fault > > > [ FAILED ] ExamplesTest.LowLevelSchedulerPthread (7561 ms) > > > The test > > > > > > > > -- > > This message was sent by Atlassian JIRA > > (v6.3.4#6332) > > >
