[ https://issues.apache.org/jira/browse/MESOS-547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris Lambert updated MESOS-547: -------------------------------- Sprint: Q2'14 Sprint 3 > MasterTest.MasterLost is flaky on Jenkins > ----------------------------------------- > > Key: MESOS-547 > URL: https://issues.apache.org/jira/browse/MESOS-547 > Project: Mesos > Issue Type: Bug > Reporter: Benjamin Mahler > Labels: twitter > > This is because the MasterLost test expects the Scheduler.registered call to > happen a single time. However, on the Apache Jenkins machine, it runs slow > enough for the reliable registration in the scheduler driver to retry > registration (1 second timeout). > The fix here will affect a lot of tests, so I'd like to do the right thing, > possible fixes: > EXPECT_CALL(sched, registered(&driver, _, _)) > .Times(1); // Change this to .WillRepeatedly(Return()) > Also, pausing the clock prior to starting the driver will work. But may have > further impact on the tests. > [ RUN ] MasterTest.MasterLost > I0712 03:48:50.415976 4397 master.cpp:230] Master started on > 67.195.138.9:47248 > I0712 03:48:50.416594 4399 slave.cpp:113] Slave started on > 11)@67.195.138.9:47248 > I0712 03:48:50.417034 4395 sched.cpp:177] New master at > master@67.195.138.9:47248 > I0712 03:48:50.423070 4397 master.cpp:245] Master ID: > 201307120348-160088899-47248-4046 > I0712 03:48:50.431115 4399 slave.cpp:205] Slave resources: cpus=2; mem=1024; > ports=[31000-32000]; disk=1024 > I0712 03:48:50.508512 4398 hierarchical_allocator_process.hpp:305] > Initializing hierarchical allocator process with master : > master@67.195.138.9:47248 > W0712 03:48:50.508554 4394 master.cpp:83] No whitelist given. Advertising > offers for all slaves > I0712 03:48:50.508767 4397 master.cpp:583] Elected as master! > I0712 03:48:50.791844 4399 slave.cpp:529] New master detected at > master@67.195.138.9:47248 > I0712 03:48:51.564163 4399 slave.cpp:544] Postponing registration until > recovery is complete > I0712 03:48:51.564257 4396 status_update_manager.cpp:155] New master > detected at master@67.195.138.9:47248 > I0712 03:48:51.564288 4397 master.cpp:633] Registering framework > 201307120348-160088899-47248-4046-0000 at scheduler(11)@67.195.138.9:47248 > I0712 03:48:51.751411 4399 slave.cpp:390] Finished recovery > I0712 03:48:51.823364 4392 sched.cpp:222] Framework registered with > 201307120348-160088899-47248-4046-0000 > I0712 03:48:51.823395 4397 master.cpp:620] Framework > 201307120348-160088899-47248-4046-0000 (scheduler(11)@67.195.138.9:47248) > already registered, resending acknowledgement > I0712 03:48:51.823452 4395 hierarchical_allocator_process.hpp:335] Added > framework 201307120348-160088899-47248-4046-0000 > I0712 03:48:52.110093 4395 hierarchical_allocator_process.hpp:704] No > resources available to allocate! > I0712 03:48:51.946835 4392 sched.cpp:194] No master detected, waiting for > another master > I0712 03:48:52.110155 4397 master.cpp:962] Attempting to register slave on > vesta.apache.org at slave(11)@67.195.138.9:47248 > I0712 03:48:52.440776 4397 master.cpp:1929] Adding slave > 201307120348-160088899-47248-4046-0 at vesta.apache.org with cpus=2; > mem=1024; ports=[31000-32000]; disk=1024 > I0712 03:48:52.249390 4392 sched.cpp:222] Framework registered with > 201307120348-160088899-47248-4046-0000 > ../../src/tests/master_tests.cpp:906: Failure > Mock function called more times than expected - returning directly. > Function call: registered(0x7fffaf90ca00, @0x2b7bfc004140 > 201307120348-160088899-47248-4046-0000, @0x2b7bfc003170 40-byte object <70-8B > 2C-F5 7B-2B 00-00 00-00 00-00 00-00 00-00 F0-54 00-FC 7B-2B 00-00 43-C3 8A-09 > 90-B8 00-00 00-00 00-00 07-00 00-00>) > Expected: to be called once > Actual: called twice - over-saturated and active > I0712 03:48:52.241426 4395 hierarchical_allocator_process.hpp:666] Performed > allocation for 0 slaves in 131.314878ms > I0712 03:48:52.448761 4399 slave.cpp:589] Registered with master > master@67.195.138.9:47248; given slave ID 201307120348-160088899-47248-4046-0 > I0712 03:48:52.448806 4397 master.cpp:436] Master terminating > I0712 03:48:52.481109 4397 slave.cpp:485] Slave asked to shut down by > master@67.195.138.9:47248 > I0712 03:48:52.489028 4397 slave.cpp:440] Slave terminating > I0712 03:48:52.456776 4392 sched.cpp:427] Stopping framework > '201307120348-160088899-47248-4046-0000' > I0712 03:48:52.481195 4046 master.cpp:209] Shutting down master > I0712 03:48:52.464653 4395 hierarchical_allocator_process.hpp:448] Added > slave 201307120348-160088899-47248-4046-0 (vesta.apache.org) with cpus=2; > mem=1024; ports=[31000-32000]; disk=1024 (and cpus=2; mem=1024; > ports=[31000-32000]; disk=1024 available) > I0712 03:48:52.560148 4395 hierarchical_allocator_process.hpp:726] Offering > cpus=2; mem=1024; ports=[31000-32000]; disk=1024 on slave > 201307120348-160088899-47248-4046-0 to framework > 201307120348-160088899-47248-4046-0000 > I0712 03:48:52.568372 4395 hierarchical_allocator_process.hpp:686] Performed > allocation for slave 201307120348-160088899-47248-4046-0 in 8.255916ms > [ FAILED ] MasterTest.MasterLost (2161 ms) > [----------] 11 tests from MasterTest (15502 ms total) -- This message was sent by Atlassian JIRA (v6.2#6252)