----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/9711/ -----------------------------------------------------------
(Updated March 4, 2013, 8:22 p.m.) Review request for mesos, Benjamin Hindman and Vinod Kone. Changes ------- Updated to trunk. Description ------- This only takes care of the noisy allocator timing logs, by moving them to VLOG(1). I'd also like to try to take care of the log noise from heavily scheduled machines with port fragmentation, check out the output below: I0301 20:50:20.594537 22294 master.hpp:296] Removing task with resources cpus=0.01; disk=1; mem=1 on slave 201302272335-1937777162-5050-7468-64 I0301 20:50:20.595034 22294 hierarchical_allocator_process.hpp:542] Recovered cpus=0.01; disk=1; mem=1 (total allocatable: cpus=12.31; mem=18442; ports=[31001-31004, 31006-31006, 31008-31011, 31013-31013, 31015-31022, 31025-31027, 31030-31037, 31039-31057, 31059-31059, 31061-31061, 31063-31064, 31067-31081, 31083-31106, 31108-31109, 31111-31114, 31116-31126, 31128-31136, 31139-31147, 31149-31150, 31152-31158, 31160-31161, 31163-31166, 31168-31169, 31171-31176, 31179-31184, 31189-31190, 31192-31194, 31196-31196, 31198-31200, 31202-31202, 31204-31208, 31210-31223, 31225-31229, 31231-31232, 31234-31235, 31237-31239, 31241-31244, 31246-31253, 31255-31265, 31267-31268, 31270-31270, 31273-31284, 31286-31290, 31292-31294, 31297-31300, 31302-31304, 31306-31310, 31312-31315, 31317-31324, 31326-31328, 31331-31339, 31343-31343, 31345-31348, 31350-31357, 31359-31359, 31361-31365, 31367-31375, 31377-31381, 31383-31396, 31398-31401, 31403-31403, 31405-31405, 31408-31422, 31424-31434, 31437-31451, 31453-31458, 31460-31464, 31467-31467, 31469-31473, 31476-31486, 31489-31493, 31496-31496, 31498-31498, 31500-31504, 31506-31507, 31509-31510, 31512-31515, 31517-31518, 31520-31528, 31530-31536, 31539-31545, 31548-31550, 31552-31552, 31554-31557, 31559-31560, 31562-31562, 31565-31569, 31571-31577, 31579-31581, 31583-31586, 31590-31590, 31592-31600, 31602-31607, 31609-31612, 31614-31614, 31616-31616, 31618-31621, 31623-31625, 31627-31628, 31630-31630, 31632-31632, 31634-31642, 31644-31666, 31668-31672, 31674-31684, 31686-31686, 31688-31693, 31695-31695, 31697-31702, 31704-31709, 31711-31711, 31713-31717, 31719-31725, 31727-31741, 31743-31743, 31745-31746, 31748-31750, 31752-31759, 31762-31771, 31773-31779, 31781-31781, 31787-31795, 31797-31797, 31800-31806, 31808-31812, 31814-31815, 31817-31823, 31825-31832, 31834-31839, 31841-31842, 31845-31851, 31853-31857, 31861-31864, 31867-31867, 31870-31879, 31881-31881, 31883-31885, 31887-31887, 31889-31889, 31891-31892, 31894-31896, 31898-31898, 31900-31901, 31904-31909, 31911-31911, 31913-31914, 31916-31916, 31918-31927, 31929-31933, 31935-31937, 31939-31943, 31945-31949, 31951-31951, 31953-31957, 31960-31961, 31963-31965, 31967-31969, 31971-31981, 31984-31985, 31988-31993, 31995-32000]; disk=387185) on slave 201302272335-1937777162-5050-7468-64 from framework 201103282247-0000000019-0000 I0301 20:50:21.489853 22289 hierarchical_allocator_process.hpp:597] Performed allocation for 9477 slaves in 89.62ms I0301 20:50:21.490259 22300 master.hpp:305] Adding offer with resources cpus=12.31; mem=18442; ports=[31001-31004, 31006-31006, 31008-31011, 31013-31013, 31015-31022, 31025-31027, 31030-31037, 31039-31057, 31059-31059, 31061-31061, 31063-31064, 31067-31081, 31083-31106, 31108-31109, 31111-31114, 31116-31126, 31128-31136, 31139-31147, 31149-31150, 31152-31158, 31160-31161, 31163-31166, 31168-31169, 31171-31176, 31179-31184, 31189-31190, 31192-31194, 31196-31196, 31198-31200, 31202-31202, 31204-31208, 31210-31223, 31225-31229, 31231-31232, 31234-31235, 31237-31239, 31241-31244, 31246-31253, 31255-31265, 31267-31268, 31270-31270, 31273-31284, 31286-31290, 31292-31294, 31297-31300, 31302-31304, 31306-31310, 31312-31315, 31317-31324, 31326-31328, 31331-31339, 31343-31343, 31345-31348, 31350-31357, 31359-31359, 31361-31365, 31367-31375, 31377-31381, 31383-31396, 31398-31401, 31403-31403, 31405-31405, 31408-31422, 31424-31434, 31437-31451, 31453-31458, 31460-31464, 31467-31467, 31469-31473, 31476-31486, 31489-31493, 31496-31496, 31498-31498, 31500-31504, 31506-31507, 31509-31510, 31512-31515, 31517-31518, 31520-31528, 31530-31536, 31539-31545, 31548-31550, 31552-31552, 31554-31557, 31559-31560, 31562-31562, 31565-31569, 31571-31577, 31579-31581, 31583-31586, 31590-31590, 31592-31600, 31602-31607, 31609-31612, 31614-31614, 31616-31616, 31618-31621, 31623-31625, 31627-31628, 31630-31630, 31632-31632, 31634-31642, 31644-31666, 31668-31672, 31674-31684, 31686-31686, 31688-31693, 31695-31695, 31697-31702, 31704-31709, 31711-31711, 31713-31717, 31719-31725, 31727-31741, 31743-31743, 31745-31746, 31748-31750, 31752-31759, 31762-31771, 31773-31779, 31781-31781, 31787-31795, 31797-31797, 31800-31806, 31808-31812, 31814-31815, 31817-31823, 31825-31832, 31834-31839, 31841-31842, 31845-31851, 31853-31857, 31861-31864, 31867-31867, 31870-31879, 31881-31881, 31883-31885, 31887-31887, 31889-31889, 31891-31892, 31894-31896, 31898-31898, 31900-31901, 31904-31909, 31911-31911, 31913-31914, 31916-31916, 31918-31927, 31929-31933, 31935-31937, 31939-31943, 31945-31949, 31951-31951, 31953-31957, 31960-31961, 31963-31965, 31967-31969, 31971-31981, 31984-31985, 31988-31993, 31995-32000]; disk=387185 on slave 201302272335-1937777162-5050-7468-64 I0301 20:50:21.492420 22300 master.cpp:1262] Sending 1 offers to framework 201103282247-0000000019-0000 W0301 20:50:21.578040 22294 master.cpp:79] No whitelist given. Advertising offers for all slaves I0301 20:50:22.578544 22297 master.hpp:305] Adding offer with resources cpus=11.75; mem=18073; ports=[31001-31001, 31003-31010, 31015-31034, 31036-31040, 31042-31045, 31047-31050, 31054-31058, 31060-31060, 31062-31081, 31083-31086, 31088-31091, 31093-31093, 31095-31096, 31098-31101, 31103-31113, 31115-31115, 31117-31121, 31125-31125, 31127-31131, 31133-31134, 31136-31137, 31139-31154, 31156-31156, 31159-31161, 31164-31167, 31169-31171, 31176-31178, 31180-31180, 31182-31193, 31195-31197, 31200-31202, 31204-31208, 31211-31215, 31218-31221, 31223-31224, 31227-31227, 31229-31229, 31231-31236, 31238-31240, 31243-31257, 31261-31270, 31272-31282, 31284-31289, 31291-31298, 31300-31301, 31303-31306, 31312-31316, 31318-31318, 31320-31322, 31324-31324, 31326-31330, 31332-31335, 31338-31339, 31341-31346, 31348-31349, 31352-31353, 31355-31356, 31358-31362, 31365-31365, 31367-31377, 31379-31387, 31389-31394, 31396-31398, 31400-31409, 31411-31411, 31413-31413, 31415-31416, 31418-31428, 31430-31431, 31434-31446, 31448-31452, 31456-31461, 31463-31464, 31466-31469, 31471-31471, 31474-31474, 31476-31478, 31480-31484, 31486-31486, 31488-31490, 31492-31493, 31495-31498, 31500-31508, 31510-31520, 31522-31523, 31525-31525, 31528-31529, 31531-31539, 31541-31543, 31546-31558, 31560-31560, 31563-31564, 31566-31571, 31573-31575, 31577-31582, 31584-31585, 31588-31592, 31595-31607, 31609-31617, 31619-31628, 31631-31642, 31644-31645, 31648-31649, 31652-31652, 31654-31676, 31678-31708, 31710-31717, 31719-31724, 31727-31727, 31729-31742, 31744-31748, 31750-31752, 31754-31754, 31756-31757, 31759-31765, 31767-31770, 31772-31781, 31783-31789, 31791-31805, 31807-31811, 31813-31814, 31817-31820, 31823-31830, 31832-31833, 31835-31836, 31838-31839, 31841-31846, 31848-31848, 31850-31856, 31858-31861, 31863-31863, 31866-31870, 31872-31873, 31878-31879, 31882-31883, 31886-31891, 31893-31894, 31897-31897, 31899-31911, 31913-31915, 31917-31927, 31929-31936, 31938-31939, 31941-31942, 31944-31947, 31949-31958, 31960-31967, 31969-31978, 31980-31981, 31983-31989, 31991-31995, 31998-32000]; disk=386176 on slave 201302272335-1937777162-5050-7468-61 I0301 20:50:22.579871 22304 hierarchical_allocator_process.hpp:597] Performed allocation for 9477 slaves in 89.37ms I0301 20:50:22.580482 22297 master.cpp:1262] Sending 1 offers to framework 201103282247-0000000019-0000 W0301 20:50:23.305022 22291 master.cpp:791] Offer 201302280043-2206671370-5050-22288-65505 is no longer valid My first attempt was to examine the difference printing singular port ranges would make, but it's not significant enough: Before: 31000-31000, 31003-31006, 31008-31009, 31015-31016, 31018-31018, 31026-31026, 31035-31036, 31038-31038, 31045-31045, 31048-31048, 31051-31051, 31053-31054, 31058-31060, 31063-31067, 31069-31069, 31071-31071, 31073-31073, 31075-31075, 31077-31078, 31080-31082, 31086-31087, 31091-31091, 31095-31095, 31097-31099, 31102-31102, 31104-31104, 31109-31111, 31113-31115, 31119-31119, 31121-31122, 31125-31125, 31127-31130, 31133-31134, 31136-31136, 31141-31142, 31144-31145, 31147-31148, 31153-31154, 31156-31156, 31159-31159, 31164-31166, 31168-31168, 31171-31172, 31177-31180, 31184-31187, 31190-31190, 31192-31193, 31195-31200, 31202-31202, 31207-31207, 31209-31211, 31213-31214, 31216-31216, 31218-31219, 31221-31222, 31224-31225, 31228-31228, 31230-31230, 31234-31234, 31236-31237, 31239-31244, 31249-31250, 31252-31252, 31257-31259, 31262-31263, 31266-31267, 31271-31271, 31276-31276, 31283-31283, 31285-31285, 31287-31287, 31290-31297, 31299-31303, 31307-31307, 31313-31314, 31316-31316, 31318-31321, 31323-31323, 31325-31325, 31328-31328, 31332-31332, 31335-31338, 31340-31342, 31344-31344, 31346-31346, 31348-31349, 31351-31351, 31353-31354, 31356-31356, 31358-31358, 31363-31365, 31371-31371, 31373-31373, 31375-31377, 31380-31381, 31387-31387, 31392-31393, 31398-31399, 31402-31402, 31405-31407, 31409-31409, 31412-31412, 31415-31415, 31418-31418, 31420-31425, 31428-31430, 31434-31435, 31439-31441, 31443-31445, 31447-31447, 31449-31451, 31454-31454, 31459-31460, 31463-31463, 31471-31473, 31476-31477, 31479-31479, 31487-31489, 31491-31492, 31495-31497, 31500-31501, 31504-31504, 31506-31506, 31508-31508, 31510-31511, 31516-31516, 31520-31521, 31523-31531, 31533-31534, 31537-31544, 31546-31547, 31549-31549, 31553-31553, 31558-31558, 31560-31561, 31564-31565, 31569-31570, 31572-31576, 31578-31578, 31580-31580, 31582-31583, 31585-31588, 31590-31590, 31593-31593, 31595-31597, 31599-31600, 31602-31603, 31605-31605, 31608-31610, 31612-31613, 31616-31616, 31623-31624, 31627-31627, 31632-31632, 31634-31634, 31636-31636, 31640-31641, 31643-31643, 31649-31649, 31655-31655, 31657-31660, 31662-31662, 31664-31666, 31670-31673, 31677-31677, 31679-31680, 31684-31686, 31689-31690, 31693-31693, 31695-31697, 31699-31699, 31701-31704, 31707-31707, 31709-31709, 31711-31712, 31714-31714, 31716-31716, 31718-31718, 31723-31724, 31726-31728, 31730-31730, 31734-31734, 31736-31738, 31740-31740, 31742-31743, 31745-31745, 31747-31747, 31749-31750, 31752-31752, 31756-31757, 31759-31763, 31766-31771, 31773-31774, 31777-31778, 31786-31787, 31790-31791, 31794-31795, 31797-31797, 31799-31801, 31804-31805, 31807-31807, 31809-31809, 31811-31814, 31817-31817, 31820-31820, 31822-31822, 31827-31827, 31829-31831, 31833-31834, 31836-31836, 31838-31838, 31842-31843, 31846-31846, 31848-31849, 31853-31853, 31855-31855, 31859-31859, 31862-31864, 31866-31867, 31871-31872, 31876-31881, 31887-31887, 31890-31890, 31894-31895, 31898-31898, 31901-31904, 31906-31910, 31912-31913, 31917-31920, 31923-31924, 31926-31926, 31928-31929, 31937-31938, 31940-31940, 31942-31943, 31947-31950, 31953-31953, 31955-31956, 31958-31963, 31965-31968, 31973-31973, 31977-31978, 31980-31981, 31984-31986, 31988-31990, 31992-31992, 31994-31994, 31999-32000 After: 31000, 31003-31006, 31008-31009, 31015-31016, 31018, 31026, 31035-31036, 31038, 31045, 31048, 31051, 31053-31054, 31058-31060, 31063-31067, 31069, 31071, 31073, 31075, 31077-31078, 31080-31082, 31086-31087, 31091, 31095, 31097-31099, 31102, 31104, 31109-31111, 31113-31115, 31119, 31121-31122, 31125, 31127-31130, 31133-31134, 31136, 31141-31142, 31144-31145, 31147-31148, 31153-31154, 31156, 31159, 31164-31166, 31168, 31171-31172, 31177-31180, 31184-31187, 31190, 31192-31193, 31195-31200, 31202, 31207, 31209-31211, 31213-31214, 31216, 31218-31219, 31221-31222, 31224-31225, 31228, 31230, 31234, 31236-31237, 31239-31244, 31249-31250, 31252, 31257-31259, 31262-31263, 31266-31267, 31271, 31276, 31283, 31285, 31287, 31290-31297, 31299-31303, 31307, 31313-31314, 31316, 31318-31321, 31323, 31325, 31328, 31332, 31335-31338, 31340-31342, 31344, 31346, 31348-31349, 31351, 31353-31354, 31356, 31358, 31363-31365, 31371, 31373, 31375-31377, 31380-31381, 31387, 31392-31393, 31398-31399, 31402, 31405-31407, 31409, 31412, 31415, 31418, 31420-31425, 31428-31430, 31434-31435, 31439-31441, 31443-31445, 31447, 31449-31451, 31454, 31459-31460, 31463, 31471-31473, 31476-31477, 31479, 31487-31489, 31491-31492, 31495-31497, 31500-31501, 31504, 31506, 31508, 31510-31511, 31516, 31520-31521, 31523-31531, 31533-31534, 31537-31544, 31546-31547, 31549, 31553, 31558, 31560-31561, 31564-31565, 31569-31570, 31572-31576, 31578, 31580, 31582-31583, 31585-31588, 31590, 31593, 31595-31597, 31599-31600, 31602-31603, 31605, 31608-31610, 31612-31613, 31616, 31623-31624, 31627, 31632, 31634, 31636, 31640-31641, 31643, 31649, 31655, 31657-31660, 31662, 31664-31666, 31670-31673, 31677, 31679-31680, 31684-31686, 31689-31690, 31693, 31695-31697, 31699, 31701-31704, 31707, 31709, 31711-31712, 31714, 31716, 31718, 31723-31724, 31726-31728, 31730, 31734, 31736-31738, 31740, 31742-31743, 31745, 31747, 31749-31750, 31752, 31756-31757, 31759-31763, 31766-31771, 31773-31774, 31777-31778, 31786-31787, 31790-31791, 31794-31795, 31797, 31799-31801, 31804-31805, 31807, 31809, 31811-31814, 31817, 31820, 31822, 31827, 31829-31831, 31833-31834, 31836, 31838, 31842-31843, 31846, 31848-31849, 31853, 31855, 31859, 31862-31864, 31866-31867, 31871-31872, 31876-31881, 31887, 31890, 31894-31895, 31898, 31901-31904, 31906-31910, 31912-31913, 31917-31920, 31923-31924, 31926, 31928-31929, 31937-31938, 31940, 31942-31943, 31947-31950, 31953, 31955-31956, 31958-31963, 31965-31968, 31973, 31977-31978, 31980-31981, 31984-31986, 31988-31990, 31992, 31994, 31999-32000, Diffs (updated) ----- src/master/hierarchical_allocator_process.hpp 33e059c10bb87db88b1fb9a100b7504bc89ac896 Diff: https://reviews.apache.org/r/9711/diff/ Testing ------- Thanks, Ben Mahler
