[ https://issues.apache.org/jira/browse/MESOS-8134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16479276#comment-16479276 ]
Andrei Budnik commented on MESOS-8134: -------------------------------------- Steps to reproduce the race condition: 1. Wrap [`os::getenv()` code|https://github.com/apache/mesos/blob/40b40d9b73221388e583fc140280f1eb2b48b832/src/slave/slave.cpp#L9948-L9951] with a loop `for (int i=0; i<1000 * 1000; ++i) {` 2. Run {{./src/mesos-tests --gtest_filter=SlaveTest.ContainersEndpoint --gtest_break_on_failure --gtest_repeat=100 --verbose}} > SlaveTest.ContainersEndpoint is flaky due to getenv crash. > ---------------------------------------------------------- > > Key: MESOS-8134 > URL: https://issues.apache.org/jira/browse/MESOS-8134 > Project: Mesos > Issue Type: Bug > Components: test > Reporter: Benjamin Mahler > Priority: Major > Labels: flaky-test > > Looks like this test also has the getenv during setenv crash: > {noformat} > [ RUN ] SlaveTest.ContainersEndpoint > I1025 04:02:53.061488 6805 cluster.cpp:162] Creating default 'local' > authorizer > I1025 04:02:53.065587 6824 master.cpp:445] Master > 2dc7ad46-f111-4762-9bf6-ef428a6f6d53 (a4020869f68c) started on > 172.17.0.2:38626 > I1025 04:02:53.065665 6824 master.cpp:447] Flags at startup: --acls="" > --agent_ping_timeout="15secs" --agent_reregister_timeout="10mins" > --allocation_interval="1secs" --allocator="HierarchicalDRF" > --authenticate_agents="true" --authenticate_frameworks="true" > --authenticate_http_frameworks="true" --authenticate_http_readonly="true" > --authenticate_http_readwrite="true" --authenticators="crammd5" > --authorizers="local" --credentials="/tmp/lq9Ngb/credentials" > --filter_gpu_resources="true" --framework_sorter="drf" --help="false" > --hostname_lookup="true" --http_authenticators="basic" > --http_framework_authenticators="basic" --initialize_driver_logging="true" > --log_auto_initialize="true" --logbufsecs="0" --logging_level="INFO" > --max_agent_ping_timeouts="5" --max_completed_frameworks="50" > --max_completed_tasks_per_framework="1000" > --max_unreachable_tasks_per_framework="1000" --port="5050" --quiet="false" > --recovery_agent_removal_limit="100%" --registry="in_memory" > --registry_fetch_timeout="1mins" --registry_gc_interval="15mins" > --registry_max_agent_age="2weeks" --registry_max_agent_count="102400" > --registry_store_timeout="100secs" --registry_strict="false" > --root_submissions="true" --user_sorter="drf" --version="false" > --webui_dir="/mesos/mesos-1.5.0/_inst/share/mesos/webui" > --work_dir="/tmp/lq9Ngb/master" --zk_session_timeout="10secs" > I1025 04:02:53.066131 6824 master.cpp:496] Master only allowing > authenticated frameworks to register > I1025 04:02:53.066145 6824 master.cpp:502] Master only allowing > authenticated agents to register > I1025 04:02:53.066153 6824 master.cpp:508] Master only allowing > authenticated HTTP frameworks to register > I1025 04:02:53.066165 6824 credentials.hpp:37] Loading credentials for > authentication from '/tmp/lq9Ngb/credentials' > I1025 04:02:53.066561 6824 master.cpp:552] Using default 'crammd5' > authenticator > I1025 04:02:53.066746 6824 http.cpp:1045] Creating default 'basic' HTTP > authenticator for realm 'mesos-master-readonly' > I1025 04:02:53.066949 6824 http.cpp:1045] Creating default 'basic' HTTP > authenticator for realm 'mesos-master-readwrite' > I1025 04:02:53.067095 6824 http.cpp:1045] Creating default 'basic' HTTP > authenticator for realm 'mesos-master-scheduler' > I1025 04:02:53.067230 6824 master.cpp:631] Authorization enabled > I1025 04:02:53.067430 6818 hierarchical.cpp:171] Initialized hierarchical > allocator process > I1025 04:02:53.067477 6807 whitelist_watcher.cpp:77] No whitelist given > I1025 04:02:53.070369 6825 master.cpp:2198] Elected as the leading master! > I1025 04:02:53.070421 6825 master.cpp:1687] Recovering from registrar > I1025 04:02:53.070796 6816 registrar.cpp:347] Recovering registrar > I1025 04:02:53.071532 6816 registrar.cpp:391] Successfully fetched the > registry (0B) in 690944ns > I1025 04:02:53.071671 6816 registrar.cpp:495] Applied 1 operations in > 54316ns; attempting to update the registry > I1025 04:02:53.072278 6816 registrar.cpp:552] Successfully updated the > registry in 538880ns > I1025 04:02:53.072394 6816 registrar.cpp:424] Successfully recovered > registrar > I1025 04:02:53.072808 6823 master.cpp:1791] Recovered 0 agents from the > registry (129B); allowing 10mins for agents to re-register > I1025 04:02:53.072983 6828 hierarchical.cpp:209] Skipping recovery of > hierarchical allocator: nothing to recover > W1025 04:02:53.077972 6805 process.cpp:3193] Attempted to spawn already > running process files@172.17.0.2:38626 > I1025 04:02:53.078305 6805 cluster.cpp:448] Creating default 'local' > authorizer > I1025 04:02:53.080204 6811 slave.cpp:254] Mesos agent started on > (539)@172.17.0.2:38626 > I1025 04:02:53.080246 6811 slave.cpp:255] Flags at startup: --acls="" > --appc_simple_discovery_uri_prefix="http://" > --appc_store_dir="/tmp/SlaveTest_ContainersEndpoint_kzLnW6/store/appc" > --authenticate_http_executors="true" --authenticate_http_readonly="true" > --authenticate_http_readwrite="true" --authenticatee="crammd5" > --authentication_backoff_factor="1secs" --authorizer="local" > --cgroups_cpu_enable_pids_and_tids_count="false" --cgroups_enable_cfs="false" > --cgroups_hierarchy="/sys/fs/cgroup" --cgroups_limit_swap="false" > --cgroups_root="mesos" --container_disk_watch_interval="15secs" > --containerizers="mesos" > --credential="/tmp/SlaveTest_ContainersEndpoint_kzLnW6/credential" > --default_role="*" --disallow_sharing_agent_pid_namespace="false" > --disk_watch_interval="1mins" --docker="docker" --docker_kill_orphans="true" > --docker_registry="https://registry-1.docker.io" --docker_remove_delay="6hrs" > --docker_socket="/var/run/docker.sock" --docker_stop_timeout="0ns" > --docker_store_dir="/tmp/SlaveTest_ContainersEndpoint_kzLnW6/store/docker" > --docker_volume_checkpoint_dir="/var/run/mesos/isolators/docker/volume" > --enforce_container_disk_quota="false" > --executor_registration_timeout="1mins" > --executor_reregistration_timeout="2secs" > --executor_secret_key="/tmp/SlaveTest_ContainersEndpoint_kzLnW6/executor_secret_key" > --executor_shutdown_grace_period="5secs" > --fetcher_cache_dir="/tmp/SlaveTest_ContainersEndpoint_kzLnW6/fetch" > --fetcher_cache_size="2GB" --frameworks_home="" --gc_delay="1weeks" > --gc_disk_headroom="0.1" --hadoop_home="" --help="false" > --hostname_lookup="true" --http_command_executor="false" > --http_credentials="/tmp/SlaveTest_ContainersEndpoint_kzLnW6/http_credentials" > --http_heartbeat_interval="30secs" --initialize_driver_logging="true" > --isolation="posix/cpu,posix/mem" --launcher="posix" > --launcher_dir="/mesos/mesos-1.5.0/_build/src" --logbufsecs="0" > --logging_level="INFO" --max_completed_executors_per_framework="150" > --oversubscribed_resources_interval="15secs" --perf_duration="10secs" > --perf_interval="1mins" --port="5051" --qos_correction_interval_min="0ns" > --quiet="false" --recover="reconnect" --recovery_timeout="15mins" > --registration_backoff_factor="10ms" > --resources="cpus:2;gpus:0;mem:1024;disk:1024;ports:[31000-32000]" > --revocable_cpu_low_priority="true" > --runtime_dir="/tmp/SlaveTest_ContainersEndpoint_kzLnW6" > --sandbox_directory="/mnt/mesos/sandbox" --strict="true" --switch_user="true" > --systemd_enable_support="true" > --systemd_runtime_directory="/run/systemd/system" --version="false" > --work_dir="/tmp/SlaveTest_ContainersEndpoint_RzfA6D" > --zk_session_timeout="10secs" > I1025 04:02:53.080750 6811 credentials.hpp:86] Loading credential for > authentication from '/tmp/SlaveTest_ContainersEndpoint_kzLnW6/credential' > W1025 04:02:53.080813 6805 process.cpp:3193] Attempted to spawn already > running process version@172.17.0.2:38626 > I1025 04:02:53.080951 6811 slave.cpp:287] Agent using credential for: > test-principal > I1025 04:02:53.080979 6811 credentials.hpp:37] Loading credentials for > authentication from > '/tmp/SlaveTest_ContainersEndpoint_kzLnW6/http_credentials' > I1025 04:02:53.081332 6811 http.cpp:1045] Creating default 'basic' HTTP > authenticator for realm 'mesos-agent-executor' > I1025 04:02:53.081487 6811 http.cpp:1066] Creating default 'jwt' HTTP > authenticator for realm 'mesos-agent-executor' > I1025 04:02:53.081723 6811 http.cpp:1045] Creating default 'basic' HTTP > authenticator for realm 'mesos-agent-readonly' > I1025 04:02:53.081825 6811 http.cpp:1066] Creating default 'jwt' HTTP > authenticator for realm 'mesos-agent-readonly' > I1025 04:02:53.081881 6805 sched.cpp:232] Version: 1.5.0 > I1025 04:02:53.082005 6811 http.cpp:1045] Creating default 'basic' HTTP > authenticator for realm 'mesos-agent-readwrite' > I1025 04:02:53.082109 6811 http.cpp:1066] Creating default 'jwt' HTTP > authenticator for realm 'mesos-agent-readwrite' > I1025 04:02:53.082706 6810 sched.cpp:336] New master detected at > master@172.17.0.2:38626 > I1025 04:02:53.082825 6810 sched.cpp:396] Authenticating with master > master@172.17.0.2:38626 > I1025 04:02:53.082901 6810 sched.cpp:403] Using default CRAM-MD5 > authenticatee > I1025 04:02:53.083366 6822 authenticatee.cpp:121] Creating new client SASL > connection > I1025 04:02:53.083696 6808 master.cpp:7947] Authenticating > scheduler-188f89c7-317a-44aa-a04e-35d0f7ef0309@172.17.0.2:38626 > I1025 04:02:53.083904 6808 authenticator.cpp:414] Starting authentication > session for crammd5-authenticatee(1077)@172.17.0.2:38626 > I1025 04:02:53.083873 6811 slave.cpp:585] Agent resources: > [{"name":"cpus","scalar":{"value":2.0},"type":"SCALAR"},{"name":"mem","scalar":{"value":1024.0},"type":"SCALAR"},{"name":"disk","scalar":{"value":1024.0},"type":"SCALAR"},{"name":"ports","ranges":{"range":[{"begin":31000,"end":32000}]},"type":"RANGES"}] > I1025 04:02:53.084163 6811 slave.cpp:593] Agent attributes: [ ] > I1025 04:02:53.084215 6827 authenticator.cpp:98] Creating new server SASL > connection > I1025 04:02:53.084241 6811 slave.cpp:602] Agent hostname: a4020869f68c > I1025 04:02:53.084460 6823 status_update_manager.cpp:177] Pausing sending > status updates > I1025 04:02:53.084522 6827 authenticatee.cpp:213] Received SASL > authentication mechanisms: CRAM-MD5 > I1025 04:02:53.084548 6827 authenticatee.cpp:239] Attempting to authenticate > with mechanism 'CRAM-MD5' > I1025 04:02:53.084641 6807 authenticator.cpp:204] Received SASL > authentication start > I1025 04:02:53.084709 6807 authenticator.cpp:326] Authentication requires > more steps > I1025 04:02:53.084834 6820 authenticatee.cpp:259] Received SASL > authentication step > I1025 04:02:53.085031 6814 authenticator.cpp:232] Received SASL > authentication step > I1025 04:02:53.085105 6814 auxprop.cpp:109] Request to lookup properties for > user: 'test-principal' realm: 'a4020869f68c' server FQDN: 'a4020869f68c' > SASL_AUXPROP_VERIFY_AGAINST_HASH: false SASL_AUXPROP_OVERRIDE: false > SASL_AUXPROP_AUTHZID: false > I1025 04:02:53.085129 6814 auxprop.cpp:181] Looking up auxiliary property > '*userPassword' > I1025 04:02:53.085182 6814 auxprop.cpp:181] Looking up auxiliary property > '*cmusaslsecretCRAM-MD5' > I1025 04:02:53.085214 6814 auxprop.cpp:109] Request to lookup properties for > user: 'test-principal' realm: 'a4020869f68c' server FQDN: 'a4020869f68c' > SASL_AUXPROP_VERIFY_AGAINST_HASH: false SASL_AUXPROP_OVERRIDE: false > SASL_AUXPROP_AUTHZID: true > I1025 04:02:53.085232 6814 auxprop.cpp:131] Skipping auxiliary property > '*userPassword' since SASL_AUXPROP_AUTHZID == true > I1025 04:02:53.085247 6814 auxprop.cpp:131] Skipping auxiliary property > '*cmusaslsecretCRAM-MD5' since SASL_AUXPROP_AUTHZID == true > I1025 04:02:53.085270 6814 authenticator.cpp:318] Authentication success > I1025 04:02:53.085429 6817 master.cpp:7977] Successfully authenticated > principal 'test-principal' at > scheduler-188f89c7-317a-44aa-a04e-35d0f7ef0309@172.17.0.2:38626 > I1025 04:02:53.085454 6818 authenticator.cpp:432] Authentication session > cleanup for crammd5-authenticatee(1077)@172.17.0.2:38626 > I1025 04:02:53.085482 6820 authenticatee.cpp:299] Authentication success > I1025 04:02:53.085904 6820 sched.cpp:502] Successfully authenticated with > master master@172.17.0.2:38626 > I1025 04:02:53.085937 6820 sched.cpp:820] Sending SUBSCRIBE call to > master@172.17.0.2:38626 > I1025 04:02:53.086078 6820 sched.cpp:853] Will retry registration in > 466.181463ms if necessary > I1025 04:02:53.086318 6810 master.cpp:2929] Received SUBSCRIBE call for > framework 'default' at > scheduler-188f89c7-317a-44aa-a04e-35d0f7ef0309@172.17.0.2:38626 > I1025 04:02:53.086372 6810 master.cpp:2263] Authorizing framework principal > 'test-principal' to receive offers for roles '{ * }' > I1025 04:02:53.086642 6822 state.cpp:64] Recovering state from > '/tmp/SlaveTest_ContainersEndpoint_RzfA6D/meta' > I1025 04:02:53.086926 6812 status_update_manager.cpp:203] Recovering status > update manager > I1025 04:02:53.087033 6808 master.cpp:3009] Subscribing framework default > with checkpointing disabled and capabilities [ RESERVATION_REFINEMENT ] > I1025 04:02:53.087266 6806 slave.cpp:6310] Finished recovery > I1025 04:02:53.087702 6809 sched.cpp:747] Framework registered with > 2dc7ad46-f111-4762-9bf6-ef428a6f6d53-0000 > I1025 04:02:53.087786 6809 sched.cpp:761] Scheduler::registered took 38714ns > I1025 04:02:53.087977 6826 hierarchical.cpp:303] Added framework > 2dc7ad46-f111-4762-9bf6-ef428a6f6d53-0000 > I1025 04:02:53.088042 6814 status_update_manager.cpp:177] Pausing sending > status updates > I1025 04:02:53.088057 6825 slave.cpp:993] New master detected at > master@172.17.0.2:38626 > I1025 04:02:53.088184 6825 slave.cpp:1028] Detecting new master > I1025 04:02:53.088302 6826 hierarchical.cpp:1488] Performed allocation for 0 > agents in 115530ns > I1025 04:02:53.096532 6818 slave.cpp:1055] Authenticating with master > master@172.17.0.2:38626 > I1025 04:02:53.096709 6818 slave.cpp:1064] Using default CRAM-MD5 > authenticatee > I1025 04:02:53.097173 6818 authenticatee.cpp:121] Creating new client SASL > connection > I1025 04:02:53.097568 6810 master.cpp:7947] Authenticating > slave(539)@172.17.0.2:38626 > I1025 04:02:53.097718 6819 authenticator.cpp:414] Starting authentication > session for crammd5-authenticatee(1078)@172.17.0.2:38626 > I1025 04:02:53.098012 6822 authenticator.cpp:98] Creating new server SASL > connection > I1025 04:02:53.098245 6821 authenticatee.cpp:213] Received SASL > authentication mechanisms: CRAM-MD5 > I1025 04:02:53.098291 6821 authenticatee.cpp:239] Attempting to authenticate > with mechanism 'CRAM-MD5' > I1025 04:02:53.098415 6815 authenticator.cpp:204] Received SASL > authentication start > I1025 04:02:53.098482 6815 authenticator.cpp:326] Authentication requires > more steps > I1025 04:02:53.098582 6815 authenticatee.cpp:259] Received SASL > authentication step > I1025 04:02:53.098703 6823 authenticator.cpp:232] Received SASL > authentication step > I1025 04:02:53.098743 6823 auxprop.cpp:109] Request to lookup properties for > user: 'test-principal' realm: 'a4020869f68c' server FQDN: 'a4020869f68c' > SASL_AUXPROP_VERIFY_AGAINST_HASH: false SASL_AUXPROP_OVERRIDE: false > SASL_AUXPROP_AUTHZID: false > I1025 04:02:53.098767 6823 auxprop.cpp:181] Looking up auxiliary property > '*userPassword' > I1025 04:02:53.098829 6823 auxprop.cpp:181] Looking up auxiliary property > '*cmusaslsecretCRAM-MD5' > I1025 04:02:53.098866 6823 auxprop.cpp:109] Request to lookup properties for > user: 'test-principal' realm: 'a4020869f68c' server FQDN: 'a4020869f68c' > SASL_AUXPROP_VERIFY_AGAINST_HASH: false SASL_AUXPROP_OVERRIDE: false > SASL_AUXPROP_AUTHZID: true > I1025 04:02:53.098886 6823 auxprop.cpp:131] Skipping auxiliary property > '*userPassword' since SASL_AUXPROP_AUTHZID == true > I1025 04:02:53.098917 6823 auxprop.cpp:131] Skipping auxiliary property > '*cmusaslsecretCRAM-MD5' since SASL_AUXPROP_AUTHZID == true > I1025 04:02:53.098983 6823 authenticator.cpp:318] Authentication success > I1025 04:02:53.099086 6812 authenticatee.cpp:299] Authentication success > I1025 04:02:53.099150 6827 master.cpp:7977] Successfully authenticated > principal 'test-principal' at slave(539)@172.17.0.2:38626 > I1025 04:02:53.099378 6813 authenticator.cpp:432] Authentication session > cleanup for crammd5-authenticatee(1078)@172.17.0.2:38626 > I1025 04:02:53.099433 6812 slave.cpp:1147] Successfully authenticated with > master master@172.17.0.2:38626 > I1025 04:02:53.099699 6812 slave.cpp:1626] Will retry registration in > 17.954247ms if necessary > I1025 04:02:53.099915 6829 master.cpp:5801] Received register agent message > from slave(539)@172.17.0.2:38626 (a4020869f68c) > I1025 04:02:53.099952 6829 master.cpp:3838] Authorizing agent with principal > 'test-principal' > I1025 04:02:53.100402 6814 master.cpp:5861] Authorized registration of agent > at slave(539)@172.17.0.2:38626 (a4020869f68c) > I1025 04:02:53.100524 6814 master.cpp:5954] Registering agent at > slave(539)@172.17.0.2:38626 (a4020869f68c) with id > 2dc7ad46-f111-4762-9bf6-ef428a6f6d53-S0 > I1025 04:02:53.100894 6826 registrar.cpp:495] Applied 1 operations in > 80109ns; attempting to update the registry > I1025 04:02:53.101490 6826 registrar.cpp:552] Successfully updated the > registry in 525056ns > I1025 04:02:53.101676 6810 master.cpp:6001] Admitted agent > 2dc7ad46-f111-4762-9bf6-ef428a6f6d53-S0 at slave(539)@172.17.0.2:38626 > (a4020869f68c) > I1025 04:02:53.102337 6822 slave.cpp:1193] Registered with master > master@172.17.0.2:38626; given agent ID > 2dc7ad46-f111-4762-9bf6-ef428a6f6d53-S0 > I1025 04:02:53.102442 6811 status_update_manager.cpp:184] Resuming sending > status updates > I1025 04:02:53.102229 6810 master.cpp:6032] Registered agent > 2dc7ad46-f111-4762-9bf6-ef428a6f6d53-S0 at slave(539)@172.17.0.2:38626 > (a4020869f68c) with > [{"name":"cpus","scalar":{"value":2.0},"type":"SCALAR"},{"name":"mem","scalar":{"value":1024.0},"type":"SCALAR"},{"name":"disk","scalar":{"value":1024.0},"type":"SCALAR"},{"name":"ports","ranges":{"range":[{"begin":31000,"end":32000}]},"type":"RANGES"}] > I1025 04:02:53.102644 6821 hierarchical.cpp:593] Added agent > 2dc7ad46-f111-4762-9bf6-ef428a6f6d53-S0 (a4020869f68c) with cpus:2; mem:1024; > disk:1024; ports:[31000-32000] (allocated: {}) > I1025 04:02:53.102702 6822 slave.cpp:1213] Checkpointing SlaveInfo to > '/tmp/SlaveTest_ContainersEndpoint_RzfA6D/meta/slaves/2dc7ad46-f111-4762-9bf6-ef428a6f6d53-S0/slave.info' > I1025 04:02:53.103113 6822 slave.cpp:1262] Forwarding total oversubscribed > resources {} > I1025 04:02:53.103298 6807 master.cpp:6828] Received update of agent > 2dc7ad46-f111-4762-9bf6-ef428a6f6d53-S0 at slave(539)@172.17.0.2:38626 > (a4020869f68c) with total oversubscribed resources {} > I1025 04:02:53.103512 6807 master.cpp:6839] Ignoring update on agent > 2dc7ad46-f111-4762-9bf6-ef428a6f6d53-S0 at slave(539)@172.17.0.2:38626 > (a4020869f68c) as it reports no changes > I1025 04:02:53.103955 6821 hierarchical.cpp:1488] Performed allocation for 1 > agents in 1.162098ms > I1025 04:02:53.104391 6827 master.cpp:7777] Sending 1 offers to framework > 2dc7ad46-f111-4762-9bf6-ef428a6f6d53-0000 (default) at > scheduler-188f89c7-317a-44aa-a04e-35d0f7ef0309@172.17.0.2:38626 > I1025 04:02:53.104918 6823 sched.cpp:917] Scheduler::resourceOffers took > 112626ns > I1025 04:02:53.106896 6829 master.cpp:9400] Removing offer > 2dc7ad46-f111-4762-9bf6-ef428a6f6d53-O0 > I1025 04:02:53.107036 6829 master.cpp:4196] Processing ACCEPT call for > offers: [ 2dc7ad46-f111-4762-9bf6-ef428a6f6d53-O0 ] on agent > 2dc7ad46-f111-4762-9bf6-ef428a6f6d53-S0 at slave(539)@172.17.0.2:38626 > (a4020869f68c) for framework 2dc7ad46-f111-4762-9bf6-ef428a6f6d53-0000 > (default) at scheduler-188f89c7-317a-44aa-a04e-35d0f7ef0309@172.17.0.2:38626 > I1025 04:02:53.107153 6829 master.cpp:3565] Authorizing framework principal > 'test-principal' to launch task 1 > I1025 04:02:53.107358 6829 master.cpp:3565] Authorizing framework principal > 'test-principal' to launch task 2 > W1025 04:02:53.108886 6829 validation.cpp:1338] Executor 'executor-1' for > task '1' uses less CPUs (None) than the minimum required (0.01). Please > update your executor, as this will be mandatory in future releases. > W1025 04:02:53.108920 6829 validation.cpp:1350] Executor 'executor-1' for > task '1' uses less memory (None) than the minimum required (32MB). Please > update your executor, as this will be mandatory in future releases. > I1025 04:02:53.109328 6829 master.cpp:10146] Adding task 1 with resources > [{"allocation_info":{"role":"*"},"name":"cpus","scalar":{"value":1.0},"type":"SCALAR"},{"allocation_info":{"role":"*"},"name":"mem","scalar":{"value":512.0},"type":"SCALAR"}] > on agent 2dc7ad46-f111-4762-9bf6-ef428a6f6d53-S0 at > slave(539)@172.17.0.2:38626 (a4020869f68c) > I1025 04:02:53.109657 6829 master.cpp:4879] Launching task 1 of framework > 2dc7ad46-f111-4762-9bf6-ef428a6f6d53-0000 (default) at > scheduler-188f89c7-317a-44aa-a04e-35d0f7ef0309@172.17.0.2:38626 with > resources > [{"allocation_info":{"role":"*"},"name":"cpus","scalar":{"value":1.0},"type":"SCALAR"},{"allocation_info":{"role":"*"},"name":"mem","scalar":{"value":512.0},"type":"SCALAR"}] > on agent 2dc7ad46-f111-4762-9bf6-ef428a6f6d53-S0 at > slave(539)@172.17.0.2:38626 (a4020869f68c) > I1025 04:02:53.110110 6818 slave.cpp:1747] Got assigned task '1' for > framework 2dc7ad46-f111-4762-9bf6-ef428a6f6d53-0000 > W1025 04:02:53.110733 6829 validation.cpp:1338] Executor 'executor-2' for > task '2' uses less CPUs (None) than the minimum required (0.01). Please > update your executor, as this will be mandatory in future releases. > I1025 04:02:53.110743 6818 slave.cpp:2015] Authorizing task '1' for > framework 2dc7ad46-f111-4762-9bf6-ef428a6f6d53-0000 > W1025 04:02:53.110766 6829 validation.cpp:1350] Executor 'executor-2' for > task '2' uses less memory (None) than the minimum required (32MB). Please > update your executor, as this will be mandatory in future releases. > I1025 04:02:53.110800 6818 slave.cpp:6809] Authorizing framework principal > 'test-principal' to launch task 1 > I1025 04:02:53.111147 6829 master.cpp:10146] Adding task 2 with resources > [{"allocation_info":{"role":"*"},"name":"cpus","scalar":{"value":1.0},"type":"SCALAR"},{"allocation_info":{"role":"*"},"name":"mem","scalar":{"value":512.0},"type":"SCALAR"}] > on agent 2dc7ad46-f111-4762-9bf6-ef428a6f6d53-S0 at > slave(539)@172.17.0.2:38626 (a4020869f68c) > I1025 04:02:53.111491 6811 slave.cpp:2183] Launching task '1' for framework > 2dc7ad46-f111-4762-9bf6-ef428a6f6d53-0000 > I1025 04:02:53.111441 6829 master.cpp:4879] Launching task 2 of framework > 2dc7ad46-f111-4762-9bf6-ef428a6f6d53-0000 (default) at > scheduler-188f89c7-317a-44aa-a04e-35d0f7ef0309@172.17.0.2:38626 with > resources > [{"allocation_info":{"role":"*"},"name":"cpus","scalar":{"value":1.0},"type":"SCALAR"},{"allocation_info":{"role":"*"},"name":"mem","scalar":{"value":512.0},"type":"SCALAR"}] > on agent 2dc7ad46-f111-4762-9bf6-ef428a6f6d53-S0 at > slave(539)@172.17.0.2:38626 (a4020869f68c) > I1025 04:02:53.111994 6811 paths.cpp:605] Trying to chown > '/tmp/SlaveTest_ContainersEndpoint_RzfA6D/slaves/2dc7ad46-f111-4762-9bf6-ef428a6f6d53-S0/frameworks/2dc7ad46-f111-4762-9bf6-ef428a6f6d53-0000/executors/executor-1/runs/deb5e9ee-b64c-4dcb-8fac-9eefdbc721ed' > to user 'mesos' > I1025 04:02:53.112248 6811 slave.cpp:7283] Launching executor 'executor-1' > of framework 2dc7ad46-f111-4762-9bf6-ef428a6f6d53-0000 with resources [] in > work directory > '/tmp/SlaveTest_ContainersEndpoint_RzfA6D/slaves/2dc7ad46-f111-4762-9bf6-ef428a6f6d53-S0/frameworks/2dc7ad46-f111-4762-9bf6-ef428a6f6d53-0000/executors/executor-1/runs/deb5e9ee-b64c-4dcb-8fac-9eefdbc721ed' > I1025 04:02:53.113008 6811 slave.cpp:2411] Queued task '1' for executor > 'executor-1' of framework 2dc7ad46-f111-4762-9bf6-ef428a6f6d53-0000 > I1025 04:02:53.113246 6826 hierarchical.cpp:887] Updated allocation of > framework 2dc7ad46-f111-4762-9bf6-ef428a6f6d53-0000 on agent > 2dc7ad46-f111-4762-9bf6-ef428a6f6d53-S0 from cpus(allocated: *):2; > mem(allocated: *):1024; disk(allocated: *):1024; ports(allocated: > *):[31000-32000] to cpus(allocated: *):2; mem(allocated: *):1024; > disk(allocated: *):1024; ports(allocated: *):[31000-32000] > I1025 04:02:53.113325 6811 slave.cpp:1747] Got assigned task '2' for > framework 2dc7ad46-f111-4762-9bf6-ef428a6f6d53-0000 > I1025 04:02:53.113775 6811 slave.cpp:944] Successfully attached > '/tmp/SlaveTest_ContainersEndpoint_RzfA6D/slaves/2dc7ad46-f111-4762-9bf6-ef428a6f6d53-S0/frameworks/2dc7ad46-f111-4762-9bf6-ef428a6f6d53-0000/executors/executor-1/runs/deb5e9ee-b64c-4dcb-8fac-9eefdbc721ed' > to virtual path > '/tmp/SlaveTest_ContainersEndpoint_RzfA6D/slaves/2dc7ad46-f111-4762-9bf6-ef428a6f6d53-S0/frameworks/2dc7ad46-f111-4762-9bf6-ef428a6f6d53-0000/executors/executor-1/runs/latest' > I1025 04:02:53.113832 6811 slave.cpp:944] Successfully attached > '/tmp/SlaveTest_ContainersEndpoint_RzfA6D/slaves/2dc7ad46-f111-4762-9bf6-ef428a6f6d53-S0/frameworks/2dc7ad46-f111-4762-9bf6-ef428a6f6d53-0000/executors/executor-1/runs/deb5e9ee-b64c-4dcb-8fac-9eefdbc721ed' > to virtual path > '/frameworks/2dc7ad46-f111-4762-9bf6-ef428a6f6d53-0000/executors/executor-1/runs/latest' > I1025 04:02:53.113829 6826 hierarchical.cpp:1163] Recovered ports(allocated: > *):[31000-32000]; disk(allocated: *):1024 (total: cpus:2; mem:1024; > disk:1024; ports:[31000-32000], allocated: cpus(allocated: *):2; > mem(allocated: *):1024) on agent 2dc7ad46-f111-4762-9bf6-ef428a6f6d53-S0 from > framework 2dc7ad46-f111-4762-9bf6-ef428a6f6d53-0000 > I1025 04:02:53.113880 6826 hierarchical.cpp:1209] Framework > 2dc7ad46-f111-4762-9bf6-ef428a6f6d53-0000 filtered agent > 2dc7ad46-f111-4762-9bf6-ef428a6f6d53-S0 for 5secs > I1025 04:02:53.113889 6811 slave.cpp:944] Successfully attached > '/tmp/SlaveTest_ContainersEndpoint_RzfA6D/slaves/2dc7ad46-f111-4762-9bf6-ef428a6f6d53-S0/frameworks/2dc7ad46-f111-4762-9bf6-ef428a6f6d53-0000/executors/executor-1/runs/deb5e9ee-b64c-4dcb-8fac-9eefdbc721ed' > to virtual path > '/tmp/SlaveTest_ContainersEndpoint_RzfA6D/slaves/2dc7ad46-f111-4762-9bf6-ef428a6f6d53-S0/frameworks/2dc7ad46-f111-4762-9bf6-ef428a6f6d53-0000/executors/executor-1/runs/deb5e9ee-b64c-4dcb-8fac-9eefdbc721ed' > I1025 04:02:53.114147 6811 slave.cpp:2874] Launching container > deb5e9ee-b64c-4dcb-8fac-9eefdbc721ed for executor 'executor-1' of framework > 2dc7ad46-f111-4762-9bf6-ef428a6f6d53-0000 > I1025 04:02:53.114493 6811 slave.cpp:2015] Authorizing task '2' for > framework 2dc7ad46-f111-4762-9bf6-ef428a6f6d53-0000 > I1025 04:02:53.114563 6811 slave.cpp:6809] Authorizing framework principal > 'test-principal' to launch task 2 > I1025 04:02:53.115202 6811 slave.cpp:2183] Launching task '2' for framework > 2dc7ad46-f111-4762-9bf6-ef428a6f6d53-0000 > I1025 04:02:53.115674 6811 paths.cpp:605] Trying to chown > '/tmp/SlaveTest_ContainersEndpoint_RzfA6D/slaves/2dc7ad46-f111-4762-9bf6-ef428a6f6d53-S0/frameworks/2dc7ad46-f111-4762-9bf6-ef428a6f6d53-0000/executors/executor-2/runs/a542c575-457d-49f0-b271-472245d3878d' > to user 'mesos' > I1025 04:02:53.115883 6811 slave.cpp:7283] Launching executor 'executor-2' > of framework 2dc7ad46-f111-4762-9bf6-ef428a6f6d53-0000 with resources [] in > work directory > '/tmp/SlaveTest_ContainersEndpoint_RzfA6D/slaves/2dc7ad46-f111-4762-9bf6-ef428a6f6d53-S0/frameworks/2dc7ad46-f111-4762-9bf6-ef428a6f6d53-0000/executors/executor-2/runs/a542c575-457d-49f0-b271-472245d3878d' > I1025 04:02:53.116703 6811 slave.cpp:2411] Queued task '2' for executor > 'executor-2' of framework 2dc7ad46-f111-4762-9bf6-ef428a6f6d53-0000 > I1025 04:02:53.116796 6811 slave.cpp:944] Successfully attached > '/tmp/SlaveTest_ContainersEndpoint_RzfA6D/slaves/2dc7ad46-f111-4762-9bf6-ef428a6f6d53-S0/frameworks/2dc7ad46-f111-4762-9bf6-ef428a6f6d53-0000/executors/executor-2/runs/a542c575-457d-49f0-b271-472245d3878d' > to virtual path > '/tmp/SlaveTest_ContainersEndpoint_RzfA6D/slaves/2dc7ad46-f111-4762-9bf6-ef428a6f6d53-S0/frameworks/2dc7ad46-f111-4762-9bf6-ef428a6f6d53-0000/executors/executor-2/runs/latest' > I1025 04:02:53.116850 6811 slave.cpp:944] Successfully attached > '/tmp/SlaveTest_ContainersEndpoint_RzfA6D/slaves/2dc7ad46-f111-4762-9bf6-ef428a6f6d53-S0/frameworks/2dc7ad46-f111-4762-9bf6-ef428a6f6d53-0000/executors/executor-2/runs/a542c575-457d-49f0-b271-472245d3878d' > to virtual path > '/frameworks/2dc7ad46-f111-4762-9bf6-ef428a6f6d53-0000/executors/executor-2/runs/latest' > I1025 04:02:53.116902 6811 slave.cpp:944] Successfully attached > '/tmp/SlaveTest_ContainersEndpoint_RzfA6D/slaves/2dc7ad46-f111-4762-9bf6-ef428a6f6d53-S0/frameworks/2dc7ad46-f111-4762-9bf6-ef428a6f6d53-0000/executors/executor-2/runs/a542c575-457d-49f0-b271-472245d3878d' > to virtual path > '/tmp/SlaveTest_ContainersEndpoint_RzfA6D/slaves/2dc7ad46-f111-4762-9bf6-ef428a6f6d53-S0/frameworks/2dc7ad46-f111-4762-9bf6-ef428a6f6d53-0000/executors/executor-2/runs/a542c575-457d-49f0-b271-472245d3878d' > *** Aborted at 1508904173 (unix time) try "date -d @1508904173" if you are > using GNU date *** > PC: @ 0x2ba56a14582d (unknown) > W1025 04:02:53.117486 6807 process.cpp:3193] Attempted to spawn already > running process version@172.17.0.2:38626 > *** SIGSEGV (@0xd0) received by PID 6805 (TID 0x2ba573896700) from PID 208; > stack trace: *** > I1025 04:02:53.117677 6807 exec.cpp:162] Version: 1.5.0 > I1025 04:02:53.117945 6806 exec.cpp:212] Executor started at: > executor(138)@172.17.0.2:38626 with pid 6805 > @ 0x2ba5e07dcaf7 (unknown) > @ 0x2ba5e07e1449 (unknown) > @ 0x2ba569efc330 (unknown) > @ 0x2ba56a14582d (unknown) > @ 0x55adfc9771a8 os::getenv() > @ 0x2ba565c5510b mesos::internal::slave::executorEnvironment() > @ 0x2ba565c50779 mesos::internal::slave::Slave::launchExecutor() > @ 0x2ba565d159f8 > _ZZN7process8dispatchIN5mesos8internal5slave5SlaveERK6OptionINS_6FutureINS1_6SecretEEEERKNS1_11FrameworkIDERKNS1_10ExecutorIDERKS5_INS1_8TaskInfoEESB_SE_SH_SL_EEvRKNS_3PIDIT_EEMSN_FvT0_T1_T2_T3_EOT4_OT5_OT6_OT7_ENKUlRS9_RSC_RSF_RSJ_PNS_11ProcessBaseEE_clES15_S16_S17_S18_S1A_ > @ 0x2ba565d1586b > _ZNSt5_BindIFZN7process8dispatchIN5mesos8internal5slave5SlaveERK6OptionINS0_6FutureINS2_6SecretEEEERKNS2_11FrameworkIDERKNS2_10ExecutorIDERKS6_INS2_8TaskInfoEESC_SF_SI_SM_EEvRKNS0_3PIDIT_EEMSO_FvT0_T1_T2_T3_EOT4_OT5_OT6_OT7_EUlRSA_RSD_RSG_RSK_PNS0_11ProcessBaseEE_SA_SD_SG_SK_St12_PlaceholderILi1EEEE6__callIvJOS1B_EJLm0ELm1ELm2ELm3ELm4EEEESO_OSt5tupleIJDpT0_EESt12_Index_tupleIJXspT1_EEE > @ 0x2ba565d15766 > _ZNSt5_BindIFZN7process8dispatchIN5mesos8internal5slave5SlaveERK6OptionINS0_6FutureINS2_6SecretEEEERKNS2_11FrameworkIDERKNS2_10ExecutorIDERKS6_INS2_8TaskInfoEESC_SF_SI_SM_EEvRKNS0_3PIDIT_EEMSO_FvT0_T1_T2_T3_EOT4_OT5_OT6_OT7_EUlRSA_RSD_RSG_RSK_PNS0_11ProcessBaseEE_SA_SD_SG_SK_St12_PlaceholderILi1EEEEclIJS1B_EvEESS_DpOT_ > @ 0x2ba565d1521f > _ZNSt17_Function_handlerIFvPN7process11ProcessBaseEESt5_BindIFZNS0_8dispatchIN5mesos8internal5slave5SlaveERK6OptionINS0_6FutureINS6_6SecretEEEERKNS6_11FrameworkIDERKNS6_10ExecutorIDERKSA_INS6_8TaskInfoEESG_SJ_SM_SQ_EEvRKNS0_3PIDIT_EEMSS_FvT0_T1_T2_T3_EOT4_OT5_OT6_OT7_EUlRSE_RSH_RSK_RSO_S2_E_SE_SH_SK_SO_St12_PlaceholderILi1EEEEE9_M_invokeERKSt9_Any_dataS2_ > @ 0x2ba567123774 std::function<>::operator()() > @ 0x2ba567107cc4 process::ProcessBase::visit() > @ 0x2ba567190b7e process::DispatchEvent::visit() > @ 0x55adfc95c7b1 process::ProcessBase::serve() > @ 0x2ba5671053c2 process::ProcessManager::resume() > @ 0x2ba5671151d1 > process::ProcessManager::init_threads()::$_9::operator()() > @ 0x2ba567115115 > _ZNSt12_Bind_simpleIFZN7process14ProcessManager12init_threadsEvE3$_9vEE9_M_invokeIJEEEvSt12_Index_tupleIJXspT_EEE > @ 0x2ba5671150e5 std::_Bind_simple<>::operator()() > @ 0x2ba5671150bc std::thread::_Impl<>::_M_run() > @ 0x2ba56977da60 (unknown) > @ 0x2ba569ef4184 start_thread > @ 0x2ba56a207ffd (unknown) > {noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005)