[ 
https://issues.apache.org/jira/browse/MESOS-7106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Wu reassigned MESOS-7106:
--------------------------------

    Assignee: Joseph Wu

> Test ContentTypeAndSSLConfig/SchedulerSSLTest.RunTaskAndTeardown/1 segfaults
> ----------------------------------------------------------------------------
>
>                 Key: MESOS-7106
>                 URL: https://issues.apache.org/jira/browse/MESOS-7106
>             Project: Mesos
>          Issue Type: Bug
>    Affects Versions: 1.2.0
>         Environment: centos7, SSL build
>            Reporter: Benjamin Bannier
>            Assignee: Joseph Wu
>              Labels: flaky-test, test
>
> {{ContentTypeAndSSLConfig/SchedulerSSLTest.RunTaskAndTeardown/1}} segfaulted 
> in our internal CI:
> {noformat}
> [ RUN      ] ContentTypeAndSSLConfig/SchedulerSSLTest.RunTaskAndTeardown/1
> W0210 03:08:05.018744  1020 process.cpp:3029] Attempted to spawn a process 
> (__http_connection__(1079)@10.168.212.35:42363) after finalizing libprocess!
> *** Aborted at 1486696085 (unix time) try "date -d @1486696085" if you are 
> using GNU date ***
> I0210 03:08:05.023609  6019 process.cpp:1246] libprocess is initialized on 
> 10.168.212.35:44850 with 8 worker threads
> I0210 03:08:05.024163  6019 cluster.cpp:160] Creating default 'local' 
> authorizer
> I0210 03:08:05.025065  1025 master.cpp:383] Master 
> 7adcbe15-38a9-4512-aa9c-8d5f7538e4ee (ip-10-168-212-35.ec2.internal) started 
> on 10.168.212.35:44850
> I0210 03:08:05.025089  1025 master.cpp:385] Flags at startup: --acls="" 
> --agent_ping_timeout="15secs" --agent_reregister_timeout="10mins" 
> --allocation_interval="1secs" --allocator="HierarchicalDRF" 
> --authenticate_agents="true" --authenticate_frameworks="true" 
> --authenticate_http_frameworks="true" --authenticate_http_readonly="true" 
> --authenticate_http_readwrite="true" --authenticators="crammd5" 
> --authorizers="local" --credentials="/tmp/5DRa8u/credentials" 
> --framework_sorter="drf" --help="false" --hostname_lookup="true" 
> --http_authenticators="basic" --http_framework_authenticators="basic" 
> --initialize_driver_logging="true" --log_auto_initialize="true" 
> --logbufsecs="0" --logging_level="INFO" --max_agent_ping_timeouts="5" 
> --max_completed_frameworks="50" --max_completed_tasks_per_framework="1000" 
> --max_unreachable_tasks_per_framework="1000" --quiet="false" 
> --recovery_agent_removal_limit="100%" --registry="in_memory" 
> --registry_fetch_timeout="1mins" --registry_gc_interval="15mins" 
> --registry_max_agent_age="2weeks" --registry_max_agent_count="102400" 
> --registry_store_timeout="100secs" --registry_strict="false" 
> --root_submissions="true" --user_sorter="drf" --version="false" 
> --webui_dir="/usr/local/share/mesos/webui" --work_dir="/tmp/5DRa8u/master" 
> --zk_session_timeout="10secs"
> I0210 03:08:05.025264  1025 master.cpp:435] Master only allowing 
> authenticated frameworks to register
> I0210 03:08:05.025276  1025 master.cpp:449] Master only allowing 
> authenticated agents to register
> I0210 03:08:05.025285  1025 master.cpp:462] Master only allowing 
> authenticated HTTP frameworks to register
> I0210 03:08:05.025293  1025 credentials.hpp:37] Loading credentials for 
> authentication from '/tmp/5DRa8u/credentials'
> I0210 03:08:05.025387  1025 master.cpp:507] Using default 'crammd5' 
> authenticator
> I0210 03:08:05.025441  1025 http.cpp:919] Using default 'basic' HTTP 
> authenticator for realm 'mesos-master-readonly'
> I0210 03:08:05.025512  1025 http.cpp:919] Using default 'basic' HTTP 
> authenticator for realm 'mesos-master-readwrite'
> I0210 03:08:05.025560  1025 http.cpp:919] Using default 'basic' HTTP 
> authenticator for realm 'mesos-master-scheduler'
> I0210 03:08:05.025619  1025 master.cpp:587] Authorization enabled
> I0210 03:08:05.025728  1023 hierarchical.cpp:161] Initialized hierarchical 
> allocator process
> I0210 03:08:05.025754  1027 whitelist_watcher.cpp:77] No whitelist given
> PC: @     0x7f69d2296012 process::ProcessManager::spawn()
> *** SIGSEGV (@0x0) received by PID 6019 (TID 0x7f69c46d5700) from PID 0; 
> stack trace: ***
>     @     0x7f69c2408725 (unknown)
> I0210 03:08:05.026340  1023 master.cpp:2124] Elected as the leading master!
> I0210 03:08:05.026357  1023 master.cpp:1646] Recovering from registrar
> I0210 03:08:05.026406  1025 registrar.cpp:329] Recovering registrar
>     @     0x7f69c240d2f1 (unknown)
>     @     0x7f69c24011e8 (unknown)
> I0210 03:08:05.027294  1024 registrar.cpp:362] Successfully fetched the 
> registry (0B) in 865024ns
> I0210 03:08:05.027330  1024 registrar.cpp:461] Applied 1 operations in 
> 2848ns; attempting to update the registry
>     @     0x7f69d027b370 (unknown)
> I0210 03:08:05.028261  1028 registrar.cpp:506] Successfully updated the 
> registry in 916992ns
> I0210 03:08:05.028313  1028 registrar.cpp:392] Successfully recovered 
> registrar
> I0210 03:08:05.028419  1028 master.cpp:1762] Recovered 0 agents from the 
> registry (172B); allowing 10mins for agents to re-register
> I0210 03:08:05.028448  1026 hierarchical.cpp:188] Skipping recovery of 
> hierarchical allocator: nothing to recover
>     @     0x7f69d2296012 process::ProcessManager::spawn()
> I0210 03:08:05.030078  6019 cluster.cpp:446] Creating default 'local' 
> authorizer
> I0210 03:08:05.030418  1021 slave.cpp:211] Mesos agent started on 
> (818)@10.168.212.35:44850
> I0210 03:08:05.030581  6019 scheduler.cpp:184] Version: 1.3.0
> I0210 03:08:05.030442  1021 slave.cpp:212] Flags at startup: --acls="" 
> --appc_simple_discovery_uri_prefix="http://"; 
> --appc_store_dir="/tmp/mesos/store/appc" --authenticate_http_readonly="true" 
> --authenticate_http_readwrite="true" --authenticatee="crammd5" 
> --authentication_backoff_factor="1secs" --authorizer="local" 
> --cgroups_cpu_enable_pids_and_tids_count="false" --cgroups_enable_cfs="false" 
> --cgroups_hierarchy="/sys/fs/cgroup" --cgroups_limit_swap="false" 
> --cgroups_root="mesos" --container_disk_watch_interval="15secs" 
> --containerizers="mesos" 
> --credential="/tmp/ContentTypeAndSSLConfig_SchedulerSSLTest_RunTaskAndTeardown_1_ZqeHXq/credential"
>  --default_role="*" --disk_watch_interval="1mins" --docker="docker" 
> --docker_kill_orphans="true" --docker_registry="https://registry-1.docker.io"; 
> --docker_remove_delay="6hrs" --docker_socket="/var/run/docker.sock" 
> --docker_stop_timeout="0ns" --docker_store_dir="/tmp/mesos/store/docker" 
> --docker_volume_checkpoint_dir="/var/run/mesos/isolators/docker/volume" 
> --enforce_container_disk_quota="false" 
> --executor_registration_timeout="1mins" 
> --executor_shutdown_grace_period="5secs" 
> --fetcher_cache_dir="/tmp/ContentTypeAndSSLConfig_SchedulerSSLTest_RunTaskAndTeardown_1_ZqeHXq/fetch"
>  --fetcher_cache_size="2GB" --frameworks_home="" --gc_delay="1weeks" 
> --gc_disk_headroom="0.1" --hadoop_home="" --help="false" 
> --hostname_lookup="true" --http_authenticators="basic" 
> --http_command_executor="false" 
> --http_credentials="/tmp/ContentTypeAndSSLConfig_SchedulerSSLTest_RunTaskAndTeardown_1_ZqeHXq/http_credentials"
>  --http_heartbeat_interval="30secs" --initialize_driver_logging="true" 
> --isolation="posix/cpu,posix/mem" --launcher="linux" 
> --launcher_dir="/home/centos/workspace/mesos/Mesos_CI-build/FLAG/SSL/label/mesos-ec2-centos-7/mesos/build/src"
>  --logbufsecs="0" --logging_level="INFO" 
> --max_completed_executors_per_framework="150" 
> --oversubscribed_resources_interval="15secs" --perf_duration="10secs" 
> --perf_interval="1mins" --qos_correction_interval_min="0ns" --quiet="false" 
> --recover="reconnect" --recovery_timeout="15mins" 
> --registration_backoff_factor="10ms" 
> --resources="cpus:2;gpus:0;mem:1024;disk:1024;ports:[31000-32000]" 
> --revocable_cpu_low_priority="true" 
> --runtime_dir="/tmp/ContentTypeAndSSLConfig_SchedulerSSLTest_RunTaskAndTeardown_1_ZqeHXq"
>  --sandbox_directory="/mnt/mesos/sandbox" --strict="true" 
> --switch_user="true" --systemd_enable_support="true" 
> --systemd_runtime_directory="/run/systemd/system" --version="false" 
> --work_dir="/tmp/ContentTypeAndSSLConfig_SchedulerSSLTest_RunTaskAndTeardown_1_FPCV2X"
> I0210 03:08:05.030650  1021 credentials.hpp:86] Loading credential for 
> authentication from 
> '/tmp/ContentTypeAndSSLConfig_SchedulerSSLTest_RunTaskAndTeardown_1_ZqeHXq/credential'
> I0210 03:08:05.030712  1021 slave.cpp:354] Agent using credential for: 
> test-principal
> I0210 03:08:05.030727  1021 credentials.hpp:37] Loading credentials for 
> authentication from 
> '/tmp/ContentTypeAndSSLConfig_SchedulerSSLTest_RunTaskAndTeardown_1_ZqeHXq/http_credentials'
> I0210 03:08:05.030791  1021 http.cpp:919] Using default 'basic' HTTP 
> authenticator for realm 'mesos-agent-readonly'
> I0210 03:08:05.030834  1021 http.cpp:919] Using default 'basic' HTTP 
> authenticator for realm 'mesos-agent-readwrite'
> I0210 03:08:05.031044  1025 scheduler.cpp:470] New master detected at 
> master@10.168.212.35:44850
> I0210 03:08:05.031404  1021 slave.cpp:541] Agent resources: cpus(*):2; 
> mem(*):1024; disk(*):1024; ports(*):[31000-32000]
> I0210 03:08:05.031440  1021 slave.cpp:549] Agent attributes: [  ]
> I0210 03:08:05.031445  1021 slave.cpp:554] Agent hostname: 
> ip-10-168-212-35.ec2.internal
> I0210 03:08:05.031496  1022 status_update_manager.cpp:177] Pausing sending 
> status updates
> I0210 03:08:05.031793  1021 state.cpp:62] Recovering state from 
> '/tmp/ContentTypeAndSSLConfig_SchedulerSSLTest_RunTaskAndTeardown_1_FPCV2X/meta'
> I0210 03:08:05.031877  1021 status_update_manager.cpp:203] Recovering status 
> update manager
> I0210 03:08:05.031976  1025 scheduler.cpp:479] Waiting for 0ns before 
> initiating a re-(connection) attempt with the master
> I0210 03:08:05.032043  1027 slave.cpp:5555] Finished recovery
> I0210 03:08:05.032328  1027 slave.cpp:5729] Querying resource estimator for 
> oversubscribable resources
>     @     0x7f69d229a646 process::spawn()
> I0210 03:08:05.032439  1027 slave.cpp:931] New master detected at 
> master@10.168.212.35:44850
> I0210 03:08:05.032445  1022 status_update_manager.cpp:177] Pausing sending 
> status updates
> I0210 03:08:05.032481  1027 slave.cpp:966] Detecting new master
> I0210 03:08:05.032542  1027 slave.cpp:5743] Received oversubscribable 
> resources {} from the resource estimator
>     @     0x7f69d222ee99 process::spawn<>()
>     @     0x7f69d2210634 process::http::Connection::Connection()
>     @     0x7f69d222b72c 
> _ZNSt17_Function_handlerIFN7process6FutureINS0_4http10ConnectionEEEvEZNS2_7connectERKNS0_7network7AddressENS2_6SchemeEEUlvE_E9_M_invokeERKSt9_Any_data
>     @     0x7f69d1b53e14 std::_Function_handler<>::_M_invoke()
>     @     0x7f69d1b6f6e6 process::internal::thenf<>()
>     @     0x7f69d33bc2d6 process::internal::run<>()
>     @     0x7f69d33bdfd7 process::Future<>::_set<>()
>     @     0x7f69d22eb1d7 
> process::network::internal::LibeventSSLSocketImpl::event_callback()
>     @     0x7f69d22eb627 
> process::network::internal::LibeventSSLSocketImpl::event_callback()
>     @     0x7f69cd7a95c0 (unknown)
>     @     0x7f69cd79fb05 (unknown)
>     @     0x7f69d22ff4cd process::EventLoop::run()
>     @     0x7f69cfc0c230 (unknown)
>     @     0x7f69d0273dc5 start_thread
>     @     0x7f69cf37573d __clone
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Reply via email to