[ https://issues.apache.org/jira/browse/IMPALA-8091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16749157#comment-16749157 ]
Michael Brown commented on IMPALA-8091: --------------------------------------- This is a lot of diagnostic info... {noformat} I0116 18:56:30.146215 30799 master_main.cc:80] Initializing master server... I0116 18:56:30.146333 30799 system_ntp.cc:122] Waiting up to --ntp_initial_sync_wait_secs=60 seconds for the clock to synchronize W0116 18:56:30.147728 30799 system_ntp.cc:132] Could not find ntp-wait; trying chrony waitsync instead: Not found: Unable to find binary: ntp-wait E0116 18:57:29.300457 30799 system_ntp.cc:157] Dumping NTP diagnostics E0116 18:57:29.302417 30799 system_ntp.cc:102] /sbin/ntptime ------------------------------------------ stdout: ntp_gettime() returns code 5 (ERROR) time dfea6d99.4d59a2ec Wed, Jan 16 2019 18:57:29.302, (.302149739), maximum error 16000000 us, estimated error 16000000 us, TAI offset 0 ntp_adjtime() returns code 5 (ERROR) modes 0x0 (), offset 0.000 us, frequency 129.100 ppm, interval 1 s, maximum error 16000000 us, estimated error 16000000 us, status 0x2041 (PLL,UNSYNC,NANO), time constant 6, precision 0.001 us, tolerance 500 ppm, E0116 18:57:29.330603 30799 system_ntp.cc:102] /sbin/ntpq -n -c timeout 1000 -c readvar -c sysinfo -c lpeers -c opeers -c version ------------------------------------------ stdout: associd=0 status=c618 leap_alarm, sync_ntp, 1 event, no_sys_peer, version="ntpd 4.2.6p5@1.2349-o Wed Apr 12 21:24:06 UTC 2017 (1)", processor="x86_64", system="Linux/3.10.0-693.5.2.el7.x86_64", leap=11, stratum=2, precision=-24, rootdelay=65.438, rootdisp=43.467, refid=96.126.122.39, reftime=dfea6d72.e0d7d2b9 Wed, Jan 16 2019 18:56:50.878, clock=dfea6d99.4f1265e8 Wed, Jan 16 2019 18:57:29.308, peer=64381, tc=6, mintc=3, offset=0.000, frequency=129.100, sys_jitter=11.220, clk_jitter=12.567, clk_wander=0.221 remote refid st t when poll reach delay offset jitter ============================================================================== +66.228.48.38 127.67.113.92 2 u 39 64 7 70.236 -35.592 28.619 +208.75.89.4 63.145.169.3 2 u 37 64 7 64.246 -20.776 29.110 *96.126.122.39 .GPS. 1 u 38 64 7 63.424 -31.810 28.640 +69.89.207.199 212.215.1.157 2 u 36 64 7 37.729 -37.207 28.643 remote local st t when poll reach delay offset disp ============================================================================== +66.228.48.38 172.28.204.72 2 u 39 64 7 70.236 -35.592 0.913 +208.75.89.4 172.28.204.72 2 u 37 64 7 64.246 -20.776 0.921 *96.126.122.39 172.28.204.72 1 u 38 64 7 63.424 -31.810 0.906 +69.89.207.199 172.28.204.72 2 u 36 64 7 37.729 -37.207 0.914 ntpq 4.2.6p5@1.2349-o Wed Apr 12 21:24:08 UTC 2017 (1) stderr: ***Command `sysinfo' unknown E0116 18:57:29.333056 30799 system_ntp.cc:102] /sbin/ntpdc -n -c timeout 1000 -c peers -c sysinfo -c sysstats -c version ------------------------------------------ stdout: remote local st poll reach delay offset disp ======================================================================= *96.126.122.39 172.28.204.72 1 64 7 0.06342 -0.031810 0.03009 =208.75.89.4 172.28.204.72 2 64 7 0.06424 -0.020776 0.03033 =69.89.207.199 172.28.204.72 2 64 7 0.03772 -0.037207 0.03023 =66.228.48.38 172.28.204.72 2 64 7 0.07022 -0.035592 0.03021 system peer: 96.126.122.39 system peer mode: client leap indicator: 11 stratum: 2 precision: -24 root distance: 0.06543 s root dispersion: 0.04346 s reference ID: [96.126.122.39] reference time: dfea6d72.e0d7d2b9 Wed, Jan 16 2019 18:56:50.878 system flags: auth ntp kernel stats jitter: 0.011215 s stability: 0.000 ppm broadcastdelay: 0.000000 s authdelay: 0.000000 s time since restart: 1459 time since reset: 1459 packets received: 180 packets processed: 124 current version: 124 previous version: 0 declined: 0 access denied: 0 bad length or format: 0 bad authentication: 0 rate exceeded: 0 ntpdc 4.2.6p5@1.2349-o Wed Apr 12 21:24:08 UTC 2017 (1) E0116 18:57:29.336165 30799 system_ntp.cc:102] /usr/bin/chronyc -n tracking ------------------------------------------ stdout: 506 Cannot talk to daemon E0116 18:57:29.339103 30799 system_ntp.cc:102] /usr/bin/chronyc -n sources ------------------------------------------ stdout: 506 Cannot talk to daemon F0116 18:57:29.339143 30799 master_main.cc:81] Check failed: _s.ok() Bad status: Runtime error: Cannot initialize clock: failed to wait for clock sync using command '/usr/bin/chronyc waitsync 60 0 0 1': /usr/bin/chronyc: process exited with non-zero status 1 {noformat} > minicluster kudu failure: Cannot initialize clock: failed to wait for clock > sync > -------------------------------------------------------------------------------- > > Key: IMPALA-8091 > URL: https://issues.apache.org/jira/browse/IMPALA-8091 > Project: IMPALA > Issue Type: Bug > Components: Infrastructure > Affects Versions: Impala 3.2.0 > Reporter: David Knupp > Assignee: Thomas Tauber-Marshall > Priority: Critical > Labels: broken-build > > *Console log*: > {noformat} > 19:10:04 19:10:04 Error executing impala SQL: > /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/logs/data_loading/sql/functional/create-functional-query-exhaustive-impala-generated-kudu-none-none.sql > See: > /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/logs/data_loading/sql/functional/create-functional-query-exhaustive-impala-generated-kudu-none-none.sql.log > 19:10:04 Background task Loading functional-query data (pid 4831) failed. > 19:10:04 Background task Loading TPC-H data (pid 4832) failed. > 19:10:04 ERROR in > /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/testdata/bin/create-load-data.sh > at line 85: CM_HOST=${2-} > [...] > 19:10:05 2019-01-16 19:10:05,461 - archive_core_dumps - INFO - Found core > files: ['./core.1547693849.30799.kudu-master', > './core.1547693849.30783.kudu-tserver', > './core.1547693849.30767.kudu-tserver', > './core.1547693849.30808.kudu-tserver'] > 19:10:05 2019-01-16 19:10:05,649 - archive_core_dumps - INFO - [New LWP 30799] > 19:10:05 [New LWP 30834] > 19:10:05 [New LWP 30835] > 19:10:05 [New LWP 30838] > 19:10:05 [New LWP 30837] > 19:10:05 [New LWP 30836] > 19:10:05 Core was generated by > `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'. > 19:10:05 Program terminated with signal SIGABRT, Aborted. > 19:10:05 #0 0x00007f9a2cc611f7 in ?? () > 19:10:05 > 19:10:05 2019-01-16 19:10:05,650 - archive_core_dumps - INFO - Found binary > path through GDB: > /data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c > 19:10:05 2019-01-16 19:10:05,893 - archive_core_dumps - WARNING - Failed to > determine binary because multiple candidate binaries were found and none of > their paths contained 'latest' to disambiguate: > 19:10:05 Core:./core.1547693849.30799.kudu-master > 19:10:05 > Binaries:['./testdata/cluster/node_templates/common/etc/init.d/kudu-master', > './testdata/cluster/cdh6/node-1/etc/init.d/kudu-master'] > 19:10:05 > 19:10:05 2019-01-16 19:10:05,917 - archive_core_dumps - INFO - [New LWP 30783] > 19:10:05 [New LWP 30810] > 19:10:05 [New LWP 30812] > 19:10:05 [New LWP 30811] > 19:10:05 [New LWP 30824] > 19:10:05 [New LWP 30820] > 19:10:05 Core was generated by > `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'. > 19:10:05 Program terminated with signal SIGABRT, Aborted. > 19:10:05 #0 0x00007f0b81fb11f7 in ?? () > {noformat} > *Backtraces*: > {noformat} > CORE: ./core.1547693849.30799.kudu-master > BINARY: ./be/build/latest/service/impalad > Core was generated by > `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'. > Program terminated with signal SIGABRT, Aborted. > #0 0x00007f9a2cc611f7 in ?? () > #0 0x00007f9a2cc611f7 in ?? () > #1 0x00007f9a2cc628e8 in ?? () > #2 0x0000000000000020 in ?? () > #3 0x0000000000000000 in ?? () > CORE: ./core.1547693849.30783.kudu-tserver > BINARY: ./be/build/latest/service/impalad > Core was generated by > `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'. > Program terminated with signal SIGABRT, Aborted. > #0 0x00007f0b81fb11f7 in ?? () > #0 0x00007f0b81fb11f7 in ?? () > #1 0x00007f0b81fb28e8 in ?? () > #2 0x0000000000000020 in ?? () > #3 0x0000000000000000 in ?? () > CORE: ./core.1547693849.30767.kudu-tserver > BINARY: ./be/build/latest/service/impalad > Core was generated by > `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'. > Program terminated with signal SIGABRT, Aborted. > #0 0x00007f60b1e2f1f7 in ?? () > #0 0x00007f60b1e2f1f7 in ?? () > #1 0x00007f60b1e308e8 in ?? () > #2 0x0000000000000020 in ?? () > #3 0x0000000000000000 in ?? () > CORE: ./core.1547693849.30808.kudu-tserver > BINARY: ./be/build/latest/service/impalad > Core was generated by > `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'. > Program terminated with signal SIGABRT, Aborted. > #0 0x00007fa3cb5591f7 in ?? () > #0 0x00007fa3cb5591f7 in ?? () > #1 0x00007fa3cb55a8e8 in ?? () > #2 0x0000000000000020 in ?? () > #3 0x0000000000000000 in ?? () > {noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org