[jira] [Commented] (IMPALA-3336) psycopg2 as used by qgen prefetches rows, can consume too much memory
[
https://issues.apache.org/jira/browse/IMPALA-3336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16847870#comment-16847870
]
Michael Brown commented on IMPALA-3336:
---
I haven't looked at this in a long time, but if it helps anyone, I came across
this today http://initd.org/psycopg/docs/faq.html#best-practices , which says:
bq. *When should I save and re-use a cursor as opposed to creating a new one as
needed?*
bq. Cursors are lightweight objects and creating lots of them should not pose
any kind of problem. But note that cursors used to fetch result sets will cache
the data and use memory in proportion to the result set size. Our suggestion is
to almost always create a new cursor and dispose old ones as soon as the data
is not required anymore (call close() on them.) The only exception are tight
loops where one usually use the same cursor for a whole bunch of INSERTs or
UPDATEs.
I *believe* in the RQG the cursors are kept persistent.
> psycopg2 as used by qgen prefetches rows, can consume too much memory
> -
>
> Key: IMPALA-3336
> URL: https://issues.apache.org/jira/browse/IMPALA-3336
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
>Affects Versions: Impala 2.6.0
>Reporter: Michael Brown
>Priority: Minor
>
> When I was experimenting with the query generator on EC2, I started having
> problems with the runner process being killed via kernel oom_killer. In
> checking in with Taras, he let me know this was happening to him as well. I
> need to figure out why that is happening so that the query generator can run.
> {noformat}
> 16:07:51 16:07:52 Query execution thread
> run_query_internal_i80swzzi4opv3q0p:hiveserver2[307]:Fetching up to 2000
> result rows
> 16:07:52 16:07:52 Query execution thread
> run_query_internal_i80swzzi4opv3q0p:hiveserver2[307]:Fetching up to 2000
> result rows
> 16:07:52 16:07:52 Query execution thread
> run_query_internal_i80swzzi4opv3q0p:hiveserver2[307]:Fetching up to 2000
> result rows
> 16:08:28 /tmp/hudson351019564031822959.sh: line 11: 4588 Killed
> ./tests/comparison/leopard/job.py
> 16:08:28 Build step 'Execute shell' marked build as failure
> 16:08:31 Finished: FAILURE
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-2990) Coordinator should timeout and cancel queries with unresponsive / stuck executors
[ https://issues.apache.org/jira/browse/IMPALA-2990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16835705#comment-16835705 ] Michael Brown commented on IMPALA-2990: --- Congrats on resolving this! > Coordinator should timeout and cancel queries with unresponsive / stuck > executors > - > > Key: IMPALA-2990 > URL: https://issues.apache.org/jira/browse/IMPALA-2990 > Project: IMPALA > Issue Type: Bug > Components: Distributed Exec >Affects Versions: Impala 2.3.0 >Reporter: Sailesh Mukil >Assignee: Thomas Tauber-Marshall >Priority: Critical > Labels: hang, observability, supportability > Fix For: Impala 3.3.0 > > > The coordinator currently waits indefinitely if it does not hear back from a > backend. This could cause a query to hang indefinitely in case of a network > error, etc. > We should add logic for determining when a backend is unresponsive and kill > the query. The logic should mostly revolve around Coordinator::Wait() and > Coordinator::UpdateFragmentExecStatus() based on whether it receives periodic > updates from a backed (via FragmentExecState::ReportStatusCb()). -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-8175) centos6: tests_minicluster_obj fails with pgrep usage error
[
https://issues.apache.org/jira/browse/IMPALA-8175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16763780#comment-16763780
]
Michael Brown commented on IMPALA-8175:
---
https://gerrit.cloudera.org/c/12412/
> centos6: tests_minicluster_obj fails with pgrep usage error
> ---
>
> Key: IMPALA-8175
> URL: https://issues.apache.org/jira/browse/IMPALA-8175
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
>Affects Versions: Impala 3.2.0
>Reporter: Michael Brown
>Assignee: Michael Brown
>Priority: Critical
> Labels: broken-build
>
> {noformat}
> F infra/test_stress_infra.py::TestStressInfra::()::tests_minicluster_obj
> infra/test_stress_infra.py:74: in tests_minicluster_obj
> assert expected_pids == len(cluster.impala.for_each_impalad(lambda i:
> i.find_pid()))
> comparison/cluster.py:611: in for_each_impalad
> results = promise.get(maxint)
> /usr/lib64/python2.6/multiprocessing/pool.py:422: in get
> raise self._value
> E ValueError: invalid literal for int() with base 10: "pgrep: invalid
> option -- 'a'\nUsage: pgrep [-flvx] [-d DELIM] [-n|-o] [-P PPIDLIST] [-g
> PGRPLIST] [-s SIDLIST]\n\t[-u EUIDLIST] [-U UIDLIST] [-G GIDLIST] [-t
> TERMLIST] [PATTERN]\n"
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-8175) centos6: tests_minicluster_obj fails with pgrep usage error
Michael Brown created IMPALA-8175:
-
Summary: centos6: tests_minicluster_obj fails with pgrep usage
error
Key: IMPALA-8175
URL: https://issues.apache.org/jira/browse/IMPALA-8175
Project: IMPALA
Issue Type: Bug
Components: Infrastructure
Affects Versions: Impala 3.2.0
Reporter: Michael Brown
Assignee: Michael Brown
{noformat}
F infra/test_stress_infra.py::TestStressInfra::()::tests_minicluster_obj
infra/test_stress_infra.py:74: in tests_minicluster_obj
assert expected_pids == len(cluster.impala.for_each_impalad(lambda i:
i.find_pid()))
comparison/cluster.py:611: in for_each_impalad
results = promise.get(maxint)
/usr/lib64/python2.6/multiprocessing/pool.py:422: in get
raise self._value
E ValueError: invalid literal for int() with base 10: "pgrep: invalid option
-- 'a'\nUsage: pgrep [-flvx] [-d DELIM] [-n|-o] [-P PPIDLIST] [-g PGRPLIST] [-s
SIDLIST]\n\t[-u EUIDLIST] [-U UIDLIST] [-G GIDLIST] [-t TERMLIST] [PATTERN]\n"
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (IMPALA-8128) Build failure: Kudu functional data load - Kudu crashed
[
https://issues.apache.org/jira/browse/IMPALA-8128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Brown resolved IMPALA-8128.
---
Resolution: Duplicate
> Build failure: Kudu functional data load - Kudu crashed
> ---
>
> Key: IMPALA-8128
> URL: https://issues.apache.org/jira/browse/IMPALA-8128
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
>Affects Versions: Impala 3.1.0
>Reporter: Paul Rogers
>Assignee: Lenisha Gandhi
>Priority: Blocker
> Labels: broken-build
> Fix For: Impala 3.1.0
>
>
> ASAN and core builds failed due, it seems, to a Kudu crash when loading data.
> Looks like tpcds, tech and functional all failed.
> {noformat}
> 20:28:01 + msg='Loading functional-query data'
> ...
> 20:31:42 20:31:41 Error executing impala SQL:
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/logs/data_loading/sql/tpch/create-tpch-core-impala-generated-kudu-none-none.sql
> See:
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/logs/data_loading/sql/tpch/create-tpch-core-impala-generated-kudu-none-none.sql.log
> ...
> 20:40:52 FAILED (Took: 12 min 51 sec)
> ...
> 20:40:52 20:40:52 Error executing impala SQL:
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/logs/data_loading/sql/functional/create-functional-query-exhaustive-impala-generated-kudu-none-none.sql
> See:
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/logs/data_loading/sql/functional/create-functional-query-exhaustive-impala-generated-kudu-none-none.sql.log
> ...
> 20:41:16 2019-01-25 20:41:16,863 - archive_core_dumps - INFO - Found binary
> path through GDB:
> /data/jenkins/workspace/impala-asf-master-core-asan/Impala-Toolchain/cdh_compon
> 20:41:17 2019-01-25 20:41:17,043 - archive_core_dumps - WARNING - Failed to
> determine binary because multiple candidate binaries were found and none of
> their paths contained 'latest' to disambiguate:
> 20:41:17 Core:./core.1548476819.29942.kudu-tserver
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (IMPALA-8175) centos6: tests_minicluster_obj fails with pgrep usage error
Michael Brown created IMPALA-8175:
-
Summary: centos6: tests_minicluster_obj fails with pgrep usage
error
Key: IMPALA-8175
URL: https://issues.apache.org/jira/browse/IMPALA-8175
Project: IMPALA
Issue Type: Bug
Components: Infrastructure
Affects Versions: Impala 3.2.0
Reporter: Michael Brown
Assignee: Michael Brown
{noformat}
F infra/test_stress_infra.py::TestStressInfra::()::tests_minicluster_obj
infra/test_stress_infra.py:74: in tests_minicluster_obj
assert expected_pids == len(cluster.impala.for_each_impalad(lambda i:
i.find_pid()))
comparison/cluster.py:611: in for_each_impalad
results = promise.get(maxint)
/usr/lib64/python2.6/multiprocessing/pool.py:422: in get
raise self._value
E ValueError: invalid literal for int() with base 10: "pgrep: invalid option
-- 'a'\nUsage: pgrep [-flvx] [-d DELIM] [-n|-o] [-P PPIDLIST] [-g PGRPLIST] [-s
SIDLIST]\n\t[-u EUIDLIST] [-U UIDLIST] [-G GIDLIST] [-t TERMLIST] [PATTERN]\n"
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-8128) Build failure: Kudu functional data load - Kudu crashed
[
https://issues.apache.org/jira/browse/IMPALA-8128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Brown resolved IMPALA-8128.
---
Resolution: Duplicate
> Build failure: Kudu functional data load - Kudu crashed
> ---
>
> Key: IMPALA-8128
> URL: https://issues.apache.org/jira/browse/IMPALA-8128
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
>Affects Versions: Impala 3.1.0
>Reporter: Paul Rogers
>Assignee: Lenisha Gandhi
>Priority: Blocker
> Labels: broken-build
> Fix For: Impala 3.1.0
>
>
> ASAN and core builds failed due, it seems, to a Kudu crash when loading data.
> Looks like tpcds, tech and functional all failed.
> {noformat}
> 20:28:01 + msg='Loading functional-query data'
> ...
> 20:31:42 20:31:41 Error executing impala SQL:
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/logs/data_loading/sql/tpch/create-tpch-core-impala-generated-kudu-none-none.sql
> See:
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/logs/data_loading/sql/tpch/create-tpch-core-impala-generated-kudu-none-none.sql.log
> ...
> 20:40:52 FAILED (Took: 12 min 51 sec)
> ...
> 20:40:52 20:40:52 Error executing impala SQL:
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/logs/data_loading/sql/functional/create-functional-query-exhaustive-impala-generated-kudu-none-none.sql
> See:
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/logs/data_loading/sql/functional/create-functional-query-exhaustive-impala-generated-kudu-none-none.sql.log
> ...
> 20:41:16 2019-01-25 20:41:16,863 - archive_core_dumps - INFO - Found binary
> path through GDB:
> /data/jenkins/workspace/impala-asf-master-core-asan/Impala-Toolchain/cdh_compon
> 20:41:17 2019-01-25 20:41:17,043 - archive_core_dumps - WARNING - Failed to
> determine binary because multiple candidate binaries were found and none of
> their paths contained 'latest' to disambiguate:
> 20:41:17 Core:./core.1548476819.29942.kudu-tserver
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-8171) stress test doesn't work against minicluster
[
https://issues.apache.org/jira/browse/IMPALA-8171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Brown resolved IMPALA-8171.
---
Resolution: Fixed
Fix Version/s: Impala 3.2.0
> stress test doesn't work against minicluster
>
>
> Key: IMPALA-8171
> URL: https://issues.apache.org/jira/browse/IMPALA-8171
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
>Affects Versions: Impala 3.2.0
>Reporter: Michael Brown
>Assignee: Michael Brown
>Priority: Critical
> Fix For: Impala 3.2.0
>
>
> {noformat}
> $ tests/stress/concurrent_select.py --tpch-db tpch_parquet
> Traceback (most recent call last):
> File "tests/stress/concurrent_select.py", line 2320, in
> main()
> File "tests/stress/concurrent_select.py", line 2159, in main
> if impala.find_stopped_impalads():
> File "/home/mikeb/Impala/tests/comparison/cluster.py", line 557, in
> find_stopped_impalads
> for idx, pid in enumerate(self.for_each_impalad(lambda i: i.find_pid())):
> File "/home/mikeb/Impala/tests/comparison/cluster.py", line 611, in
> for_each_impalad
> results = promise.get(maxint)
> File "/usr/lib/python2.7/multiprocessing/pool.py", line 567, in get
> raise self._value
> ValueError: invalid literal for int() with base 10: '48601\n48624\n'
> {noformat}
> This is due to IMPALA-7999. The refactor causes additional processes to match
> the process list.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-8171) stress test doesn't work against minicluster
[
https://issues.apache.org/jira/browse/IMPALA-8171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Brown resolved IMPALA-8171.
---
Resolution: Fixed
Fix Version/s: Impala 3.2.0
> stress test doesn't work against minicluster
>
>
> Key: IMPALA-8171
> URL: https://issues.apache.org/jira/browse/IMPALA-8171
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
>Affects Versions: Impala 3.2.0
>Reporter: Michael Brown
>Assignee: Michael Brown
>Priority: Critical
> Fix For: Impala 3.2.0
>
>
> {noformat}
> $ tests/stress/concurrent_select.py --tpch-db tpch_parquet
> Traceback (most recent call last):
> File "tests/stress/concurrent_select.py", line 2320, in
> main()
> File "tests/stress/concurrent_select.py", line 2159, in main
> if impala.find_stopped_impalads():
> File "/home/mikeb/Impala/tests/comparison/cluster.py", line 557, in
> find_stopped_impalads
> for idx, pid in enumerate(self.for_each_impalad(lambda i: i.find_pid())):
> File "/home/mikeb/Impala/tests/comparison/cluster.py", line 611, in
> for_each_impalad
> results = promise.get(maxint)
> File "/usr/lib/python2.7/multiprocessing/pool.py", line 567, in get
> raise self._value
> ValueError: invalid literal for int() with base 10: '48601\n48624\n'
> {noformat}
> This is due to IMPALA-7999. The refactor causes additional processes to match
> the process list.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (IMPALA-8169) update some random query generator infra settings
[ https://issues.apache.org/jira/browse/IMPALA-8169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Brown resolved IMPALA-8169. --- Resolution: Fixed Fix Version/s: Impala 3.2.0 > update some random query generator infra settings > - > > Key: IMPALA-8169 > URL: https://issues.apache.org/jira/browse/IMPALA-8169 > Project: IMPALA > Issue Type: Bug > Components: Infrastructure >Affects Versions: Impala 3.2.0 >Reporter: Michael Brown >Assignee: Michael Brown >Priority: Major > Fix For: Impala 3.2.0 > > > I've been running the random query generator in a downstream environment for > a long time and would like to update a few variables and other things that I > think would benefit others. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-8169) update some random query generator infra settings
[ https://issues.apache.org/jira/browse/IMPALA-8169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Brown resolved IMPALA-8169. --- Resolution: Fixed Fix Version/s: Impala 3.2.0 > update some random query generator infra settings > - > > Key: IMPALA-8169 > URL: https://issues.apache.org/jira/browse/IMPALA-8169 > Project: IMPALA > Issue Type: Bug > Components: Infrastructure >Affects Versions: Impala 3.2.0 >Reporter: Michael Brown >Assignee: Michael Brown >Priority: Major > Fix For: Impala 3.2.0 > > > I've been running the random query generator in a downstream environment for > a long time and would like to update a few variables and other things that I > think would benefit others. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-8171) stress test doesn't work against minicluster
[
https://issues.apache.org/jira/browse/IMPALA-8171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16762283#comment-16762283
]
Michael Brown commented on IMPALA-8171:
---
There's an easy one line fix to unblock this but I want to add an infra test as
well.
> stress test doesn't work against minicluster
>
>
> Key: IMPALA-8171
> URL: https://issues.apache.org/jira/browse/IMPALA-8171
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
>Affects Versions: Impala 3.2.0
>Reporter: Michael Brown
>Assignee: Michael Brown
>Priority: Critical
>
> {noformat}
> $ tests/stress/concurrent_select.py --tpch-db tpch_parquet
> Traceback (most recent call last):
> File "tests/stress/concurrent_select.py", line 2320, in
> main()
> File "tests/stress/concurrent_select.py", line 2159, in main
> if impala.find_stopped_impalads():
> File "/home/mikeb/Impala/tests/comparison/cluster.py", line 557, in
> find_stopped_impalads
> for idx, pid in enumerate(self.for_each_impalad(lambda i: i.find_pid())):
> File "/home/mikeb/Impala/tests/comparison/cluster.py", line 611, in
> for_each_impalad
> results = promise.get(maxint)
> File "/usr/lib/python2.7/multiprocessing/pool.py", line 567, in get
> raise self._value
> ValueError: invalid literal for int() with base 10: '48601\n48624\n'
> {noformat}
> This is due to IMPALA-7999. The refactor causes additional processes to match
> the process list.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-8171) stress test doesn't work against minicluster
[
https://issues.apache.org/jira/browse/IMPALA-8171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16762282#comment-16762282
]
Michael Brown commented on IMPALA-8171:
---
See all the {{bin/start-daemon.sh}} processes. In
https://gerrit.cloudera.org/#/c/12271/ start-*.sh used {{exec}}. This would
have caused the exec caller to exit.
{noformat}
48608 pts/0S 0:00 /bin/bash /home/mikeb/Impala/bin/start-daemon.sh
/home/mikeb/Impala/be/build/latest/service/impalad -kudu_client_rpc_timeout_ms
0 -kudu_master_hosts localhost -mem_limit=12884901888 -logbufsecs=5 -v=1
-max_log_files=10 -log_filename=impalad_node2
-log_dir=/home/mikeb/Impala/logs/cluster -beeswax_port=21002 -hs2_port=21052
-be_port=22002 -krpc_port=27002 -state_store_subscriber_port=23002
-webserver_port=25002
48626 pts/0Sl 0:09 \_
/home/mikeb/Impala/be/build/latest/service/impalad -kudu_client_rpc_timeout_ms
0 -kudu_master_hosts localhost -mem_limit=12884901888 -logbufsecs=5 -v=1
-max_log_files=10 -log_filename=impalad_node2
-log_dir=/home/mikeb/Impala/logs/cluster -beeswax_port=21002 -hs2_port=21052
-be_port=22002 -krpc_port=27002 -state_store_subscriber_port=23002
-webserver_port=25002
48604 pts/0S 0:00 /bin/bash /home/mikeb/Impala/bin/start-daemon.sh
/home/mikeb/Impala/be/build/latest/service/impalad -kudu_client_rpc_timeout_ms
0 -kudu_master_hosts localhost -mem_limit=12884901888 -logbufsecs=5 -v=1
-max_log_files=10 -log_filename=impalad_node1
-log_dir=/home/mikeb/Impala/logs/cluster -beeswax_port=21001 -hs2_port=21051
-be_port=22001 -krpc_port=27001 -state_store_subscriber_port=23001
-webserver_port=25001
48625 pts/0Sl 0:10 \_
/home/mikeb/Impala/be/build/latest/service/impalad -kudu_client_rpc_timeout_ms
0 -kudu_master_hosts localhost -mem_limit=12884901888 -logbufsecs=5 -v=1
-max_log_files=10 -log_filename=impalad_node1
-log_dir=/home/mikeb/Impala/logs/cluster -beeswax_port=21001 -hs2_port=21051
-be_port=22001 -krpc_port=27001 -state_store_subscriber_port=23001
-webserver_port=25001
48601 pts/0S 0:00 /bin/bash /home/mikeb/Impala/bin/start-daemon.sh
/home/mikeb/Impala/be/build/latest/service/impalad -kudu_client_rpc_timeout_ms
0 -kudu_master_hosts localhost -mem_limit=12884901888 -logbufsecs=5 -v=1
-max_log_files=10 -log_filename=impalad
-log_dir=/home/mikeb/Impala/logs/cluster -beeswax_port=21000 -hs2_port=21050
-be_port=22000 -krpc_port=27000 -state_store_subscriber_port=23000
-webserver_port=25000
48624 pts/0Sl 0:10 \_
/home/mikeb/Impala/be/build/latest/service/impalad -kudu_client_rpc_timeout_ms
0 -kudu_master_hosts localhost -mem_limit=12884901888 -logbufsecs=5 -v=1
-max_log_files=10 -log_filename=impalad
-log_dir=/home/mikeb/Impala/logs/cluster -beeswax_port=21000 -hs2_port=21050
-be_port=22000 -krpc_port=27000 -state_store_subscriber_port=23000
-webserver_port=25000
48530 pts/0S 0:00 /bin/bash /home/mikeb/Impala/bin/start-daemon.sh
/home/mikeb/Impala/be/build/latest/service/catalogd -logbufsecs=5 -v=1
-max_log_files=10 -log_filename=catalogd
-log_dir=/home/mikeb/Impala/logs/cluster -kudu_master_hosts localhost
48537 pts/0Sl 0:06 \_
/home/mikeb/Impala/be/build/latest/service/catalogd -logbufsecs=5 -v=1
-max_log_files=10 -log_filename=catalogd
-log_dir=/home/mikeb/Impala/logs/cluster -kudu_master_hosts localhost
48474 pts/0S 0:00 /bin/bash /home/mikeb/Impala/bin/start-daemon.sh
/home/mikeb/Impala/be/build/latest/service/statestored -logbufsecs=5 -v=1
-max_log_files=10 -log_filename=statestored
-log_dir=/home/mikeb/Impala/logs/cluster
48481 pts/0Sl 0:02 \_
/home/mikeb/Impala/be/build/latest/service/statestored -logbufsecs=5 -v=1
-max_log_files=10 -log_filename=statestored
-log_dir=/home/mikeb/Impala/logs/cluster
{noformat}
> stress test doesn't work against minicluster
>
>
> Key: IMPALA-8171
> URL: https://issues.apache.org/jira/browse/IMPALA-8171
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
>Affects Versions: Impala 3.2.0
>Reporter: Michael Brown
>Assignee: Michael Brown
>Priority: Critical
>
> {noformat}
> $ tests/stress/concurrent_select.py --tpch-db tpch_parquet
> Traceback (most recent call last):
> File "tests/stress/concurrent_select.py", line 2320, in
> main()
> File "tests/stress/concurrent_select.py", line 2159, in main
> if impala.find_stopped_impalads():
> File "/home/mikeb/Impala/tests/comparison/cluster.py", line 557, in
> find_stopped_impalads
> for idx, pid in enumerate(self.for_each_impalad(lambda i: i.find_pid())):
> File "/home/mikeb/Impala/tests/comparison/cluster.py", line 611, in
> for_each_impalad
> results = promise.get(maxint)
> File "/usr/lib/python2.7/multiprocessing/pool.py", line 567, in get
>
[jira] [Created] (IMPALA-8171) stress test doesn't work against minicluster
Michael Brown created IMPALA-8171:
-
Summary: stress test doesn't work against minicluster
Key: IMPALA-8171
URL: https://issues.apache.org/jira/browse/IMPALA-8171
Project: IMPALA
Issue Type: Bug
Components: Infrastructure
Affects Versions: Impala 3.2.0
Reporter: Michael Brown
Assignee: Michael Brown
{noformat}
$ tests/stress/concurrent_select.py --tpch-db tpch_parquet
Traceback (most recent call last):
File "tests/stress/concurrent_select.py", line 2320, in
main()
File "tests/stress/concurrent_select.py", line 2159, in main
if impala.find_stopped_impalads():
File "/home/mikeb/Impala/tests/comparison/cluster.py", line 557, in
find_stopped_impalads
for idx, pid in enumerate(self.for_each_impalad(lambda i: i.find_pid())):
File "/home/mikeb/Impala/tests/comparison/cluster.py", line 611, in
for_each_impalad
results = promise.get(maxint)
File "/usr/lib/python2.7/multiprocessing/pool.py", line 567, in get
raise self._value
ValueError: invalid literal for int() with base 10: '48601\n48624\n'
{noformat}
This is due to IMPALA-7999. The refactor causes additional processes to match
the process list.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (IMPALA-8171) stress test doesn't work against minicluster
Michael Brown created IMPALA-8171:
-
Summary: stress test doesn't work against minicluster
Key: IMPALA-8171
URL: https://issues.apache.org/jira/browse/IMPALA-8171
Project: IMPALA
Issue Type: Bug
Components: Infrastructure
Affects Versions: Impala 3.2.0
Reporter: Michael Brown
Assignee: Michael Brown
{noformat}
$ tests/stress/concurrent_select.py --tpch-db tpch_parquet
Traceback (most recent call last):
File "tests/stress/concurrent_select.py", line 2320, in
main()
File "tests/stress/concurrent_select.py", line 2159, in main
if impala.find_stopped_impalads():
File "/home/mikeb/Impala/tests/comparison/cluster.py", line 557, in
find_stopped_impalads
for idx, pid in enumerate(self.for_each_impalad(lambda i: i.find_pid())):
File "/home/mikeb/Impala/tests/comparison/cluster.py", line 611, in
for_each_impalad
results = promise.get(maxint)
File "/usr/lib/python2.7/multiprocessing/pool.py", line 567, in get
raise self._value
ValueError: invalid literal for int() with base 10: '48601\n48624\n'
{noformat}
This is due to IMPALA-7999. The refactor causes additional processes to match
the process list.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-8064) test_min_max_filters is flaky
[
https://issues.apache.org/jira/browse/IMPALA-8064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16762216#comment-16762216
]
Michael Brown commented on IMPALA-8064:
---
[~poojanilangekar] If you can get access to the profiles, see whether runtime
filters were waited on for at least 100,000 ms. If you look at my comments
above, in the profiles I looked at, they didn't seem to be waiting as long as
directed. I can help take a look too if needed.
> test_min_max_filters is flaky
> --
>
> Key: IMPALA-8064
> URL: https://issues.apache.org/jira/browse/IMPALA-8064
> Project: IMPALA
> Issue Type: Bug
>Reporter: Pooja Nilangekar
>Assignee: Pooja Nilangekar
>Priority: Blocker
> Labels: broken-build, flaky-test
> Fix For: Impala 3.2.0
>
> Attachments: profile.txt
>
>
> The following configuration of the test_min_max_filters:
> {code:java}
> query_test.test_runtime_filters.TestMinMaxFilters.test_min_max_filters[protocol:
> beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0,
> 'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
> 'abort_on_error': 1, 'debug_action': None, 'exec_single_node_rows_threshold':
> 0} | table_format: kudu/none]{code}
> It produces a higher aggregation of sum over the proberows than expected:
> {code:java}
> query_test/test_runtime_filters.py:113: in test_min_max_filters
> self.run_test_case('QueryTest/min_max_filters', vector)
> common/impala_test_suite.py:518: in run_test_case
> update_section=pytest.config.option.update_results)
> common/test_result_verifier.py:612: in verify_runtime_profile %
> (function, field, expected_value, actual_value, actual))
> E AssertionError: Aggregation of SUM over ProbeRows did not match expected
> results.
> E EXPECTED VALUE: E 619
> EACTUAL VALUE: E 652
> {code}
> This test was introduced in the patch for IMPALA-6533. The failure occurred
> during an ASAN build.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-8091) minicluster kudu failure: Cannot initialize clock: failed to wait for clock sync
[
https://issues.apache.org/jira/browse/IMPALA-8091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Brown resolved IMPALA-8091.
---
Resolution: Fixed
> minicluster kudu failure: Cannot initialize clock: failed to wait for clock
> sync
>
>
> Key: IMPALA-8091
> URL: https://issues.apache.org/jira/browse/IMPALA-8091
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
>Affects Versions: Impala 3.2.0
>Reporter: David Knupp
>Assignee: Michael Brown
>Priority: Blocker
> Labels: broken-build
> Fix For: Impala 3.2.0
>
>
> *Console log*:
> {noformat}
> 19:10:04 19:10:04 Error executing impala SQL:
> /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/logs/data_loading/sql/functional/create-functional-query-exhaustive-impala-generated-kudu-none-none.sql
> See:
> /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/logs/data_loading/sql/functional/create-functional-query-exhaustive-impala-generated-kudu-none-none.sql.log
> 19:10:04 Background task Loading functional-query data (pid 4831) failed.
> 19:10:04 Background task Loading TPC-H data (pid 4832) failed.
> 19:10:04 ERROR in
> /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/testdata/bin/create-load-data.sh
> at line 85: CM_HOST=${2-}
> [...]
> 19:10:05 2019-01-16 19:10:05,461 - archive_core_dumps - INFO - Found core
> files: ['./core.1547693849.30799.kudu-master',
> './core.1547693849.30783.kudu-tserver',
> './core.1547693849.30767.kudu-tserver',
> './core.1547693849.30808.kudu-tserver']
> 19:10:05 2019-01-16 19:10:05,649 - archive_core_dumps - INFO - [New LWP 30799]
> 19:10:05 [New LWP 30834]
> 19:10:05 [New LWP 30835]
> 19:10:05 [New LWP 30838]
> 19:10:05 [New LWP 30837]
> 19:10:05 [New LWP 30836]
> 19:10:05 Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
> 19:10:05 Program terminated with signal SIGABRT, Aborted.
> 19:10:05 #0 0x7f9a2cc611f7 in ?? ()
> 19:10:05
> 19:10:05 2019-01-16 19:10:05,650 - archive_core_dumps - INFO - Found binary
> path through GDB:
> /data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c
> 19:10:05 2019-01-16 19:10:05,893 - archive_core_dumps - WARNING - Failed to
> determine binary because multiple candidate binaries were found and none of
> their paths contained 'latest' to disambiguate:
> 19:10:05 Core:./core.1547693849.30799.kudu-master
> 19:10:05
> Binaries:['./testdata/cluster/node_templates/common/etc/init.d/kudu-master',
> './testdata/cluster/cdh6/node-1/etc/init.d/kudu-master']
> 19:10:05
> 19:10:05 2019-01-16 19:10:05,917 - archive_core_dumps - INFO - [New LWP 30783]
> 19:10:05 [New LWP 30810]
> 19:10:05 [New LWP 30812]
> 19:10:05 [New LWP 30811]
> 19:10:05 [New LWP 30824]
> 19:10:05 [New LWP 30820]
> 19:10:05 Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
> 19:10:05 Program terminated with signal SIGABRT, Aborted.
> 19:10:05 #0 0x7f0b81fb11f7 in ?? ()
> {noformat}
> *Backtraces*:
> {noformat}
> CORE: ./core.1547693849.30799.kudu-master
> BINARY: ./be/build/latest/service/impalad
> Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
> Program terminated with signal SIGABRT, Aborted.
> #0 0x7f9a2cc611f7 in ?? ()
> #0 0x7f9a2cc611f7 in ?? ()
> #1 0x7f9a2cc628e8 in ?? ()
> #2 0x0020 in ?? ()
> #3 0x in ?? ()
> CORE: ./core.1547693849.30783.kudu-tserver
> BINARY: ./be/build/latest/service/impalad
> Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
> Program terminated with signal SIGABRT, Aborted.
> #0 0x7f0b81fb11f7 in ?? ()
> #0 0x7f0b81fb11f7 in ?? ()
> #1 0x7f0b81fb28e8 in ?? ()
> #2 0x0020 in ?? ()
> #3 0x in ?? ()
> CORE: ./core.1547693849.30767.kudu-tserver
> BINARY: ./be/build/latest/service/impalad
> Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
> Program terminated with signal SIGABRT, Aborted.
> #0 0x7f60b1e2f1f7 in ?? ()
> #0 0x7f60b1e2f1f7 in ?? ()
> #1 0x7f60b1e308e8 in ?? ()
> #2 0x0020 in ?? ()
> #3 0x in ?? ()
> CORE: ./core.1547693849.30808.kudu-tserver
> BINARY: ./be/build/latest/service/impalad
> Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
> Program terminated with signal SIGABRT, Aborted.
> #0 0x7fa3cb5591f7 in ?? ()
> #0 0x7fa3cb5591f7 in ?? ()
> #1 0x7fa3cb55a8e8 in ?? ()
> #2
[jira] [Resolved] (IMPALA-8091) minicluster kudu failure: Cannot initialize clock: failed to wait for clock sync
[
https://issues.apache.org/jira/browse/IMPALA-8091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Brown resolved IMPALA-8091.
---
Resolution: Fixed
> minicluster kudu failure: Cannot initialize clock: failed to wait for clock
> sync
>
>
> Key: IMPALA-8091
> URL: https://issues.apache.org/jira/browse/IMPALA-8091
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
>Affects Versions: Impala 3.2.0
>Reporter: David Knupp
>Assignee: Michael Brown
>Priority: Blocker
> Labels: broken-build
> Fix For: Impala 3.2.0
>
>
> *Console log*:
> {noformat}
> 19:10:04 19:10:04 Error executing impala SQL:
> /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/logs/data_loading/sql/functional/create-functional-query-exhaustive-impala-generated-kudu-none-none.sql
> See:
> /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/logs/data_loading/sql/functional/create-functional-query-exhaustive-impala-generated-kudu-none-none.sql.log
> 19:10:04 Background task Loading functional-query data (pid 4831) failed.
> 19:10:04 Background task Loading TPC-H data (pid 4832) failed.
> 19:10:04 ERROR in
> /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/testdata/bin/create-load-data.sh
> at line 85: CM_HOST=${2-}
> [...]
> 19:10:05 2019-01-16 19:10:05,461 - archive_core_dumps - INFO - Found core
> files: ['./core.1547693849.30799.kudu-master',
> './core.1547693849.30783.kudu-tserver',
> './core.1547693849.30767.kudu-tserver',
> './core.1547693849.30808.kudu-tserver']
> 19:10:05 2019-01-16 19:10:05,649 - archive_core_dumps - INFO - [New LWP 30799]
> 19:10:05 [New LWP 30834]
> 19:10:05 [New LWP 30835]
> 19:10:05 [New LWP 30838]
> 19:10:05 [New LWP 30837]
> 19:10:05 [New LWP 30836]
> 19:10:05 Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
> 19:10:05 Program terminated with signal SIGABRT, Aborted.
> 19:10:05 #0 0x7f9a2cc611f7 in ?? ()
> 19:10:05
> 19:10:05 2019-01-16 19:10:05,650 - archive_core_dumps - INFO - Found binary
> path through GDB:
> /data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c
> 19:10:05 2019-01-16 19:10:05,893 - archive_core_dumps - WARNING - Failed to
> determine binary because multiple candidate binaries were found and none of
> their paths contained 'latest' to disambiguate:
> 19:10:05 Core:./core.1547693849.30799.kudu-master
> 19:10:05
> Binaries:['./testdata/cluster/node_templates/common/etc/init.d/kudu-master',
> './testdata/cluster/cdh6/node-1/etc/init.d/kudu-master']
> 19:10:05
> 19:10:05 2019-01-16 19:10:05,917 - archive_core_dumps - INFO - [New LWP 30783]
> 19:10:05 [New LWP 30810]
> 19:10:05 [New LWP 30812]
> 19:10:05 [New LWP 30811]
> 19:10:05 [New LWP 30824]
> 19:10:05 [New LWP 30820]
> 19:10:05 Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
> 19:10:05 Program terminated with signal SIGABRT, Aborted.
> 19:10:05 #0 0x7f0b81fb11f7 in ?? ()
> {noformat}
> *Backtraces*:
> {noformat}
> CORE: ./core.1547693849.30799.kudu-master
> BINARY: ./be/build/latest/service/impalad
> Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
> Program terminated with signal SIGABRT, Aborted.
> #0 0x7f9a2cc611f7 in ?? ()
> #0 0x7f9a2cc611f7 in ?? ()
> #1 0x7f9a2cc628e8 in ?? ()
> #2 0x0020 in ?? ()
> #3 0x in ?? ()
> CORE: ./core.1547693849.30783.kudu-tserver
> BINARY: ./be/build/latest/service/impalad
> Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
> Program terminated with signal SIGABRT, Aborted.
> #0 0x7f0b81fb11f7 in ?? ()
> #0 0x7f0b81fb11f7 in ?? ()
> #1 0x7f0b81fb28e8 in ?? ()
> #2 0x0020 in ?? ()
> #3 0x in ?? ()
> CORE: ./core.1547693849.30767.kudu-tserver
> BINARY: ./be/build/latest/service/impalad
> Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
> Program terminated with signal SIGABRT, Aborted.
> #0 0x7f60b1e2f1f7 in ?? ()
> #0 0x7f60b1e2f1f7 in ?? ()
> #1 0x7f60b1e308e8 in ?? ()
> #2 0x0020 in ?? ()
> #3 0x in ?? ()
> CORE: ./core.1547693849.30808.kudu-tserver
> BINARY: ./be/build/latest/service/impalad
> Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
> Program terminated with signal SIGABRT, Aborted.
> #0 0x7fa3cb5591f7 in ?? ()
> #0 0x7fa3cb5591f7 in ?? ()
> #1 0x7fa3cb55a8e8 in ?? ()
> #2
[jira] [Created] (IMPALA-8169) update some random query generator infra settings
Michael Brown created IMPALA-8169: - Summary: update some random query generator infra settings Key: IMPALA-8169 URL: https://issues.apache.org/jira/browse/IMPALA-8169 Project: IMPALA Issue Type: Bug Components: Infrastructure Affects Versions: Impala 3.2.0 Reporter: Michael Brown Assignee: Michael Brown I've been running the random query generator in a downstream environment for a long time and would like to update a few variables and other things that I think would benefit others. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-8169) update some random query generator infra settings
Michael Brown created IMPALA-8169: - Summary: update some random query generator infra settings Key: IMPALA-8169 URL: https://issues.apache.org/jira/browse/IMPALA-8169 Project: IMPALA Issue Type: Bug Components: Infrastructure Affects Versions: Impala 3.2.0 Reporter: Michael Brown Assignee: Michael Brown I've been running the random query generator in a downstream environment for a long time and would like to update a few variables and other things that I think would benefit others. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-8113) test_aggregation and test_avro_primitive_in_list fail in S3
[
https://issues.apache.org/jira/browse/IMPALA-8113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Brown resolved IMPALA-8113.
---
Resolution: Fixed
Fix Version/s: Impala 3.2.0
> test_aggregation and test_avro_primitive_in_list fail in S3
> ---
>
> Key: IMPALA-8113
> URL: https://issues.apache.org/jira/browse/IMPALA-8113
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
>Affects Versions: Impala 3.2.0
>Reporter: Michael Brown
>Assignee: Michael Brown
>Priority: Critical
> Fix For: Impala 3.2.0
>
>
> Likely more victims of our infra in S3.
> {noformat}
> query_test/test_aggregation.py:138: in test_aggregation
> result = self.execute_query(query, vector.get_value('exec_option'))
> common/impala_test_suite.py:597: in wrapper
> return function(*args, **kwargs)
> common/impala_test_suite.py:628: in execute_query
> return self.__execute_query(self.client, query, query_options)
> common/impala_test_suite.py:695: in __execute_query
> return impalad_client.execute(query, user=user)
> common/impala_connection.py:174: in execute
> return self.__beeswax_client.execute(sql_stmt, user=user)
> beeswax/impala_beeswax.py:182: in execute
> handle = self.__execute_query(query_string.strip(), user=user)
> beeswax/impala_beeswax.py:359: in __execute_query
> self.wait_for_finished(handle)
> beeswax/impala_beeswax.py:380: in wait_for_finished
> raise ImpalaBeeswaxException("Query aborted:" + error_log, None)
> E ImpalaBeeswaxException: ImpalaBeeswaxException:
> EQuery aborted:Disk I/O error: Error reading from HDFS file:
> s3a://impala-test-uswest2-1/test-warehouse/alltypesagg_parquet/year=2010/month=1/day=8/5642b2da93dae1ad-494132e5_592013737_data.0.parq
> E Error(255): Unknown error 255
> E Root cause: SdkClientException: Data read has a different length than the
> expected: dataLength=0; expectedLength=45494; includeSkipped=true;
> in.getClass()=class com.amazonaws.services.s3.AmazonS3Client$2;
> markedSupported=false; marked=0; resetSinceLastMarked=false; markCount=0;
> resetCount=0
> {noformat}
> {noformat}
> query_test/test_nested_types.py:263: in test_avro_primitive_in_list
> "AvroPrimitiveInList.parquet", vector)
> query_test/test_nested_types.py:287: in __test_primitive_in_list
> result = self.execute_query("select item from %s.col1" % full_name, qopts)
> common/impala_test_suite.py:597: in wrapper
> return function(*args, **kwargs)
> common/impala_test_suite.py:628: in execute_query
> return self.__execute_query(self.client, query, query_options)
> common/impala_test_suite.py:695: in __execute_query
> return impalad_client.execute(query, user=user)
> common/impala_connection.py:174: in execute
> return self.__beeswax_client.execute(sql_stmt, user=user)
> beeswax/impala_beeswax.py:182: in execute
> handle = self.__execute_query(query_string.strip(), user=user)
> beeswax/impala_beeswax.py:359: in __execute_query
> self.wait_for_finished(handle)
> beeswax/impala_beeswax.py:380: in wait_for_finished
> raise ImpalaBeeswaxException("Query aborted:" + error_log, None)
> E ImpalaBeeswaxException: ImpalaBeeswaxException:
> EQuery aborted:Disk I/O error: Failed to open HDFS file
> s3a://impala-test-uswest2-1/test-warehouse/test_avro_primitive_in_list_38f182c4.db/AvroPrimitiveInList/AvroPrimitiveInList.parquet
> E Error(2): No such file or directory
> E Root cause: FileNotFoundException: No such file or directory:
> s3a://impala-test-uswest2-1/test-warehouse/test_avro_primitive_in_list_38f182c4.db/AvroPrimitiveInList/AvroPrimitiveInList.parquet
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-8113) test_aggregation and test_avro_primitive_in_list fail in S3
[
https://issues.apache.org/jira/browse/IMPALA-8113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Brown resolved IMPALA-8113.
---
Resolution: Fixed
Fix Version/s: Impala 3.2.0
> test_aggregation and test_avro_primitive_in_list fail in S3
> ---
>
> Key: IMPALA-8113
> URL: https://issues.apache.org/jira/browse/IMPALA-8113
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
>Affects Versions: Impala 3.2.0
>Reporter: Michael Brown
>Assignee: Michael Brown
>Priority: Critical
> Fix For: Impala 3.2.0
>
>
> Likely more victims of our infra in S3.
> {noformat}
> query_test/test_aggregation.py:138: in test_aggregation
> result = self.execute_query(query, vector.get_value('exec_option'))
> common/impala_test_suite.py:597: in wrapper
> return function(*args, **kwargs)
> common/impala_test_suite.py:628: in execute_query
> return self.__execute_query(self.client, query, query_options)
> common/impala_test_suite.py:695: in __execute_query
> return impalad_client.execute(query, user=user)
> common/impala_connection.py:174: in execute
> return self.__beeswax_client.execute(sql_stmt, user=user)
> beeswax/impala_beeswax.py:182: in execute
> handle = self.__execute_query(query_string.strip(), user=user)
> beeswax/impala_beeswax.py:359: in __execute_query
> self.wait_for_finished(handle)
> beeswax/impala_beeswax.py:380: in wait_for_finished
> raise ImpalaBeeswaxException("Query aborted:" + error_log, None)
> E ImpalaBeeswaxException: ImpalaBeeswaxException:
> EQuery aborted:Disk I/O error: Error reading from HDFS file:
> s3a://impala-test-uswest2-1/test-warehouse/alltypesagg_parquet/year=2010/month=1/day=8/5642b2da93dae1ad-494132e5_592013737_data.0.parq
> E Error(255): Unknown error 255
> E Root cause: SdkClientException: Data read has a different length than the
> expected: dataLength=0; expectedLength=45494; includeSkipped=true;
> in.getClass()=class com.amazonaws.services.s3.AmazonS3Client$2;
> markedSupported=false; marked=0; resetSinceLastMarked=false; markCount=0;
> resetCount=0
> {noformat}
> {noformat}
> query_test/test_nested_types.py:263: in test_avro_primitive_in_list
> "AvroPrimitiveInList.parquet", vector)
> query_test/test_nested_types.py:287: in __test_primitive_in_list
> result = self.execute_query("select item from %s.col1" % full_name, qopts)
> common/impala_test_suite.py:597: in wrapper
> return function(*args, **kwargs)
> common/impala_test_suite.py:628: in execute_query
> return self.__execute_query(self.client, query, query_options)
> common/impala_test_suite.py:695: in __execute_query
> return impalad_client.execute(query, user=user)
> common/impala_connection.py:174: in execute
> return self.__beeswax_client.execute(sql_stmt, user=user)
> beeswax/impala_beeswax.py:182: in execute
> handle = self.__execute_query(query_string.strip(), user=user)
> beeswax/impala_beeswax.py:359: in __execute_query
> self.wait_for_finished(handle)
> beeswax/impala_beeswax.py:380: in wait_for_finished
> raise ImpalaBeeswaxException("Query aborted:" + error_log, None)
> E ImpalaBeeswaxException: ImpalaBeeswaxException:
> EQuery aborted:Disk I/O error: Failed to open HDFS file
> s3a://impala-test-uswest2-1/test-warehouse/test_avro_primitive_in_list_38f182c4.db/AvroPrimitiveInList/AvroPrimitiveInList.parquet
> E Error(2): No such file or directory
> E Root cause: FileNotFoundException: No such file or directory:
> s3a://impala-test-uswest2-1/test-warehouse/test_avro_primitive_in_list_38f182c4.db/AvroPrimitiveInList/AvroPrimitiveInList.parquet
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IMPALA-8115) some jenkins workers slow to do real work due to dpkg lock conflicts
[
https://issues.apache.org/jira/browse/IMPALA-8115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751703#comment-16751703
]
Michael Brown commented on IMPALA-8115:
---
I noticed this because I posted https://gerrit.cloudera.org/#/c/12272/ at 2:42
PT and at 3:45 PT there is still no update that the pre-review job completed.
https://jenkins.impala.io/job/gerrit-code-review-checks/1879/console shows a
slow timeline:
{noformat}
Triggered by Gerrit: http://gerrit.cloudera.org:8080/12272
Running in Durability level: MAX_SURVIVABILITY
[Pipeline] timestamps
[Pipeline] {
[Pipeline] ansiColor
[Pipeline] {
[Pipeline] timeout
22:42:09 Timeout set to expire in 10 hr
[Pipeline] {
[Pipeline] parallel
[Pipeline] { (Branch: Tidy)
[Pipeline] { (Branch: BuildOnly)
[Pipeline] { (Branch: Python26Compatibility)
[Pipeline] { (Branch: Rat)
[Pipeline] build (Building clang-tidy-ub1604)
22:42:09 Scheduling project: clang-tidy-ub1604
[Pipeline] build (Building ubuntu-16.04-build-only)
22:42:09 Scheduling project: ubuntu-16.04-build-only
[Pipeline] build (Building python26-incompatibility-check)
22:42:09 Scheduling project: python26-incompatibility-check
[Pipeline] build (Building rat-check-ub1604)
22:42:09 Scheduling project: rat-check-ub1604
22:44:22 Starting building: python26-incompatibility-check #1548
[Pipeline] }
23:35:15 Starting building: ubuntu-16.04-build-only #5027
23:36:34 Starting building: rat-check-ub1604 #5404
[Pipeline] }
23:37:14 Starting building: clang-tidy-ub1604 #4922
{noformat}
It's taken nearly an hour to start rat-check-ub1604 #5404 and clang-tidy-ub1604
#4922.
My 15 minutes math doesn't account for all of this. I'm not yet sure where else
we could accelerate/improve the process.
> some jenkins workers slow to do real work due to dpkg lock conflicts
>
>
> Key: IMPALA-8115
> URL: https://issues.apache.org/jira/browse/IMPALA-8115
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
>Reporter: Michael Brown
>Priority: Major
>
> A Jenkins worker for label {{ubuntu-16.04}} took about 15 minutes to start
> doing real work. I noticed that it was retrying {{apt-get update}}:
> {noformat}
> ++ sudo apt-get --yes install openjdk-8-jdk
> E: Could not get lock /var/lib/dpkg/lock - open (11: Resource temporarily
> unavailable)
> E: Unable to lock the administration directory (/var/lib/dpkg/), is another
> process using it?
> ++ date
> Thu Jan 24 23:37:33 UTC 2019
> ++ sudo apt-get update
> ++ sleep 10
> ++ sudo apt-get --yes install openjdk-8-jdk
> [etc]
> {noformat}
> I ssh'd into a host and saw that, yes, something else was holding onto the
> dpkg log (confirmed with lsof and not pasted here. dpkg process PID 11459 was
> the culprit)
> {noformat}
> root 1750 0.0 0.0 4508 1664 ?Ss 23:21 0:00 /bin/sh
> /usr/lib/apt/apt.systemd.daily
> root 1804 12.3 0.1 141076 80452 ?S23:22 1:24 \_
> /usr/bin/python3 /usr/bin/unattended-upgrade
> root 3263 0.0 0.1 140960 72896 ?S23:23 0:00 \_
> /usr/bin/python3 /usr/bin/unattended-upgrade
> root 11459 0.6 0.0 45920 25184 pts/1Ss+ 23:24 0:03 \_
> /usr/bin/dpkg --status-fd 10 --unpack --auto-deconfigure
> /var/cache/apt/archives/tzdata_2018i-0ubuntu0.16.04_all.deb
> /var/cache/apt/archives/distro-info-data_0.28ubuntu0.9_all.deb
> /var/cache/apt/archives/file_1%3a5.25-2ubuntu1.1_amd64.deb
> /var/cache/apt/archives/libmagic1_1%3a5.25-2ubuntu1.1_amd64.deb
> /var/cache/apt/archives/libisc-export160_1%3a9.10.3.dfsg.P4-8ubuntu1.11_amd64.deb
>
> /var/cache/apt/archives/libdns-export162_1%3a9.10.3.dfsg.P4-8ubuntu1.11_amd64.deb
> /var/cache/apt/archives/isc-dhcp-client_4.3.3-5ubuntu12.9_amd64.deb
> /var/cache/apt/archives/isc-dhcp-common_4.3.3-5ubuntu12.9_amd64.deb
> /var/cache/apt/archives/libidn11_1.32-3ubuntu1.2_amd64.deb
> /var/cache/apt/archives/libpng12-0_1.2.54-1ubuntu1.1_amd64.deb
> /var/cache/apt/archives/libtasn1-6_4.7-3ubuntu0.16.04.3_amd64.deb
> /var/cache/apt/archives/libapparmor-perl_2.10.95-0ubuntu2.10_amd64.deb
> /var/cache/apt/archives/apparmor_2.10.95-0ubuntu2.10_amd64.deb
> /var/cache/apt/archives/curl_7.47.0-1ubuntu2.11_amd64.deb
> /var/cache/apt/archives/libgssapi-krb5-2_1.13.2+dfsg-5ubuntu2.1_amd64.deb
> /var/cache/apt/archives/libkrb5-3_1.13.2+dfsg-5ubuntu2.1_amd64.deb
> /var/cache/apt/archives/libkrb5support0_1.13.2+dfsg-5ubuntu2.1_amd64.deb
> /var/cache/apt/archives/libk5crypto3_1.13.2+dfsg-5ubuntu2.1_amd64.deb
> /var/cache/apt/archives/libcurl3-gnutls_7.47.0-1ubuntu2.11_amd64.deb
> /var/cache/apt/archives/apt-transport-https_1.2.29ubuntu0.1_amd64.deb
> /var/cache/apt/archives/libicu55_55.1-7ubuntu0.4_amd64.deb
> /var/cache/apt/archives/libxml2_2.9.3+dfsg1-1ubuntu0.6_amd64.deb
>
[jira] [Updated] (IMPALA-8115) some jenkins workers slow to do real work due to dpkg lock conflicts
[
https://issues.apache.org/jira/browse/IMPALA-8115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Brown updated IMPALA-8115:
--
Summary: some jenkins workers slow to do real work due to dpkg lock
conflicts (was: some jenkins workers slow to spawn to to dpkg lock conflicts)
> some jenkins workers slow to do real work due to dpkg lock conflicts
>
>
> Key: IMPALA-8115
> URL: https://issues.apache.org/jira/browse/IMPALA-8115
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
>Reporter: Michael Brown
>Priority: Major
>
> A Jenkins worker for label {{ubuntu-16.04}} took about 15 minutes to start
> doing real work. I noticed that it was retrying {{apt-get update}}:
> {noformat}
> ++ sudo apt-get --yes install openjdk-8-jdk
> E: Could not get lock /var/lib/dpkg/lock - open (11: Resource temporarily
> unavailable)
> E: Unable to lock the administration directory (/var/lib/dpkg/), is another
> process using it?
> ++ date
> Thu Jan 24 23:37:33 UTC 2019
> ++ sudo apt-get update
> ++ sleep 10
> ++ sudo apt-get --yes install openjdk-8-jdk
> [etc]
> {noformat}
> I ssh'd into a host and saw that, yes, something else was holding onto the
> dpkg log (confirmed with lsof and not pasted here. dpkg process PID 11459 was
> the culprit)
> {noformat}
> root 1750 0.0 0.0 4508 1664 ?Ss 23:21 0:00 /bin/sh
> /usr/lib/apt/apt.systemd.daily
> root 1804 12.3 0.1 141076 80452 ?S23:22 1:24 \_
> /usr/bin/python3 /usr/bin/unattended-upgrade
> root 3263 0.0 0.1 140960 72896 ?S23:23 0:00 \_
> /usr/bin/python3 /usr/bin/unattended-upgrade
> root 11459 0.6 0.0 45920 25184 pts/1Ss+ 23:24 0:03 \_
> /usr/bin/dpkg --status-fd 10 --unpack --auto-deconfigure
> /var/cache/apt/archives/tzdata_2018i-0ubuntu0.16.04_all.deb
> /var/cache/apt/archives/distro-info-data_0.28ubuntu0.9_all.deb
> /var/cache/apt/archives/file_1%3a5.25-2ubuntu1.1_amd64.deb
> /var/cache/apt/archives/libmagic1_1%3a5.25-2ubuntu1.1_amd64.deb
> /var/cache/apt/archives/libisc-export160_1%3a9.10.3.dfsg.P4-8ubuntu1.11_amd64.deb
>
> /var/cache/apt/archives/libdns-export162_1%3a9.10.3.dfsg.P4-8ubuntu1.11_amd64.deb
> /var/cache/apt/archives/isc-dhcp-client_4.3.3-5ubuntu12.9_amd64.deb
> /var/cache/apt/archives/isc-dhcp-common_4.3.3-5ubuntu12.9_amd64.deb
> /var/cache/apt/archives/libidn11_1.32-3ubuntu1.2_amd64.deb
> /var/cache/apt/archives/libpng12-0_1.2.54-1ubuntu1.1_amd64.deb
> /var/cache/apt/archives/libtasn1-6_4.7-3ubuntu0.16.04.3_amd64.deb
> /var/cache/apt/archives/libapparmor-perl_2.10.95-0ubuntu2.10_amd64.deb
> /var/cache/apt/archives/apparmor_2.10.95-0ubuntu2.10_amd64.deb
> /var/cache/apt/archives/curl_7.47.0-1ubuntu2.11_amd64.deb
> /var/cache/apt/archives/libgssapi-krb5-2_1.13.2+dfsg-5ubuntu2.1_amd64.deb
> /var/cache/apt/archives/libkrb5-3_1.13.2+dfsg-5ubuntu2.1_amd64.deb
> /var/cache/apt/archives/libkrb5support0_1.13.2+dfsg-5ubuntu2.1_amd64.deb
> /var/cache/apt/archives/libk5crypto3_1.13.2+dfsg-5ubuntu2.1_amd64.deb
> /var/cache/apt/archives/libcurl3-gnutls_7.47.0-1ubuntu2.11_amd64.deb
> /var/cache/apt/archives/apt-transport-https_1.2.29ubuntu0.1_amd64.deb
> /var/cache/apt/archives/libicu55_55.1-7ubuntu0.4_amd64.deb
> /var/cache/apt/archives/libxml2_2.9.3+dfsg1-1ubuntu0.6_amd64.deb
> /var/cache/apt/archives/bind9-host_1%3a9.10.3.dfsg.P4-8ubuntu1.11_amd64.deb
> /var/cache/apt/archives/dnsutils_1%3a9.10.3.dfsg.P4-8ubuntu1.11_amd64.deb
> /var/cache/apt/archives/libisc160_1%3a9.10.3.dfsg.P4-8ubuntu1.11_amd64.deb
> /var/cache/apt/archives/libdns162_1%3a9.10.3.dfsg.P4-8ubuntu1.11_amd64.deb
> /var/cache/apt/archives/libisccc140_1%3a9.10.3.dfsg.P4-8ubuntu1.11_amd64.deb
> /var/cache/apt/archives/libisccfg140_1%3a9.10.3.dfsg.P4-8ubuntu1.11_amd64.deb
> /var/cache/apt/archives/liblwres141_1%3a9.10.3.dfsg.P4-8ubuntu1.11_amd64.deb
> /var/cache/apt/archives/libbind9-140_1%3a9.10.3.dfsg.P4-8ubuntu1.11_amd64.deb
> /var/cache/apt/archives/openssl_1.0.2g-1ubuntu4.14_amd64.deb
> /var/cache/apt/archives/ca-certificates_20170717~16.04.1_all.deb
> /var/cache/apt/archives/libasprintf0v5_0.19.7-2ubuntu3.1_amd64.deb
> /var/cache/apt/archives/gettext-base_0.19.7-2ubuntu3.1_amd64.deb
> /var/cache/apt/archives/krb5-locales_1.13.2+dfsg-5ubuntu2.1_all.deb
> /var/cache/apt/archives/libelf1_0.165-3ubuntu1.1_amd64.deb
> /var/cache/apt/archives/libglib2.0-data_2.48.2-0ubuntu4.1_all.deb
> /var/cache/apt/archives/libnuma1_2.0.11-1ubuntu1.1_amd64.deb
> /var/cache/apt/archives/libpolkit-gobject-1-0_0.105-14.1ubuntu0.4_amd64.deb
> /var/cache/apt/archives/libx11-data_2%3a1.6.3-1ubuntu2.1_all.deb
> /var/cache/apt/archives/libx11-6_2%3a1.6.3-1ubuntu2.1_amd64.deb
> /var/cache/apt/archives/openssh-sftp-server_1%3a7.2p2-4ubuntu2.6_amd64.deb
[jira] [Updated] (IMPALA-8115) some jenkins workers slow to spawn to to dpkg lock conflicts
[
https://issues.apache.org/jira/browse/IMPALA-8115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Brown updated IMPALA-8115:
--
Description:
A Jenkins worker for label {{ubuntu-16.04}} took about 15 minutes to start
doing real work. I noticed that it was retrying {{apt-get update}}:
{noformat}
++ sudo apt-get --yes install openjdk-8-jdk
E: Could not get lock /var/lib/dpkg/lock - open (11: Resource temporarily
unavailable)
E: Unable to lock the administration directory (/var/lib/dpkg/), is another
process using it?
++ date
Thu Jan 24 23:37:33 UTC 2019
++ sudo apt-get update
++ sleep 10
++ sudo apt-get --yes install openjdk-8-jdk
[etc]
{noformat}
I ssh'd into a host and saw that, yes, something else was holding onto the dpkg
log (confirmed with lsof and not pasted here. dpkg process PID 11459 was the
culprit)
{noformat}
root 1750 0.0 0.0 4508 1664 ?Ss 23:21 0:00 /bin/sh
/usr/lib/apt/apt.systemd.daily
root 1804 12.3 0.1 141076 80452 ?S23:22 1:24 \_
/usr/bin/python3 /usr/bin/unattended-upgrade
root 3263 0.0 0.1 140960 72896 ?S23:23 0:00 \_
/usr/bin/python3 /usr/bin/unattended-upgrade
root 11459 0.6 0.0 45920 25184 pts/1Ss+ 23:24 0:03 \_
/usr/bin/dpkg --status-fd 10 --unpack --auto-deconfigure
/var/cache/apt/archives/tzdata_2018i-0ubuntu0.16.04_all.deb
/var/cache/apt/archives/distro-info-data_0.28ubuntu0.9_all.deb
/var/cache/apt/archives/file_1%3a5.25-2ubuntu1.1_amd64.deb
/var/cache/apt/archives/libmagic1_1%3a5.25-2ubuntu1.1_amd64.deb
/var/cache/apt/archives/libisc-export160_1%3a9.10.3.dfsg.P4-8ubuntu1.11_amd64.deb
/var/cache/apt/archives/libdns-export162_1%3a9.10.3.dfsg.P4-8ubuntu1.11_amd64.deb
/var/cache/apt/archives/isc-dhcp-client_4.3.3-5ubuntu12.9_amd64.deb
/var/cache/apt/archives/isc-dhcp-common_4.3.3-5ubuntu12.9_amd64.deb
/var/cache/apt/archives/libidn11_1.32-3ubuntu1.2_amd64.deb
/var/cache/apt/archives/libpng12-0_1.2.54-1ubuntu1.1_amd64.deb
/var/cache/apt/archives/libtasn1-6_4.7-3ubuntu0.16.04.3_amd64.deb
/var/cache/apt/archives/libapparmor-perl_2.10.95-0ubuntu2.10_amd64.deb
/var/cache/apt/archives/apparmor_2.10.95-0ubuntu2.10_amd64.deb
/var/cache/apt/archives/curl_7.47.0-1ubuntu2.11_amd64.deb
/var/cache/apt/archives/libgssapi-krb5-2_1.13.2+dfsg-5ubuntu2.1_amd64.deb
/var/cache/apt/archives/libkrb5-3_1.13.2+dfsg-5ubuntu2.1_amd64.deb
/var/cache/apt/archives/libkrb5support0_1.13.2+dfsg-5ubuntu2.1_amd64.deb
/var/cache/apt/archives/libk5crypto3_1.13.2+dfsg-5ubuntu2.1_amd64.deb
/var/cache/apt/archives/libcurl3-gnutls_7.47.0-1ubuntu2.11_amd64.deb
/var/cache/apt/archives/apt-transport-https_1.2.29ubuntu0.1_amd64.deb
/var/cache/apt/archives/libicu55_55.1-7ubuntu0.4_amd64.deb
/var/cache/apt/archives/libxml2_2.9.3+dfsg1-1ubuntu0.6_amd64.deb
/var/cache/apt/archives/bind9-host_1%3a9.10.3.dfsg.P4-8ubuntu1.11_amd64.deb
/var/cache/apt/archives/dnsutils_1%3a9.10.3.dfsg.P4-8ubuntu1.11_amd64.deb
/var/cache/apt/archives/libisc160_1%3a9.10.3.dfsg.P4-8ubuntu1.11_amd64.deb
/var/cache/apt/archives/libdns162_1%3a9.10.3.dfsg.P4-8ubuntu1.11_amd64.deb
/var/cache/apt/archives/libisccc140_1%3a9.10.3.dfsg.P4-8ubuntu1.11_amd64.deb
/var/cache/apt/archives/libisccfg140_1%3a9.10.3.dfsg.P4-8ubuntu1.11_amd64.deb
/var/cache/apt/archives/liblwres141_1%3a9.10.3.dfsg.P4-8ubuntu1.11_amd64.deb
/var/cache/apt/archives/libbind9-140_1%3a9.10.3.dfsg.P4-8ubuntu1.11_amd64.deb
/var/cache/apt/archives/openssl_1.0.2g-1ubuntu4.14_amd64.deb
/var/cache/apt/archives/ca-certificates_20170717~16.04.1_all.deb
/var/cache/apt/archives/libasprintf0v5_0.19.7-2ubuntu3.1_amd64.deb
/var/cache/apt/archives/gettext-base_0.19.7-2ubuntu3.1_amd64.deb
/var/cache/apt/archives/krb5-locales_1.13.2+dfsg-5ubuntu2.1_all.deb
/var/cache/apt/archives/libelf1_0.165-3ubuntu1.1_amd64.deb
/var/cache/apt/archives/libglib2.0-data_2.48.2-0ubuntu4.1_all.deb
/var/cache/apt/archives/libnuma1_2.0.11-1ubuntu1.1_amd64.deb
/var/cache/apt/archives/libpolkit-gobject-1-0_0.105-14.1ubuntu0.4_amd64.deb
/var/cache/apt/archives/libx11-data_2%3a1.6.3-1ubuntu2.1_all.deb
/var/cache/apt/archives/libx11-6_2%3a1.6.3-1ubuntu2.1_amd64.deb
/var/cache/apt/archives/openssh-sftp-server_1%3a7.2p2-4ubuntu2.6_amd64.deb
/var/cache/apt/archives/openssh-server_1%3a7.2p2-4ubuntu2.6_amd64.deb
/var/cache/apt/archives/openssh-client_1%3a7.2p2-4ubuntu2.6_amd64.deb
/var/cache/apt/archives/rsync_3.1.1-3ubuntu1.2_amd64.deb
/var/cache/apt/archives/tcpdump_4.9.2-0ubuntu0.16.04.1_amd64.deb
/var/cache/apt/archives/wget_1.17.1-1ubuntu1.4_amd64.deb
/var/cache/apt/archives/python3-problem-report_2.20.1-0ubuntu2.18_all.deb
/var/cache/apt/archives/python3-apport_2.20.1-0ubuntu2.18_all.deb
/var/cache/apt/archives/apport_2.20.1-0ubuntu2.18_all.deb
/var/cache/apt/archives/dns-root-data_2018013001~16.04.1_all.deb
/var/cache/apt/archives/dnsmasq-base_2.75-1ubuntu0.16.04.5_amd64.deb
[jira] [Created] (IMPALA-8115) some jenkins workers slow to spawn to to dpkg lock conflicts
Michael Brown created IMPALA-8115:
-
Summary: some jenkins workers slow to spawn to to dpkg lock
conflicts
Key: IMPALA-8115
URL: https://issues.apache.org/jira/browse/IMPALA-8115
Project: IMPALA
Issue Type: Bug
Components: Infrastructure
Reporter: Michael Brown
A Jenkins worker for label {{ubuntu-16.04}} took about 15 minutes to start
doing real work. I noticed that it was retrying {{apt-get update}}:
{noformat}
++ sudo apt-get --yes install openjdk-8-jdk
E: Could not get lock /var/lib/dpkg/lock - open (11: Resource temporarily
unavailable)
E: Unable to lock the administration directory (/var/lib/dpkg/), is another
process using it?
++ date
Thu Jan 24 23:37:33 UTC 2019
++ sudo apt-get update
++ sleep 10
++ sudo apt-get --yes install openjdk-8-jdk
[etc]
{noformat}
I ssh'd into a host and saw that, yes, something else was holding onto the dpkg
log (confirmed with lsof and not pasted here. dpkg process PID 11459 was the
culprit)
{noformat}
root 1750 0.0 0.0 4508 1664 ?Ss 23:21 0:00 /bin/sh
/usr/lib/apt/apt.systemd.daily
root 1804 12.3 0.1 141076 80452 ?S23:22 1:24 \_
/usr/bin/python3 /usr/bin/unattended-upgrade
root 3263 0.0 0.1 140960 72896 ?S23:23 0:00 \_
/usr/bin/python3 /usr/bin/unattended-upgrade
root 11459 0.6 0.0 45920 25184 pts/1Ss+ 23:24 0:03 \_
/usr/bin/dpkg --status-fd 10 --unpack --auto-deconfigure
/var/cache/apt/archives/tzdata_2018i-0ubuntu0.16.04_all.deb
/var/cache/apt/archives/distro-info-data_0.28ubuntu0.9_all.deb
/var/cache/apt/archives/file_1%3a5.25-2ubuntu1.1_amd64.deb
/var/cache/apt/archives/libmagic1_1%3a5.25-2ubuntu1.1_amd64.deb
/var/cache/apt/archives/libisc-export160_1%3a9.10.3.dfsg.P4-8ubuntu1.11_amd64.deb
/var/cache/apt/archives/libdns-export162_1%3a9.10.3.dfsg.P4-8ubuntu1.11_amd64.deb
/var/cache/apt/archives/isc-dhcp-client_4.3.3-5ubuntu12.9_amd64.deb
/var/cache/apt/archives/isc-dhcp-common_4.3.3-5ubuntu12.9_amd64.deb
/var/cache/apt/archives/libidn11_1.32-3ubuntu1.2_amd64.deb
/var/cache/apt/archives/libpng12-0_1.2.54-1ubuntu1.1_amd64.deb
/var/cache/apt/archives/libtasn1-6_4.7-3ubuntu0.16.04.3_amd64.deb
/var/cache/apt/archives/libapparmor-perl_2.10.95-0ubuntu2.10_amd64.deb
/var/cache/apt/archives/apparmor_2.10.95-0ubuntu2.10_amd64.deb
/var/cache/apt/archives/curl_7.47.0-1ubuntu2.11_amd64.deb
/var/cache/apt/archives/libgssapi-krb5-2_1.13.2+dfsg-5ubuntu2.1_amd64.deb
/var/cache/apt/archives/libkrb5-3_1.13.2+dfsg-5ubuntu2.1_amd64.deb
/var/cache/apt/archives/libkrb5support0_1.13.2+dfsg-5ubuntu2.1_amd64.deb
/var/cache/apt/archives/libk5crypto3_1.13.2+dfsg-5ubuntu2.1_amd64.deb
/var/cache/apt/archives/libcurl3-gnutls_7.47.0-1ubuntu2.11_amd64.deb
/var/cache/apt/archives/apt-transport-https_1.2.29ubuntu0.1_amd64.deb
/var/cache/apt/archives/libicu55_55.1-7ubuntu0.4_amd64.deb
/var/cache/apt/archives/libxml2_2.9.3+dfsg1-1ubuntu0.6_amd64.deb
/var/cache/apt/archives/bind9-host_1%3a9.10.3.dfsg.P4-8ubuntu1.11_amd64.deb
/var/cache/apt/archives/dnsutils_1%3a9.10.3.dfsg.P4-8ubuntu1.11_amd64.deb
/var/cache/apt/archives/libisc160_1%3a9.10.3.dfsg.P4-8ubuntu1.11_amd64.deb
/var/cache/apt/archives/libdns162_1%3a9.10.3.dfsg.P4-8ubuntu1.11_amd64.deb
/var/cache/apt/archives/libisccc140_1%3a9.10.3.dfsg.P4-8ubuntu1.11_amd64.deb
/var/cache/apt/archives/libisccfg140_1%3a9.10.3.dfsg.P4-8ubuntu1.11_amd64.deb
/var/cache/apt/archives/liblwres141_1%3a9.10.3.dfsg.P4-8ubuntu1.11_amd64.deb
/var/cache/apt/archives/libbind9-140_1%3a9.10.3.dfsg.P4-8ubuntu1.11_amd64.deb
/var/cache/apt/archives/openssl_1.0.2g-1ubuntu4.14_amd64.deb
/var/cache/apt/archives/ca-certificates_20170717~16.04.1_all.deb
/var/cache/apt/archives/libasprintf0v5_0.19.7-2ubuntu3.1_amd64.deb
/var/cache/apt/archives/gettext-base_0.19.7-2ubuntu3.1_amd64.deb
/var/cache/apt/archives/krb5-locales_1.13.2+dfsg-5ubuntu2.1_all.deb
/var/cache/apt/archives/libelf1_0.165-3ubuntu1.1_amd64.deb
/var/cache/apt/archives/libglib2.0-data_2.48.2-0ubuntu4.1_all.deb
/var/cache/apt/archives/libnuma1_2.0.11-1ubuntu1.1_amd64.deb
/var/cache/apt/archives/libpolkit-gobject-1-0_0.105-14.1ubuntu0.4_amd64.deb
/var/cache/apt/archives/libx11-data_2%3a1.6.3-1ubuntu2.1_all.deb
/var/cache/apt/archives/libx11-6_2%3a1.6.3-1ubuntu2.1_amd64.deb
/var/cache/apt/archives/openssh-sftp-server_1%3a7.2p2-4ubuntu2.6_amd64.deb
/var/cache/apt/archives/openssh-server_1%3a7.2p2-4ubuntu2.6_amd64.deb
/var/cache/apt/archives/openssh-client_1%3a7.2p2-4ubuntu2.6_amd64.deb
/var/cache/apt/archives/rsync_3.1.1-3ubuntu1.2_amd64.deb
/var/cache/apt/archives/tcpdump_4.9.2-0ubuntu0.16.04.1_amd64.deb
/var/cache/apt/archives/wget_1.17.1-1ubuntu1.4_amd64.deb
/var/cache/apt/archives/python3-problem-report_2.20.1-0ubuntu2.18_all.deb
/var/cache/apt/archives/python3-apport_2.20.1-0ubuntu2.18_all.deb
[jira] [Created] (IMPALA-8115) some jenkins workers slow to spawn to to dpkg lock conflicts
Michael Brown created IMPALA-8115:
-
Summary: some jenkins workers slow to spawn to to dpkg lock
conflicts
Key: IMPALA-8115
URL: https://issues.apache.org/jira/browse/IMPALA-8115
Project: IMPALA
Issue Type: Bug
Components: Infrastructure
Reporter: Michael Brown
A Jenkins worker for label {{ubuntu-16.04}} took about 15 minutes to start
doing real work. I noticed that it was retrying {{apt-get update}}:
{noformat}
++ sudo apt-get --yes install openjdk-8-jdk
E: Could not get lock /var/lib/dpkg/lock - open (11: Resource temporarily
unavailable)
E: Unable to lock the administration directory (/var/lib/dpkg/), is another
process using it?
++ date
Thu Jan 24 23:37:33 UTC 2019
++ sudo apt-get update
++ sleep 10
++ sudo apt-get --yes install openjdk-8-jdk
[etc]
{noformat}
I ssh'd into a host and saw that, yes, something else was holding onto the dpkg
log (confirmed with lsof and not pasted here. dpkg process PID 11459 was the
culprit)
{noformat}
root 1750 0.0 0.0 4508 1664 ?Ss 23:21 0:00 /bin/sh
/usr/lib/apt/apt.systemd.daily
root 1804 12.3 0.1 141076 80452 ?S23:22 1:24 \_
/usr/bin/python3 /usr/bin/unattended-upgrade
root 3263 0.0 0.1 140960 72896 ?S23:23 0:00 \_
/usr/bin/python3 /usr/bin/unattended-upgrade
root 11459 0.6 0.0 45920 25184 pts/1Ss+ 23:24 0:03 \_
/usr/bin/dpkg --status-fd 10 --unpack --auto-deconfigure
/var/cache/apt/archives/tzdata_2018i-0ubuntu0.16.04_all.deb
/var/cache/apt/archives/distro-info-data_0.28ubuntu0.9_all.deb
/var/cache/apt/archives/file_1%3a5.25-2ubuntu1.1_amd64.deb
/var/cache/apt/archives/libmagic1_1%3a5.25-2ubuntu1.1_amd64.deb
/var/cache/apt/archives/libisc-export160_1%3a9.10.3.dfsg.P4-8ubuntu1.11_amd64.deb
/var/cache/apt/archives/libdns-export162_1%3a9.10.3.dfsg.P4-8ubuntu1.11_amd64.deb
/var/cache/apt/archives/isc-dhcp-client_4.3.3-5ubuntu12.9_amd64.deb
/var/cache/apt/archives/isc-dhcp-common_4.3.3-5ubuntu12.9_amd64.deb
/var/cache/apt/archives/libidn11_1.32-3ubuntu1.2_amd64.deb
/var/cache/apt/archives/libpng12-0_1.2.54-1ubuntu1.1_amd64.deb
/var/cache/apt/archives/libtasn1-6_4.7-3ubuntu0.16.04.3_amd64.deb
/var/cache/apt/archives/libapparmor-perl_2.10.95-0ubuntu2.10_amd64.deb
/var/cache/apt/archives/apparmor_2.10.95-0ubuntu2.10_amd64.deb
/var/cache/apt/archives/curl_7.47.0-1ubuntu2.11_amd64.deb
/var/cache/apt/archives/libgssapi-krb5-2_1.13.2+dfsg-5ubuntu2.1_amd64.deb
/var/cache/apt/archives/libkrb5-3_1.13.2+dfsg-5ubuntu2.1_amd64.deb
/var/cache/apt/archives/libkrb5support0_1.13.2+dfsg-5ubuntu2.1_amd64.deb
/var/cache/apt/archives/libk5crypto3_1.13.2+dfsg-5ubuntu2.1_amd64.deb
/var/cache/apt/archives/libcurl3-gnutls_7.47.0-1ubuntu2.11_amd64.deb
/var/cache/apt/archives/apt-transport-https_1.2.29ubuntu0.1_amd64.deb
/var/cache/apt/archives/libicu55_55.1-7ubuntu0.4_amd64.deb
/var/cache/apt/archives/libxml2_2.9.3+dfsg1-1ubuntu0.6_amd64.deb
/var/cache/apt/archives/bind9-host_1%3a9.10.3.dfsg.P4-8ubuntu1.11_amd64.deb
/var/cache/apt/archives/dnsutils_1%3a9.10.3.dfsg.P4-8ubuntu1.11_amd64.deb
/var/cache/apt/archives/libisc160_1%3a9.10.3.dfsg.P4-8ubuntu1.11_amd64.deb
/var/cache/apt/archives/libdns162_1%3a9.10.3.dfsg.P4-8ubuntu1.11_amd64.deb
/var/cache/apt/archives/libisccc140_1%3a9.10.3.dfsg.P4-8ubuntu1.11_amd64.deb
/var/cache/apt/archives/libisccfg140_1%3a9.10.3.dfsg.P4-8ubuntu1.11_amd64.deb
/var/cache/apt/archives/liblwres141_1%3a9.10.3.dfsg.P4-8ubuntu1.11_amd64.deb
/var/cache/apt/archives/libbind9-140_1%3a9.10.3.dfsg.P4-8ubuntu1.11_amd64.deb
/var/cache/apt/archives/openssl_1.0.2g-1ubuntu4.14_amd64.deb
/var/cache/apt/archives/ca-certificates_20170717~16.04.1_all.deb
/var/cache/apt/archives/libasprintf0v5_0.19.7-2ubuntu3.1_amd64.deb
/var/cache/apt/archives/gettext-base_0.19.7-2ubuntu3.1_amd64.deb
/var/cache/apt/archives/krb5-locales_1.13.2+dfsg-5ubuntu2.1_all.deb
/var/cache/apt/archives/libelf1_0.165-3ubuntu1.1_amd64.deb
/var/cache/apt/archives/libglib2.0-data_2.48.2-0ubuntu4.1_all.deb
/var/cache/apt/archives/libnuma1_2.0.11-1ubuntu1.1_amd64.deb
/var/cache/apt/archives/libpolkit-gobject-1-0_0.105-14.1ubuntu0.4_amd64.deb
/var/cache/apt/archives/libx11-data_2%3a1.6.3-1ubuntu2.1_all.deb
/var/cache/apt/archives/libx11-6_2%3a1.6.3-1ubuntu2.1_amd64.deb
/var/cache/apt/archives/openssh-sftp-server_1%3a7.2p2-4ubuntu2.6_amd64.deb
/var/cache/apt/archives/openssh-server_1%3a7.2p2-4ubuntu2.6_amd64.deb
/var/cache/apt/archives/openssh-client_1%3a7.2p2-4ubuntu2.6_amd64.deb
/var/cache/apt/archives/rsync_3.1.1-3ubuntu1.2_amd64.deb
/var/cache/apt/archives/tcpdump_4.9.2-0ubuntu0.16.04.1_amd64.deb
/var/cache/apt/archives/wget_1.17.1-1ubuntu1.4_amd64.deb
/var/cache/apt/archives/python3-problem-report_2.20.1-0ubuntu2.18_all.deb
/var/cache/apt/archives/python3-apport_2.20.1-0ubuntu2.18_all.deb
[jira] [Created] (IMPALA-8113) test_aggregation and test_avro_primitive_in_list fail in S3
Michael Brown created IMPALA-8113:
-
Summary: test_aggregation and test_avro_primitive_in_list fail in
S3
Key: IMPALA-8113
URL: https://issues.apache.org/jira/browse/IMPALA-8113
Project: IMPALA
Issue Type: Bug
Components: Infrastructure
Affects Versions: Impala 3.2.0
Reporter: Michael Brown
Assignee: Michael Brown
Likely more victims of our infra in S3.
{noformat}
query_test/test_aggregation.py:138: in test_aggregation
result = self.execute_query(query, vector.get_value('exec_option'))
common/impala_test_suite.py:597: in wrapper
return function(*args, **kwargs)
common/impala_test_suite.py:628: in execute_query
return self.__execute_query(self.client, query, query_options)
common/impala_test_suite.py:695: in __execute_query
return impalad_client.execute(query, user=user)
common/impala_connection.py:174: in execute
return self.__beeswax_client.execute(sql_stmt, user=user)
beeswax/impala_beeswax.py:182: in execute
handle = self.__execute_query(query_string.strip(), user=user)
beeswax/impala_beeswax.py:359: in __execute_query
self.wait_for_finished(handle)
beeswax/impala_beeswax.py:380: in wait_for_finished
raise ImpalaBeeswaxException("Query aborted:" + error_log, None)
E ImpalaBeeswaxException: ImpalaBeeswaxException:
EQuery aborted:Disk I/O error: Error reading from HDFS file:
s3a://impala-test-uswest2-1/test-warehouse/alltypesagg_parquet/year=2010/month=1/day=8/5642b2da93dae1ad-494132e5_592013737_data.0.parq
E Error(255): Unknown error 255
E Root cause: SdkClientException: Data read has a different length than the
expected: dataLength=0; expectedLength=45494; includeSkipped=true;
in.getClass()=class com.amazonaws.services.s3.AmazonS3Client$2;
markedSupported=false; marked=0; resetSinceLastMarked=false; markCount=0;
resetCount=0
{noformat}
{noformat}
query_test/test_nested_types.py:263: in test_avro_primitive_in_list
"AvroPrimitiveInList.parquet", vector)
query_test/test_nested_types.py:287: in __test_primitive_in_list
result = self.execute_query("select item from %s.col1" % full_name, qopts)
common/impala_test_suite.py:597: in wrapper
return function(*args, **kwargs)
common/impala_test_suite.py:628: in execute_query
return self.__execute_query(self.client, query, query_options)
common/impala_test_suite.py:695: in __execute_query
return impalad_client.execute(query, user=user)
common/impala_connection.py:174: in execute
return self.__beeswax_client.execute(sql_stmt, user=user)
beeswax/impala_beeswax.py:182: in execute
handle = self.__execute_query(query_string.strip(), user=user)
beeswax/impala_beeswax.py:359: in __execute_query
self.wait_for_finished(handle)
beeswax/impala_beeswax.py:380: in wait_for_finished
raise ImpalaBeeswaxException("Query aborted:" + error_log, None)
E ImpalaBeeswaxException: ImpalaBeeswaxException:
EQuery aborted:Disk I/O error: Failed to open HDFS file
s3a://impala-test-uswest2-1/test-warehouse/test_avro_primitive_in_list_38f182c4.db/AvroPrimitiveInList/AvroPrimitiveInList.parquet
E Error(2): No such file or directory
E Root cause: FileNotFoundException: No such file or directory:
s3a://impala-test-uswest2-1/test-warehouse/test_avro_primitive_in_list_38f182c4.db/AvroPrimitiveInList/AvroPrimitiveInList.parquet
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-8113) test_aggregation and test_avro_primitive_in_list fail in S3
Michael Brown created IMPALA-8113:
-
Summary: test_aggregation and test_avro_primitive_in_list fail in
S3
Key: IMPALA-8113
URL: https://issues.apache.org/jira/browse/IMPALA-8113
Project: IMPALA
Issue Type: Bug
Components: Infrastructure
Affects Versions: Impala 3.2.0
Reporter: Michael Brown
Assignee: Michael Brown
Likely more victims of our infra in S3.
{noformat}
query_test/test_aggregation.py:138: in test_aggregation
result = self.execute_query(query, vector.get_value('exec_option'))
common/impala_test_suite.py:597: in wrapper
return function(*args, **kwargs)
common/impala_test_suite.py:628: in execute_query
return self.__execute_query(self.client, query, query_options)
common/impala_test_suite.py:695: in __execute_query
return impalad_client.execute(query, user=user)
common/impala_connection.py:174: in execute
return self.__beeswax_client.execute(sql_stmt, user=user)
beeswax/impala_beeswax.py:182: in execute
handle = self.__execute_query(query_string.strip(), user=user)
beeswax/impala_beeswax.py:359: in __execute_query
self.wait_for_finished(handle)
beeswax/impala_beeswax.py:380: in wait_for_finished
raise ImpalaBeeswaxException("Query aborted:" + error_log, None)
E ImpalaBeeswaxException: ImpalaBeeswaxException:
EQuery aborted:Disk I/O error: Error reading from HDFS file:
s3a://impala-test-uswest2-1/test-warehouse/alltypesagg_parquet/year=2010/month=1/day=8/5642b2da93dae1ad-494132e5_592013737_data.0.parq
E Error(255): Unknown error 255
E Root cause: SdkClientException: Data read has a different length than the
expected: dataLength=0; expectedLength=45494; includeSkipped=true;
in.getClass()=class com.amazonaws.services.s3.AmazonS3Client$2;
markedSupported=false; marked=0; resetSinceLastMarked=false; markCount=0;
resetCount=0
{noformat}
{noformat}
query_test/test_nested_types.py:263: in test_avro_primitive_in_list
"AvroPrimitiveInList.parquet", vector)
query_test/test_nested_types.py:287: in __test_primitive_in_list
result = self.execute_query("select item from %s.col1" % full_name, qopts)
common/impala_test_suite.py:597: in wrapper
return function(*args, **kwargs)
common/impala_test_suite.py:628: in execute_query
return self.__execute_query(self.client, query, query_options)
common/impala_test_suite.py:695: in __execute_query
return impalad_client.execute(query, user=user)
common/impala_connection.py:174: in execute
return self.__beeswax_client.execute(sql_stmt, user=user)
beeswax/impala_beeswax.py:182: in execute
handle = self.__execute_query(query_string.strip(), user=user)
beeswax/impala_beeswax.py:359: in __execute_query
self.wait_for_finished(handle)
beeswax/impala_beeswax.py:380: in wait_for_finished
raise ImpalaBeeswaxException("Query aborted:" + error_log, None)
E ImpalaBeeswaxException: ImpalaBeeswaxException:
EQuery aborted:Disk I/O error: Failed to open HDFS file
s3a://impala-test-uswest2-1/test-warehouse/test_avro_primitive_in_list_38f182c4.db/AvroPrimitiveInList/AvroPrimitiveInList.parquet
E Error(2): No such file or directory
E Root cause: FileNotFoundException: No such file or directory:
s3a://impala-test-uswest2-1/test-warehouse/test_avro_primitive_in_list_38f182c4.db/AvroPrimitiveInList/AvroPrimitiveInList.parquet
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (IMPALA-8112) test_cancel_select with debug action failed with unexpected error
[
https://issues.apache.org/jira/browse/IMPALA-8112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Brown updated IMPALA-8112:
--
Labels: flaky (was: )
> test_cancel_select with debug action failed with unexpected error
> -
>
> Key: IMPALA-8112
> URL: https://issues.apache.org/jira/browse/IMPALA-8112
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 3.2.0
>Reporter: Michael Brown
>Assignee: Andrew Sherman
>Priority: Critical
> Labels: flaky
>
> Stacktrace
> {noformat}
> query_test/test_cancellation.py:241: in test_cancel_select
> self.execute_cancel_test(vector)
> query_test/test_cancellation.py:213: in execute_cancel_test
> assert 'Cancelled' in str(thread.fetch_results_error)
> E assert 'Cancelled' in "ImpalaBeeswaxException:\n INNER EXCEPTION: 'beeswaxd.ttypes.BeeswaxException'>\n MESSAGE: Unable to open Kudu table:
> Network error: recv error from 0.0.0.0:0: Transport endpoint is not connected
> (error 107)\n"
> E+ where "ImpalaBeeswaxException:\n INNER EXCEPTION: 'beeswaxd.ttypes.BeeswaxException'>\n MESSAGE: Unable to open Kudu table:
> Network error: recv error from 0.0.0.0:0: Transport endpoint is not connected
> (error 107)\n" = str(ImpalaBeeswaxException())
> E+where ImpalaBeeswaxException() = 140481071658752)>.fetch_results_error
> {noformat}
> Standard Error
> {noformat}
> SET
> client_identifier=query_test/test_cancellation.py::TestCancellationParallel::()::test_cancel_select[protocol:beeswax|table_format:kudu/none|exec_option:{'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':0;'disable_codegen':False;'abort_on_error':1;'debug_action;
> -- executing against localhost:21000
> use tpch_kudu;
> -- 2019-01-18 17:50:03,100 INFO MainThread: Started query
> 4e4b3ab4cc7d:11efc3f5
> SET
> client_identifier=query_test/test_cancellation.py::TestCancellationParallel::()::test_cancel_select[protocol:beeswax|table_format:kudu/none|exec_option:{'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':0;'disable_codegen':False;'abort_on_error':1;'debug_action;
> SET batch_size=0;
> SET num_nodes=0;
> SET disable_codegen_rows_threshold=0;
> SET disable_codegen=False;
> SET abort_on_error=1;
> SET cpu_limit_s=10;
> SET debug_action=0:GETNEXT:WAIT|COORD_CANCEL_QUERY_FINSTANCES_RPC:FAIL;
> SET exec_single_node_rows_threshold=0;
> SET buffer_pool_limit=0;
> -- executing async: localhost:21000
> select l_returnflag from lineitem;
> -- 2019-01-18 17:50:03,139 INFO MainThread: Started query
> fa4ddb9e62a01240:54c86ad
> SET
> client_identifier=query_test/test_cancellation.py::TestCancellationParallel::()::test_cancel_select[protocol:beeswax|table_format:kudu/none|exec_option:{'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':0;'disable_codegen':False;'abort_on_error':1;'debug_action;
> -- connecting to: localhost:21000
> -- fetching results from: object at 0x6235e90>
> -- getting state for operation:
>
> -- canceling operation: object at 0x6235e90>
> -- 2019-01-18 17:50:08,196 INFO Thread-4: Starting new HTTP connection
> (1): localhost
> -- closing query for operation handle:
>
> {noformat}
> [~asherman] please take a look since it looks like you touched code around
> this area last.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Issue Comment Deleted] (IMPALA-8112) test_cancel_select with debug action failed with unexpected error
[
https://issues.apache.org/jira/browse/IMPALA-8112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Brown updated IMPALA-8112:
--
Comment: was deleted
(was: This has happened once on a downstream CI job
https://master-02.jenkins.cloudera.com/job/impala-cdh6.x-core/532/ .)
> test_cancel_select with debug action failed with unexpected error
> -
>
> Key: IMPALA-8112
> URL: https://issues.apache.org/jira/browse/IMPALA-8112
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 3.2.0
>Reporter: Michael Brown
>Assignee: Andrew Sherman
>Priority: Critical
> Labels: flaky
>
> Stacktrace
> {noformat}
> query_test/test_cancellation.py:241: in test_cancel_select
> self.execute_cancel_test(vector)
> query_test/test_cancellation.py:213: in execute_cancel_test
> assert 'Cancelled' in str(thread.fetch_results_error)
> E assert 'Cancelled' in "ImpalaBeeswaxException:\n INNER EXCEPTION: 'beeswaxd.ttypes.BeeswaxException'>\n MESSAGE: Unable to open Kudu table:
> Network error: recv error from 0.0.0.0:0: Transport endpoint is not connected
> (error 107)\n"
> E+ where "ImpalaBeeswaxException:\n INNER EXCEPTION: 'beeswaxd.ttypes.BeeswaxException'>\n MESSAGE: Unable to open Kudu table:
> Network error: recv error from 0.0.0.0:0: Transport endpoint is not connected
> (error 107)\n" = str(ImpalaBeeswaxException())
> E+where ImpalaBeeswaxException() = 140481071658752)>.fetch_results_error
> {noformat}
> Standard Error
> {noformat}
> SET
> client_identifier=query_test/test_cancellation.py::TestCancellationParallel::()::test_cancel_select[protocol:beeswax|table_format:kudu/none|exec_option:{'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':0;'disable_codegen':False;'abort_on_error':1;'debug_action;
> -- executing against localhost:21000
> use tpch_kudu;
> -- 2019-01-18 17:50:03,100 INFO MainThread: Started query
> 4e4b3ab4cc7d:11efc3f5
> SET
> client_identifier=query_test/test_cancellation.py::TestCancellationParallel::()::test_cancel_select[protocol:beeswax|table_format:kudu/none|exec_option:{'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':0;'disable_codegen':False;'abort_on_error':1;'debug_action;
> SET batch_size=0;
> SET num_nodes=0;
> SET disable_codegen_rows_threshold=0;
> SET disable_codegen=False;
> SET abort_on_error=1;
> SET cpu_limit_s=10;
> SET debug_action=0:GETNEXT:WAIT|COORD_CANCEL_QUERY_FINSTANCES_RPC:FAIL;
> SET exec_single_node_rows_threshold=0;
> SET buffer_pool_limit=0;
> -- executing async: localhost:21000
> select l_returnflag from lineitem;
> -- 2019-01-18 17:50:03,139 INFO MainThread: Started query
> fa4ddb9e62a01240:54c86ad
> SET
> client_identifier=query_test/test_cancellation.py::TestCancellationParallel::()::test_cancel_select[protocol:beeswax|table_format:kudu/none|exec_option:{'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':0;'disable_codegen':False;'abort_on_error':1;'debug_action;
> -- connecting to: localhost:21000
> -- fetching results from: object at 0x6235e90>
> -- getting state for operation:
>
> -- canceling operation: object at 0x6235e90>
> -- 2019-01-18 17:50:08,196 INFO Thread-4: Starting new HTTP connection
> (1): localhost
> -- closing query for operation handle:
>
> {noformat}
> [~asherman] please take a look since it looks like you touched code around
> this area last.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-8112) test_cancel_select with debug action failed with unexpected error
[
https://issues.apache.org/jira/browse/IMPALA-8112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751610#comment-16751610
]
Michael Brown commented on IMPALA-8112:
---
This has happened once on a downstream CI job
https://master-02.jenkins.cloudera.com/job/impala-cdh6.x-core/532/ .
> test_cancel_select with debug action failed with unexpected error
> -
>
> Key: IMPALA-8112
> URL: https://issues.apache.org/jira/browse/IMPALA-8112
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 3.2.0
>Reporter: Michael Brown
>Assignee: Andrew Sherman
>Priority: Critical
>
> Stacktrace
> {noformat}
> query_test/test_cancellation.py:241: in test_cancel_select
> self.execute_cancel_test(vector)
> query_test/test_cancellation.py:213: in execute_cancel_test
> assert 'Cancelled' in str(thread.fetch_results_error)
> E assert 'Cancelled' in "ImpalaBeeswaxException:\n INNER EXCEPTION: 'beeswaxd.ttypes.BeeswaxException'>\n MESSAGE: Unable to open Kudu table:
> Network error: recv error from 0.0.0.0:0: Transport endpoint is not connected
> (error 107)\n"
> E+ where "ImpalaBeeswaxException:\n INNER EXCEPTION: 'beeswaxd.ttypes.BeeswaxException'>\n MESSAGE: Unable to open Kudu table:
> Network error: recv error from 0.0.0.0:0: Transport endpoint is not connected
> (error 107)\n" = str(ImpalaBeeswaxException())
> E+where ImpalaBeeswaxException() = 140481071658752)>.fetch_results_error
> {noformat}
> Standard Error
> {noformat}
> SET
> client_identifier=query_test/test_cancellation.py::TestCancellationParallel::()::test_cancel_select[protocol:beeswax|table_format:kudu/none|exec_option:{'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':0;'disable_codegen':False;'abort_on_error':1;'debug_action;
> -- executing against localhost:21000
> use tpch_kudu;
> -- 2019-01-18 17:50:03,100 INFO MainThread: Started query
> 4e4b3ab4cc7d:11efc3f5
> SET
> client_identifier=query_test/test_cancellation.py::TestCancellationParallel::()::test_cancel_select[protocol:beeswax|table_format:kudu/none|exec_option:{'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':0;'disable_codegen':False;'abort_on_error':1;'debug_action;
> SET batch_size=0;
> SET num_nodes=0;
> SET disable_codegen_rows_threshold=0;
> SET disable_codegen=False;
> SET abort_on_error=1;
> SET cpu_limit_s=10;
> SET debug_action=0:GETNEXT:WAIT|COORD_CANCEL_QUERY_FINSTANCES_RPC:FAIL;
> SET exec_single_node_rows_threshold=0;
> SET buffer_pool_limit=0;
> -- executing async: localhost:21000
> select l_returnflag from lineitem;
> -- 2019-01-18 17:50:03,139 INFO MainThread: Started query
> fa4ddb9e62a01240:54c86ad
> SET
> client_identifier=query_test/test_cancellation.py::TestCancellationParallel::()::test_cancel_select[protocol:beeswax|table_format:kudu/none|exec_option:{'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':0;'disable_codegen':False;'abort_on_error':1;'debug_action;
> -- connecting to: localhost:21000
> -- fetching results from: object at 0x6235e90>
> -- getting state for operation:
>
> -- canceling operation: object at 0x6235e90>
> -- 2019-01-18 17:50:08,196 INFO Thread-4: Starting new HTTP connection
> (1): localhost
> -- closing query for operation handle:
>
> {noformat}
> [~asherman] please take a look since it looks like you touched code around
> this area last.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-8112) test_cancel_select with debug action failed with unexpected error
Michael Brown created IMPALA-8112:
-
Summary: test_cancel_select with debug action failed with
unexpected error
Key: IMPALA-8112
URL: https://issues.apache.org/jira/browse/IMPALA-8112
Project: IMPALA
Issue Type: Bug
Components: Backend
Affects Versions: Impala 3.2.0
Reporter: Michael Brown
Assignee: Andrew Sherman
Stacktrace
{noformat}
query_test/test_cancellation.py:241: in test_cancel_select
self.execute_cancel_test(vector)
query_test/test_cancellation.py:213: in execute_cancel_test
assert 'Cancelled' in str(thread.fetch_results_error)
E assert 'Cancelled' in "ImpalaBeeswaxException:\n INNER EXCEPTION: \n MESSAGE: Unable to open Kudu table:
Network error: recv error from 0.0.0.0:0: Transport endpoint is not connected
(error 107)\n"
E+ where "ImpalaBeeswaxException:\n INNER EXCEPTION: \n MESSAGE: Unable to open Kudu table:
Network error: recv error from 0.0.0.0:0: Transport endpoint is not connected
(error 107)\n" = str(ImpalaBeeswaxException())
E+where ImpalaBeeswaxException() = .fetch_results_error
{noformat}
Standard Error
{noformat}
SET
client_identifier=query_test/test_cancellation.py::TestCancellationParallel::()::test_cancel_select[protocol:beeswax|table_format:kudu/none|exec_option:{'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':0;'disable_codegen':False;'abort_on_error':1;'debug_action;
-- executing against localhost:21000
use tpch_kudu;
-- 2019-01-18 17:50:03,100 INFO MainThread: Started query
4e4b3ab4cc7d:11efc3f5
SET
client_identifier=query_test/test_cancellation.py::TestCancellationParallel::()::test_cancel_select[protocol:beeswax|table_format:kudu/none|exec_option:{'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':0;'disable_codegen':False;'abort_on_error':1;'debug_action;
SET batch_size=0;
SET num_nodes=0;
SET disable_codegen_rows_threshold=0;
SET disable_codegen=False;
SET abort_on_error=1;
SET cpu_limit_s=10;
SET debug_action=0:GETNEXT:WAIT|COORD_CANCEL_QUERY_FINSTANCES_RPC:FAIL;
SET exec_single_node_rows_threshold=0;
SET buffer_pool_limit=0;
-- executing async: localhost:21000
select l_returnflag from lineitem;
-- 2019-01-18 17:50:03,139 INFO MainThread: Started query
fa4ddb9e62a01240:54c86ad
SET
client_identifier=query_test/test_cancellation.py::TestCancellationParallel::()::test_cancel_select[protocol:beeswax|table_format:kudu/none|exec_option:{'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':0;'disable_codegen':False;'abort_on_error':1;'debug_action;
-- connecting to: localhost:21000
-- fetching results from:
-- getting state for operation:
-- canceling operation:
-- 2019-01-18 17:50:08,196 INFO Thread-4: Starting new HTTP connection (1):
localhost
-- closing query for operation handle:
{noformat}
[~asherman] please take a look since it looks like you touched code around this
area last.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (IMPALA-8112) test_cancel_select with debug action failed with unexpected error
Michael Brown created IMPALA-8112:
-
Summary: test_cancel_select with debug action failed with
unexpected error
Key: IMPALA-8112
URL: https://issues.apache.org/jira/browse/IMPALA-8112
Project: IMPALA
Issue Type: Bug
Components: Backend
Affects Versions: Impala 3.2.0
Reporter: Michael Brown
Assignee: Andrew Sherman
Stacktrace
{noformat}
query_test/test_cancellation.py:241: in test_cancel_select
self.execute_cancel_test(vector)
query_test/test_cancellation.py:213: in execute_cancel_test
assert 'Cancelled' in str(thread.fetch_results_error)
E assert 'Cancelled' in "ImpalaBeeswaxException:\n INNER EXCEPTION: \n MESSAGE: Unable to open Kudu table:
Network error: recv error from 0.0.0.0:0: Transport endpoint is not connected
(error 107)\n"
E+ where "ImpalaBeeswaxException:\n INNER EXCEPTION: \n MESSAGE: Unable to open Kudu table:
Network error: recv error from 0.0.0.0:0: Transport endpoint is not connected
(error 107)\n" = str(ImpalaBeeswaxException())
E+where ImpalaBeeswaxException() = .fetch_results_error
{noformat}
Standard Error
{noformat}
SET
client_identifier=query_test/test_cancellation.py::TestCancellationParallel::()::test_cancel_select[protocol:beeswax|table_format:kudu/none|exec_option:{'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':0;'disable_codegen':False;'abort_on_error':1;'debug_action;
-- executing against localhost:21000
use tpch_kudu;
-- 2019-01-18 17:50:03,100 INFO MainThread: Started query
4e4b3ab4cc7d:11efc3f5
SET
client_identifier=query_test/test_cancellation.py::TestCancellationParallel::()::test_cancel_select[protocol:beeswax|table_format:kudu/none|exec_option:{'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':0;'disable_codegen':False;'abort_on_error':1;'debug_action;
SET batch_size=0;
SET num_nodes=0;
SET disable_codegen_rows_threshold=0;
SET disable_codegen=False;
SET abort_on_error=1;
SET cpu_limit_s=10;
SET debug_action=0:GETNEXT:WAIT|COORD_CANCEL_QUERY_FINSTANCES_RPC:FAIL;
SET exec_single_node_rows_threshold=0;
SET buffer_pool_limit=0;
-- executing async: localhost:21000
select l_returnflag from lineitem;
-- 2019-01-18 17:50:03,139 INFO MainThread: Started query
fa4ddb9e62a01240:54c86ad
SET
client_identifier=query_test/test_cancellation.py::TestCancellationParallel::()::test_cancel_select[protocol:beeswax|table_format:kudu/none|exec_option:{'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':0;'disable_codegen':False;'abort_on_error':1;'debug_action;
-- connecting to: localhost:21000
-- fetching results from:
-- getting state for operation:
-- canceling operation:
-- 2019-01-18 17:50:08,196 INFO Thread-4: Starting new HTTP connection (1):
localhost
-- closing query for operation handle:
{noformat}
[~asherman] please take a look since it looks like you touched code around this
area last.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-8091) minicluster kudu failure: Cannot initialize clock: failed to wait for clock sync
[
https://issues.apache.org/jira/browse/IMPALA-8091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Brown resolved IMPALA-8091.
---
Resolution: Fixed
Fix Version/s: Impala 3.2.0
This is all upstream is able to do at this time. Downstream environments should
re-run bootstrap_system.sh or equivalent on their CI hosts.
> minicluster kudu failure: Cannot initialize clock: failed to wait for clock
> sync
>
>
> Key: IMPALA-8091
> URL: https://issues.apache.org/jira/browse/IMPALA-8091
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
>Affects Versions: Impala 3.2.0
>Reporter: David Knupp
>Assignee: Michael Brown
>Priority: Blocker
> Labels: broken-build
> Fix For: Impala 3.2.0
>
>
> *Console log*:
> {noformat}
> 19:10:04 19:10:04 Error executing impala SQL:
> /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/logs/data_loading/sql/functional/create-functional-query-exhaustive-impala-generated-kudu-none-none.sql
> See:
> /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/logs/data_loading/sql/functional/create-functional-query-exhaustive-impala-generated-kudu-none-none.sql.log
> 19:10:04 Background task Loading functional-query data (pid 4831) failed.
> 19:10:04 Background task Loading TPC-H data (pid 4832) failed.
> 19:10:04 ERROR in
> /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/testdata/bin/create-load-data.sh
> at line 85: CM_HOST=${2-}
> [...]
> 19:10:05 2019-01-16 19:10:05,461 - archive_core_dumps - INFO - Found core
> files: ['./core.1547693849.30799.kudu-master',
> './core.1547693849.30783.kudu-tserver',
> './core.1547693849.30767.kudu-tserver',
> './core.1547693849.30808.kudu-tserver']
> 19:10:05 2019-01-16 19:10:05,649 - archive_core_dumps - INFO - [New LWP 30799]
> 19:10:05 [New LWP 30834]
> 19:10:05 [New LWP 30835]
> 19:10:05 [New LWP 30838]
> 19:10:05 [New LWP 30837]
> 19:10:05 [New LWP 30836]
> 19:10:05 Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
> 19:10:05 Program terminated with signal SIGABRT, Aborted.
> 19:10:05 #0 0x7f9a2cc611f7 in ?? ()
> 19:10:05
> 19:10:05 2019-01-16 19:10:05,650 - archive_core_dumps - INFO - Found binary
> path through GDB:
> /data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c
> 19:10:05 2019-01-16 19:10:05,893 - archive_core_dumps - WARNING - Failed to
> determine binary because multiple candidate binaries were found and none of
> their paths contained 'latest' to disambiguate:
> 19:10:05 Core:./core.1547693849.30799.kudu-master
> 19:10:05
> Binaries:['./testdata/cluster/node_templates/common/etc/init.d/kudu-master',
> './testdata/cluster/cdh6/node-1/etc/init.d/kudu-master']
> 19:10:05
> 19:10:05 2019-01-16 19:10:05,917 - archive_core_dumps - INFO - [New LWP 30783]
> 19:10:05 [New LWP 30810]
> 19:10:05 [New LWP 30812]
> 19:10:05 [New LWP 30811]
> 19:10:05 [New LWP 30824]
> 19:10:05 [New LWP 30820]
> 19:10:05 Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
> 19:10:05 Program terminated with signal SIGABRT, Aborted.
> 19:10:05 #0 0x7f0b81fb11f7 in ?? ()
> {noformat}
> *Backtraces*:
> {noformat}
> CORE: ./core.1547693849.30799.kudu-master
> BINARY: ./be/build/latest/service/impalad
> Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
> Program terminated with signal SIGABRT, Aborted.
> #0 0x7f9a2cc611f7 in ?? ()
> #0 0x7f9a2cc611f7 in ?? ()
> #1 0x7f9a2cc628e8 in ?? ()
> #2 0x0020 in ?? ()
> #3 0x in ?? ()
> CORE: ./core.1547693849.30783.kudu-tserver
> BINARY: ./be/build/latest/service/impalad
> Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
> Program terminated with signal SIGABRT, Aborted.
> #0 0x7f0b81fb11f7 in ?? ()
> #0 0x7f0b81fb11f7 in ?? ()
> #1 0x7f0b81fb28e8 in ?? ()
> #2 0x0020 in ?? ()
> #3 0x in ?? ()
> CORE: ./core.1547693849.30767.kudu-tserver
> BINARY: ./be/build/latest/service/impalad
> Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
> Program terminated with signal SIGABRT, Aborted.
> #0 0x7f60b1e2f1f7 in ?? ()
> #0 0x7f60b1e2f1f7 in ?? ()
> #1 0x7f60b1e308e8 in ?? ()
> #2 0x0020 in ?? ()
> #3 0x in ?? ()
> CORE: ./core.1547693849.30808.kudu-tserver
> BINARY: ./be/build/latest/service/impalad
> Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
[jira] [Resolved] (IMPALA-8091) minicluster kudu failure: Cannot initialize clock: failed to wait for clock sync
[
https://issues.apache.org/jira/browse/IMPALA-8091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Brown resolved IMPALA-8091.
---
Resolution: Fixed
Fix Version/s: Impala 3.2.0
This is all upstream is able to do at this time. Downstream environments should
re-run bootstrap_system.sh or equivalent on their CI hosts.
> minicluster kudu failure: Cannot initialize clock: failed to wait for clock
> sync
>
>
> Key: IMPALA-8091
> URL: https://issues.apache.org/jira/browse/IMPALA-8091
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
>Affects Versions: Impala 3.2.0
>Reporter: David Knupp
>Assignee: Michael Brown
>Priority: Blocker
> Labels: broken-build
> Fix For: Impala 3.2.0
>
>
> *Console log*:
> {noformat}
> 19:10:04 19:10:04 Error executing impala SQL:
> /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/logs/data_loading/sql/functional/create-functional-query-exhaustive-impala-generated-kudu-none-none.sql
> See:
> /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/logs/data_loading/sql/functional/create-functional-query-exhaustive-impala-generated-kudu-none-none.sql.log
> 19:10:04 Background task Loading functional-query data (pid 4831) failed.
> 19:10:04 Background task Loading TPC-H data (pid 4832) failed.
> 19:10:04 ERROR in
> /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/testdata/bin/create-load-data.sh
> at line 85: CM_HOST=${2-}
> [...]
> 19:10:05 2019-01-16 19:10:05,461 - archive_core_dumps - INFO - Found core
> files: ['./core.1547693849.30799.kudu-master',
> './core.1547693849.30783.kudu-tserver',
> './core.1547693849.30767.kudu-tserver',
> './core.1547693849.30808.kudu-tserver']
> 19:10:05 2019-01-16 19:10:05,649 - archive_core_dumps - INFO - [New LWP 30799]
> 19:10:05 [New LWP 30834]
> 19:10:05 [New LWP 30835]
> 19:10:05 [New LWP 30838]
> 19:10:05 [New LWP 30837]
> 19:10:05 [New LWP 30836]
> 19:10:05 Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
> 19:10:05 Program terminated with signal SIGABRT, Aborted.
> 19:10:05 #0 0x7f9a2cc611f7 in ?? ()
> 19:10:05
> 19:10:05 2019-01-16 19:10:05,650 - archive_core_dumps - INFO - Found binary
> path through GDB:
> /data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c
> 19:10:05 2019-01-16 19:10:05,893 - archive_core_dumps - WARNING - Failed to
> determine binary because multiple candidate binaries were found and none of
> their paths contained 'latest' to disambiguate:
> 19:10:05 Core:./core.1547693849.30799.kudu-master
> 19:10:05
> Binaries:['./testdata/cluster/node_templates/common/etc/init.d/kudu-master',
> './testdata/cluster/cdh6/node-1/etc/init.d/kudu-master']
> 19:10:05
> 19:10:05 2019-01-16 19:10:05,917 - archive_core_dumps - INFO - [New LWP 30783]
> 19:10:05 [New LWP 30810]
> 19:10:05 [New LWP 30812]
> 19:10:05 [New LWP 30811]
> 19:10:05 [New LWP 30824]
> 19:10:05 [New LWP 30820]
> 19:10:05 Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
> 19:10:05 Program terminated with signal SIGABRT, Aborted.
> 19:10:05 #0 0x7f0b81fb11f7 in ?? ()
> {noformat}
> *Backtraces*:
> {noformat}
> CORE: ./core.1547693849.30799.kudu-master
> BINARY: ./be/build/latest/service/impalad
> Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
> Program terminated with signal SIGABRT, Aborted.
> #0 0x7f9a2cc611f7 in ?? ()
> #0 0x7f9a2cc611f7 in ?? ()
> #1 0x7f9a2cc628e8 in ?? ()
> #2 0x0020 in ?? ()
> #3 0x in ?? ()
> CORE: ./core.1547693849.30783.kudu-tserver
> BINARY: ./be/build/latest/service/impalad
> Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
> Program terminated with signal SIGABRT, Aborted.
> #0 0x7f0b81fb11f7 in ?? ()
> #0 0x7f0b81fb11f7 in ?? ()
> #1 0x7f0b81fb28e8 in ?? ()
> #2 0x0020 in ?? ()
> #3 0x in ?? ()
> CORE: ./core.1547693849.30767.kudu-tserver
> BINARY: ./be/build/latest/service/impalad
> Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
> Program terminated with signal SIGABRT, Aborted.
> #0 0x7f60b1e2f1f7 in ?? ()
> #0 0x7f60b1e2f1f7 in ?? ()
> #1 0x7f60b1e308e8 in ?? ()
> #2 0x0020 in ?? ()
> #3 0x in ?? ()
> CORE: ./core.1547693849.30808.kudu-tserver
> BINARY: ./be/build/latest/service/impalad
> Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
[jira] [Work started] (IMPALA-8091) minicluster kudu failure: Cannot initialize clock: failed to wait for clock sync
[
https://issues.apache.org/jira/browse/IMPALA-8091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on IMPALA-8091 started by Michael Brown.
-
> minicluster kudu failure: Cannot initialize clock: failed to wait for clock
> sync
>
>
> Key: IMPALA-8091
> URL: https://issues.apache.org/jira/browse/IMPALA-8091
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
>Affects Versions: Impala 3.2.0
>Reporter: David Knupp
>Assignee: Michael Brown
>Priority: Blocker
> Labels: broken-build
>
> *Console log*:
> {noformat}
> 19:10:04 19:10:04 Error executing impala SQL:
> /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/logs/data_loading/sql/functional/create-functional-query-exhaustive-impala-generated-kudu-none-none.sql
> See:
> /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/logs/data_loading/sql/functional/create-functional-query-exhaustive-impala-generated-kudu-none-none.sql.log
> 19:10:04 Background task Loading functional-query data (pid 4831) failed.
> 19:10:04 Background task Loading TPC-H data (pid 4832) failed.
> 19:10:04 ERROR in
> /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/testdata/bin/create-load-data.sh
> at line 85: CM_HOST=${2-}
> [...]
> 19:10:05 2019-01-16 19:10:05,461 - archive_core_dumps - INFO - Found core
> files: ['./core.1547693849.30799.kudu-master',
> './core.1547693849.30783.kudu-tserver',
> './core.1547693849.30767.kudu-tserver',
> './core.1547693849.30808.kudu-tserver']
> 19:10:05 2019-01-16 19:10:05,649 - archive_core_dumps - INFO - [New LWP 30799]
> 19:10:05 [New LWP 30834]
> 19:10:05 [New LWP 30835]
> 19:10:05 [New LWP 30838]
> 19:10:05 [New LWP 30837]
> 19:10:05 [New LWP 30836]
> 19:10:05 Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
> 19:10:05 Program terminated with signal SIGABRT, Aborted.
> 19:10:05 #0 0x7f9a2cc611f7 in ?? ()
> 19:10:05
> 19:10:05 2019-01-16 19:10:05,650 - archive_core_dumps - INFO - Found binary
> path through GDB:
> /data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c
> 19:10:05 2019-01-16 19:10:05,893 - archive_core_dumps - WARNING - Failed to
> determine binary because multiple candidate binaries were found and none of
> their paths contained 'latest' to disambiguate:
> 19:10:05 Core:./core.1547693849.30799.kudu-master
> 19:10:05
> Binaries:['./testdata/cluster/node_templates/common/etc/init.d/kudu-master',
> './testdata/cluster/cdh6/node-1/etc/init.d/kudu-master']
> 19:10:05
> 19:10:05 2019-01-16 19:10:05,917 - archive_core_dumps - INFO - [New LWP 30783]
> 19:10:05 [New LWP 30810]
> 19:10:05 [New LWP 30812]
> 19:10:05 [New LWP 30811]
> 19:10:05 [New LWP 30824]
> 19:10:05 [New LWP 30820]
> 19:10:05 Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
> 19:10:05 Program terminated with signal SIGABRT, Aborted.
> 19:10:05 #0 0x7f0b81fb11f7 in ?? ()
> {noformat}
> *Backtraces*:
> {noformat}
> CORE: ./core.1547693849.30799.kudu-master
> BINARY: ./be/build/latest/service/impalad
> Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
> Program terminated with signal SIGABRT, Aborted.
> #0 0x7f9a2cc611f7 in ?? ()
> #0 0x7f9a2cc611f7 in ?? ()
> #1 0x7f9a2cc628e8 in ?? ()
> #2 0x0020 in ?? ()
> #3 0x in ?? ()
> CORE: ./core.1547693849.30783.kudu-tserver
> BINARY: ./be/build/latest/service/impalad
> Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
> Program terminated with signal SIGABRT, Aborted.
> #0 0x7f0b81fb11f7 in ?? ()
> #0 0x7f0b81fb11f7 in ?? ()
> #1 0x7f0b81fb28e8 in ?? ()
> #2 0x0020 in ?? ()
> #3 0x in ?? ()
> CORE: ./core.1547693849.30767.kudu-tserver
> BINARY: ./be/build/latest/service/impalad
> Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
> Program terminated with signal SIGABRT, Aborted.
> #0 0x7f60b1e2f1f7 in ?? ()
> #0 0x7f60b1e2f1f7 in ?? ()
> #1 0x7f60b1e308e8 in ?? ()
> #2 0x0020 in ?? ()
> #3 0x in ?? ()
> CORE: ./core.1547693849.30808.kudu-tserver
> BINARY: ./be/build/latest/service/impalad
> Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
> Program terminated with signal SIGABRT, Aborted.
> #0 0x7fa3cb5591f7 in ?? ()
> #0 0x7fa3cb5591f7 in ?? ()
> #1 0x7fa3cb55a8e8 in ?? ()
> #2 0x0020 in ?? ()
> #3 0x in ??
[jira] [Assigned] (IMPALA-8091) minicluster kudu failure: Cannot initialize clock: failed to wait for clock sync
[
https://issues.apache.org/jira/browse/IMPALA-8091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Brown reassigned IMPALA-8091:
-
Assignee: Michael Brown (was: Thomas Tauber-Marshall)
> minicluster kudu failure: Cannot initialize clock: failed to wait for clock
> sync
>
>
> Key: IMPALA-8091
> URL: https://issues.apache.org/jira/browse/IMPALA-8091
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
>Affects Versions: Impala 3.2.0
>Reporter: David Knupp
>Assignee: Michael Brown
>Priority: Blocker
> Labels: broken-build
>
> *Console log*:
> {noformat}
> 19:10:04 19:10:04 Error executing impala SQL:
> /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/logs/data_loading/sql/functional/create-functional-query-exhaustive-impala-generated-kudu-none-none.sql
> See:
> /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/logs/data_loading/sql/functional/create-functional-query-exhaustive-impala-generated-kudu-none-none.sql.log
> 19:10:04 Background task Loading functional-query data (pid 4831) failed.
> 19:10:04 Background task Loading TPC-H data (pid 4832) failed.
> 19:10:04 ERROR in
> /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/testdata/bin/create-load-data.sh
> at line 85: CM_HOST=${2-}
> [...]
> 19:10:05 2019-01-16 19:10:05,461 - archive_core_dumps - INFO - Found core
> files: ['./core.1547693849.30799.kudu-master',
> './core.1547693849.30783.kudu-tserver',
> './core.1547693849.30767.kudu-tserver',
> './core.1547693849.30808.kudu-tserver']
> 19:10:05 2019-01-16 19:10:05,649 - archive_core_dumps - INFO - [New LWP 30799]
> 19:10:05 [New LWP 30834]
> 19:10:05 [New LWP 30835]
> 19:10:05 [New LWP 30838]
> 19:10:05 [New LWP 30837]
> 19:10:05 [New LWP 30836]
> 19:10:05 Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
> 19:10:05 Program terminated with signal SIGABRT, Aborted.
> 19:10:05 #0 0x7f9a2cc611f7 in ?? ()
> 19:10:05
> 19:10:05 2019-01-16 19:10:05,650 - archive_core_dumps - INFO - Found binary
> path through GDB:
> /data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c
> 19:10:05 2019-01-16 19:10:05,893 - archive_core_dumps - WARNING - Failed to
> determine binary because multiple candidate binaries were found and none of
> their paths contained 'latest' to disambiguate:
> 19:10:05 Core:./core.1547693849.30799.kudu-master
> 19:10:05
> Binaries:['./testdata/cluster/node_templates/common/etc/init.d/kudu-master',
> './testdata/cluster/cdh6/node-1/etc/init.d/kudu-master']
> 19:10:05
> 19:10:05 2019-01-16 19:10:05,917 - archive_core_dumps - INFO - [New LWP 30783]
> 19:10:05 [New LWP 30810]
> 19:10:05 [New LWP 30812]
> 19:10:05 [New LWP 30811]
> 19:10:05 [New LWP 30824]
> 19:10:05 [New LWP 30820]
> 19:10:05 Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
> 19:10:05 Program terminated with signal SIGABRT, Aborted.
> 19:10:05 #0 0x7f0b81fb11f7 in ?? ()
> {noformat}
> *Backtraces*:
> {noformat}
> CORE: ./core.1547693849.30799.kudu-master
> BINARY: ./be/build/latest/service/impalad
> Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
> Program terminated with signal SIGABRT, Aborted.
> #0 0x7f9a2cc611f7 in ?? ()
> #0 0x7f9a2cc611f7 in ?? ()
> #1 0x7f9a2cc628e8 in ?? ()
> #2 0x0020 in ?? ()
> #3 0x in ?? ()
> CORE: ./core.1547693849.30783.kudu-tserver
> BINARY: ./be/build/latest/service/impalad
> Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
> Program terminated with signal SIGABRT, Aborted.
> #0 0x7f0b81fb11f7 in ?? ()
> #0 0x7f0b81fb11f7 in ?? ()
> #1 0x7f0b81fb28e8 in ?? ()
> #2 0x0020 in ?? ()
> #3 0x in ?? ()
> CORE: ./core.1547693849.30767.kudu-tserver
> BINARY: ./be/build/latest/service/impalad
> Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
> Program terminated with signal SIGABRT, Aborted.
> #0 0x7f60b1e2f1f7 in ?? ()
> #0 0x7f60b1e2f1f7 in ?? ()
> #1 0x7f60b1e308e8 in ?? ()
> #2 0x0020 in ?? ()
> #3 0x in ?? ()
> CORE: ./core.1547693849.30808.kudu-tserver
> BINARY: ./be/build/latest/service/impalad
> Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
> Program terminated with signal SIGABRT, Aborted.
> #0 0x7fa3cb5591f7 in ?? ()
> #0 0x7fa3cb5591f7 in ?? ()
> #1 0x7fa3cb55a8e8 in ?? ()
> #2
[jira] [Commented] (IMPALA-8091) minicluster kudu failure: Cannot initialize clock: failed to wait for clock sync
[
https://issues.apache.org/jira/browse/IMPALA-8091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16749232#comment-16749232
]
Michael Brown commented on IMPALA-8091:
---
Relevant code in {{testdata/cluster/admin}}:
{noformat}
# line 333
if [[ "${SERVICE}" == "kudu" ]]; then
NTP_CANARY=pool.ntp.org
if ! ping -c 1 -w 5 "${NTP_CANARY}" >/dev/null 2>/dev/null; then
echo "WARNING: cannot reach ${NTP_CANARY}; ntp sync recommended for
Kudu"
elif ntp-wait --help >/dev/null 2>/dev/null && ! ntp-wait -v; then
echo "ntp-wait failed; cannot start kudu"
return 1
fi
fi
{noformat}
We silently move on if ntp-wait isn't present. We should warn. We don't try to
install ntp-wait, which I get, because it's rude to do that to somebody's
system. We should install ntp-perl for CentOS 7 in bootstrap_system.sh.
> minicluster kudu failure: Cannot initialize clock: failed to wait for clock
> sync
>
>
> Key: IMPALA-8091
> URL: https://issues.apache.org/jira/browse/IMPALA-8091
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
>Affects Versions: Impala 3.2.0
>Reporter: David Knupp
>Assignee: Thomas Tauber-Marshall
>Priority: Blocker
> Labels: broken-build
>
> *Console log*:
> {noformat}
> 19:10:04 19:10:04 Error executing impala SQL:
> /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/logs/data_loading/sql/functional/create-functional-query-exhaustive-impala-generated-kudu-none-none.sql
> See:
> /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/logs/data_loading/sql/functional/create-functional-query-exhaustive-impala-generated-kudu-none-none.sql.log
> 19:10:04 Background task Loading functional-query data (pid 4831) failed.
> 19:10:04 Background task Loading TPC-H data (pid 4832) failed.
> 19:10:04 ERROR in
> /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/testdata/bin/create-load-data.sh
> at line 85: CM_HOST=${2-}
> [...]
> 19:10:05 2019-01-16 19:10:05,461 - archive_core_dumps - INFO - Found core
> files: ['./core.1547693849.30799.kudu-master',
> './core.1547693849.30783.kudu-tserver',
> './core.1547693849.30767.kudu-tserver',
> './core.1547693849.30808.kudu-tserver']
> 19:10:05 2019-01-16 19:10:05,649 - archive_core_dumps - INFO - [New LWP 30799]
> 19:10:05 [New LWP 30834]
> 19:10:05 [New LWP 30835]
> 19:10:05 [New LWP 30838]
> 19:10:05 [New LWP 30837]
> 19:10:05 [New LWP 30836]
> 19:10:05 Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
> 19:10:05 Program terminated with signal SIGABRT, Aborted.
> 19:10:05 #0 0x7f9a2cc611f7 in ?? ()
> 19:10:05
> 19:10:05 2019-01-16 19:10:05,650 - archive_core_dumps - INFO - Found binary
> path through GDB:
> /data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c
> 19:10:05 2019-01-16 19:10:05,893 - archive_core_dumps - WARNING - Failed to
> determine binary because multiple candidate binaries were found and none of
> their paths contained 'latest' to disambiguate:
> 19:10:05 Core:./core.1547693849.30799.kudu-master
> 19:10:05
> Binaries:['./testdata/cluster/node_templates/common/etc/init.d/kudu-master',
> './testdata/cluster/cdh6/node-1/etc/init.d/kudu-master']
> 19:10:05
> 19:10:05 2019-01-16 19:10:05,917 - archive_core_dumps - INFO - [New LWP 30783]
> 19:10:05 [New LWP 30810]
> 19:10:05 [New LWP 30812]
> 19:10:05 [New LWP 30811]
> 19:10:05 [New LWP 30824]
> 19:10:05 [New LWP 30820]
> 19:10:05 Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
> 19:10:05 Program terminated with signal SIGABRT, Aborted.
> 19:10:05 #0 0x7f0b81fb11f7 in ?? ()
> {noformat}
> *Backtraces*:
> {noformat}
> CORE: ./core.1547693849.30799.kudu-master
> BINARY: ./be/build/latest/service/impalad
> Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
> Program terminated with signal SIGABRT, Aborted.
> #0 0x7f9a2cc611f7 in ?? ()
> #0 0x7f9a2cc611f7 in ?? ()
> #1 0x7f9a2cc628e8 in ?? ()
> #2 0x0020 in ?? ()
> #3 0x in ?? ()
> CORE: ./core.1547693849.30783.kudu-tserver
> BINARY: ./be/build/latest/service/impalad
> Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
> Program terminated with signal SIGABRT, Aborted.
> #0 0x7f0b81fb11f7 in ?? ()
> #0 0x7f0b81fb11f7 in ?? ()
> #1 0x7f0b81fb28e8 in ?? ()
> #2 0x0020 in ?? ()
> #3 0x in ?? ()
> CORE: ./core.1547693849.30767.kudu-tserver
> BINARY: ./be/build/latest/service/impalad
> Core
[jira] [Commented] (IMPALA-8091) minicluster kudu failure: Cannot initialize clock: failed to wait for clock sync
[
https://issues.apache.org/jira/browse/IMPALA-8091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16749190#comment-16749190
]
Michael Brown commented on IMPALA-8091:
---
On Ubuntu 16, the {{ntp}} package provides {{ntp-wait}}. On CentOS 7, it's
provided by {{ntp-perl}}. This is happening in a downstream environment where
{{ntp-perl}} is not installed. On the same downstream environment, {{chrony}}
seems to be installed but isn't running a daemon (ok). I'm looking to see if
there is an upstream improvement that can be made here.
> minicluster kudu failure: Cannot initialize clock: failed to wait for clock
> sync
>
>
> Key: IMPALA-8091
> URL: https://issues.apache.org/jira/browse/IMPALA-8091
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
>Affects Versions: Impala 3.2.0
>Reporter: David Knupp
>Assignee: Thomas Tauber-Marshall
>Priority: Blocker
> Labels: broken-build
>
> *Console log*:
> {noformat}
> 19:10:04 19:10:04 Error executing impala SQL:
> /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/logs/data_loading/sql/functional/create-functional-query-exhaustive-impala-generated-kudu-none-none.sql
> See:
> /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/logs/data_loading/sql/functional/create-functional-query-exhaustive-impala-generated-kudu-none-none.sql.log
> 19:10:04 Background task Loading functional-query data (pid 4831) failed.
> 19:10:04 Background task Loading TPC-H data (pid 4832) failed.
> 19:10:04 ERROR in
> /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/testdata/bin/create-load-data.sh
> at line 85: CM_HOST=${2-}
> [...]
> 19:10:05 2019-01-16 19:10:05,461 - archive_core_dumps - INFO - Found core
> files: ['./core.1547693849.30799.kudu-master',
> './core.1547693849.30783.kudu-tserver',
> './core.1547693849.30767.kudu-tserver',
> './core.1547693849.30808.kudu-tserver']
> 19:10:05 2019-01-16 19:10:05,649 - archive_core_dumps - INFO - [New LWP 30799]
> 19:10:05 [New LWP 30834]
> 19:10:05 [New LWP 30835]
> 19:10:05 [New LWP 30838]
> 19:10:05 [New LWP 30837]
> 19:10:05 [New LWP 30836]
> 19:10:05 Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
> 19:10:05 Program terminated with signal SIGABRT, Aborted.
> 19:10:05 #0 0x7f9a2cc611f7 in ?? ()
> 19:10:05
> 19:10:05 2019-01-16 19:10:05,650 - archive_core_dumps - INFO - Found binary
> path through GDB:
> /data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c
> 19:10:05 2019-01-16 19:10:05,893 - archive_core_dumps - WARNING - Failed to
> determine binary because multiple candidate binaries were found and none of
> their paths contained 'latest' to disambiguate:
> 19:10:05 Core:./core.1547693849.30799.kudu-master
> 19:10:05
> Binaries:['./testdata/cluster/node_templates/common/etc/init.d/kudu-master',
> './testdata/cluster/cdh6/node-1/etc/init.d/kudu-master']
> 19:10:05
> 19:10:05 2019-01-16 19:10:05,917 - archive_core_dumps - INFO - [New LWP 30783]
> 19:10:05 [New LWP 30810]
> 19:10:05 [New LWP 30812]
> 19:10:05 [New LWP 30811]
> 19:10:05 [New LWP 30824]
> 19:10:05 [New LWP 30820]
> 19:10:05 Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
> 19:10:05 Program terminated with signal SIGABRT, Aborted.
> 19:10:05 #0 0x7f0b81fb11f7 in ?? ()
> {noformat}
> *Backtraces*:
> {noformat}
> CORE: ./core.1547693849.30799.kudu-master
> BINARY: ./be/build/latest/service/impalad
> Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
> Program terminated with signal SIGABRT, Aborted.
> #0 0x7f9a2cc611f7 in ?? ()
> #0 0x7f9a2cc611f7 in ?? ()
> #1 0x7f9a2cc628e8 in ?? ()
> #2 0x0020 in ?? ()
> #3 0x in ?? ()
> CORE: ./core.1547693849.30783.kudu-tserver
> BINARY: ./be/build/latest/service/impalad
> Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
> Program terminated with signal SIGABRT, Aborted.
> #0 0x7f0b81fb11f7 in ?? ()
> #0 0x7f0b81fb11f7 in ?? ()
> #1 0x7f0b81fb28e8 in ?? ()
> #2 0x0020 in ?? ()
> #3 0x in ?? ()
> CORE: ./core.1547693849.30767.kudu-tserver
> BINARY: ./be/build/latest/service/impalad
> Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
> Program terminated with signal SIGABRT, Aborted.
> #0 0x7f60b1e2f1f7 in ?? ()
> #0 0x7f60b1e2f1f7 in ?? ()
> #1 0x7f60b1e308e8 in ?? ()
> #2 0x0020 in ?? ()
> #3 0x in ?? ()
> CORE:
[jira] [Commented] (IMPALA-8091) minicluster kudu failure: Cannot initialize clock: failed to wait for clock sync
[
https://issues.apache.org/jira/browse/IMPALA-8091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16749176#comment-16749176
]
Michael Brown commented on IMPALA-8091:
---
This affected numerous runs over a quiet weekend without much commit churn.
> minicluster kudu failure: Cannot initialize clock: failed to wait for clock
> sync
>
>
> Key: IMPALA-8091
> URL: https://issues.apache.org/jira/browse/IMPALA-8091
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
>Affects Versions: Impala 3.2.0
>Reporter: David Knupp
>Assignee: Thomas Tauber-Marshall
>Priority: Blocker
> Labels: broken-build
>
> *Console log*:
> {noformat}
> 19:10:04 19:10:04 Error executing impala SQL:
> /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/logs/data_loading/sql/functional/create-functional-query-exhaustive-impala-generated-kudu-none-none.sql
> See:
> /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/logs/data_loading/sql/functional/create-functional-query-exhaustive-impala-generated-kudu-none-none.sql.log
> 19:10:04 Background task Loading functional-query data (pid 4831) failed.
> 19:10:04 Background task Loading TPC-H data (pid 4832) failed.
> 19:10:04 ERROR in
> /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/testdata/bin/create-load-data.sh
> at line 85: CM_HOST=${2-}
> [...]
> 19:10:05 2019-01-16 19:10:05,461 - archive_core_dumps - INFO - Found core
> files: ['./core.1547693849.30799.kudu-master',
> './core.1547693849.30783.kudu-tserver',
> './core.1547693849.30767.kudu-tserver',
> './core.1547693849.30808.kudu-tserver']
> 19:10:05 2019-01-16 19:10:05,649 - archive_core_dumps - INFO - [New LWP 30799]
> 19:10:05 [New LWP 30834]
> 19:10:05 [New LWP 30835]
> 19:10:05 [New LWP 30838]
> 19:10:05 [New LWP 30837]
> 19:10:05 [New LWP 30836]
> 19:10:05 Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
> 19:10:05 Program terminated with signal SIGABRT, Aborted.
> 19:10:05 #0 0x7f9a2cc611f7 in ?? ()
> 19:10:05
> 19:10:05 2019-01-16 19:10:05,650 - archive_core_dumps - INFO - Found binary
> path through GDB:
> /data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c
> 19:10:05 2019-01-16 19:10:05,893 - archive_core_dumps - WARNING - Failed to
> determine binary because multiple candidate binaries were found and none of
> their paths contained 'latest' to disambiguate:
> 19:10:05 Core:./core.1547693849.30799.kudu-master
> 19:10:05
> Binaries:['./testdata/cluster/node_templates/common/etc/init.d/kudu-master',
> './testdata/cluster/cdh6/node-1/etc/init.d/kudu-master']
> 19:10:05
> 19:10:05 2019-01-16 19:10:05,917 - archive_core_dumps - INFO - [New LWP 30783]
> 19:10:05 [New LWP 30810]
> 19:10:05 [New LWP 30812]
> 19:10:05 [New LWP 30811]
> 19:10:05 [New LWP 30824]
> 19:10:05 [New LWP 30820]
> 19:10:05 Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
> 19:10:05 Program terminated with signal SIGABRT, Aborted.
> 19:10:05 #0 0x7f0b81fb11f7 in ?? ()
> {noformat}
> *Backtraces*:
> {noformat}
> CORE: ./core.1547693849.30799.kudu-master
> BINARY: ./be/build/latest/service/impalad
> Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
> Program terminated with signal SIGABRT, Aborted.
> #0 0x7f9a2cc611f7 in ?? ()
> #0 0x7f9a2cc611f7 in ?? ()
> #1 0x7f9a2cc628e8 in ?? ()
> #2 0x0020 in ?? ()
> #3 0x in ?? ()
> CORE: ./core.1547693849.30783.kudu-tserver
> BINARY: ./be/build/latest/service/impalad
> Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
> Program terminated with signal SIGABRT, Aborted.
> #0 0x7f0b81fb11f7 in ?? ()
> #0 0x7f0b81fb11f7 in ?? ()
> #1 0x7f0b81fb28e8 in ?? ()
> #2 0x0020 in ?? ()
> #3 0x in ?? ()
> CORE: ./core.1547693849.30767.kudu-tserver
> BINARY: ./be/build/latest/service/impalad
> Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
> Program terminated with signal SIGABRT, Aborted.
> #0 0x7f60b1e2f1f7 in ?? ()
> #0 0x7f60b1e2f1f7 in ?? ()
> #1 0x7f60b1e308e8 in ?? ()
> #2 0x0020 in ?? ()
> #3 0x in ?? ()
> CORE: ./core.1547693849.30808.kudu-tserver
> BINARY: ./be/build/latest/service/impalad
> Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
> Program terminated with signal SIGABRT, Aborted.
> #0 0x7fa3cb5591f7 in ?? ()
> #0 0x7fa3cb5591f7 in
[jira] [Updated] (IMPALA-8091) minicluster kudu failure: Cannot initialize clock: failed to wait for clock sync
[
https://issues.apache.org/jira/browse/IMPALA-8091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Brown updated IMPALA-8091:
--
Priority: Blocker (was: Critical)
> minicluster kudu failure: Cannot initialize clock: failed to wait for clock
> sync
>
>
> Key: IMPALA-8091
> URL: https://issues.apache.org/jira/browse/IMPALA-8091
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
>Affects Versions: Impala 3.2.0
>Reporter: David Knupp
>Assignee: Thomas Tauber-Marshall
>Priority: Blocker
> Labels: broken-build
>
> *Console log*:
> {noformat}
> 19:10:04 19:10:04 Error executing impala SQL:
> /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/logs/data_loading/sql/functional/create-functional-query-exhaustive-impala-generated-kudu-none-none.sql
> See:
> /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/logs/data_loading/sql/functional/create-functional-query-exhaustive-impala-generated-kudu-none-none.sql.log
> 19:10:04 Background task Loading functional-query data (pid 4831) failed.
> 19:10:04 Background task Loading TPC-H data (pid 4832) failed.
> 19:10:04 ERROR in
> /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/testdata/bin/create-load-data.sh
> at line 85: CM_HOST=${2-}
> [...]
> 19:10:05 2019-01-16 19:10:05,461 - archive_core_dumps - INFO - Found core
> files: ['./core.1547693849.30799.kudu-master',
> './core.1547693849.30783.kudu-tserver',
> './core.1547693849.30767.kudu-tserver',
> './core.1547693849.30808.kudu-tserver']
> 19:10:05 2019-01-16 19:10:05,649 - archive_core_dumps - INFO - [New LWP 30799]
> 19:10:05 [New LWP 30834]
> 19:10:05 [New LWP 30835]
> 19:10:05 [New LWP 30838]
> 19:10:05 [New LWP 30837]
> 19:10:05 [New LWP 30836]
> 19:10:05 Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
> 19:10:05 Program terminated with signal SIGABRT, Aborted.
> 19:10:05 #0 0x7f9a2cc611f7 in ?? ()
> 19:10:05
> 19:10:05 2019-01-16 19:10:05,650 - archive_core_dumps - INFO - Found binary
> path through GDB:
> /data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c
> 19:10:05 2019-01-16 19:10:05,893 - archive_core_dumps - WARNING - Failed to
> determine binary because multiple candidate binaries were found and none of
> their paths contained 'latest' to disambiguate:
> 19:10:05 Core:./core.1547693849.30799.kudu-master
> 19:10:05
> Binaries:['./testdata/cluster/node_templates/common/etc/init.d/kudu-master',
> './testdata/cluster/cdh6/node-1/etc/init.d/kudu-master']
> 19:10:05
> 19:10:05 2019-01-16 19:10:05,917 - archive_core_dumps - INFO - [New LWP 30783]
> 19:10:05 [New LWP 30810]
> 19:10:05 [New LWP 30812]
> 19:10:05 [New LWP 30811]
> 19:10:05 [New LWP 30824]
> 19:10:05 [New LWP 30820]
> 19:10:05 Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
> 19:10:05 Program terminated with signal SIGABRT, Aborted.
> 19:10:05 #0 0x7f0b81fb11f7 in ?? ()
> {noformat}
> *Backtraces*:
> {noformat}
> CORE: ./core.1547693849.30799.kudu-master
> BINARY: ./be/build/latest/service/impalad
> Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
> Program terminated with signal SIGABRT, Aborted.
> #0 0x7f9a2cc611f7 in ?? ()
> #0 0x7f9a2cc611f7 in ?? ()
> #1 0x7f9a2cc628e8 in ?? ()
> #2 0x0020 in ?? ()
> #3 0x in ?? ()
> CORE: ./core.1547693849.30783.kudu-tserver
> BINARY: ./be/build/latest/service/impalad
> Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
> Program terminated with signal SIGABRT, Aborted.
> #0 0x7f0b81fb11f7 in ?? ()
> #0 0x7f0b81fb11f7 in ?? ()
> #1 0x7f0b81fb28e8 in ?? ()
> #2 0x0020 in ?? ()
> #3 0x in ?? ()
> CORE: ./core.1547693849.30767.kudu-tserver
> BINARY: ./be/build/latest/service/impalad
> Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
> Program terminated with signal SIGABRT, Aborted.
> #0 0x7f60b1e2f1f7 in ?? ()
> #0 0x7f60b1e2f1f7 in ?? ()
> #1 0x7f60b1e308e8 in ?? ()
> #2 0x0020 in ?? ()
> #3 0x in ?? ()
> CORE: ./core.1547693849.30808.kudu-tserver
> BINARY: ./be/build/latest/service/impalad
> Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
> Program terminated with signal SIGABRT, Aborted.
> #0 0x7fa3cb5591f7 in ?? ()
> #0 0x7fa3cb5591f7 in ?? ()
> #1 0x7fa3cb55a8e8 in ?? ()
> #2 0x0020 in ?? ()
>
[jira] [Commented] (IMPALA-8091) minicluster kudu failure: Cannot initialize clock: failed to wait for clock sync
[
https://issues.apache.org/jira/browse/IMPALA-8091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16749157#comment-16749157
]
Michael Brown commented on IMPALA-8091:
---
This is a lot of diagnostic info...
{noformat}
I0116 18:56:30.146215 30799 master_main.cc:80] Initializing master server...
I0116 18:56:30.146333 30799 system_ntp.cc:122] Waiting up to
--ntp_initial_sync_wait_secs=60 seconds for the clock to synchronize
W0116 18:56:30.147728 30799 system_ntp.cc:132] Could not find ntp-wait; trying
chrony waitsync instead: Not found: Unable to find binary: ntp-wait
E0116 18:57:29.300457 30799 system_ntp.cc:157] Dumping NTP diagnostics
E0116 18:57:29.302417 30799 system_ntp.cc:102] /sbin/ntptime
--
stdout:
ntp_gettime() returns code 5 (ERROR)
time dfea6d99.4d59a2ec Wed, Jan 16 2019 18:57:29.302, (.302149739),
maximum error 1600 us, estimated error 1600 us, TAI offset 0
ntp_adjtime() returns code 5 (ERROR)
modes 0x0 (),
offset 0.000 us, frequency 129.100 ppm, interval 1 s,
maximum error 1600 us, estimated error 1600 us,
status 0x2041 (PLL,UNSYNC,NANO),
time constant 6, precision 0.001 us, tolerance 500 ppm,
E0116 18:57:29.330603 30799 system_ntp.cc:102] /sbin/ntpq -n -c timeout 1000 -c
readvar -c sysinfo -c lpeers -c opeers -c version
--
stdout:
associd=0 status=c618 leap_alarm, sync_ntp, 1 event, no_sys_peer,
version="ntpd 4.2.6p5@1.2349-o Wed Apr 12 21:24:06 UTC 2017 (1)",
processor="x86_64", system="Linux/3.10.0-693.5.2.el7.x86_64", leap=11,
stratum=2, precision=-24, rootdelay=65.438, rootdisp=43.467,
refid=96.126.122.39,
reftime=dfea6d72.e0d7d2b9 Wed, Jan 16 2019 18:56:50.878,
clock=dfea6d99.4f1265e8 Wed, Jan 16 2019 18:57:29.308, peer=64381, tc=6,
mintc=3, offset=0.000, frequency=129.100, sys_jitter=11.220,
clk_jitter=12.567, clk_wander=0.221
remote refid st t when poll reach delay offset jitter
==
+66.228.48.38127.67.113.922 u 39 647 70.236 -35.592 28.619
+208.75.89.4 63.145.169.3 2 u 37 647 64.246 -20.776 29.110
*96.126.122.39 .GPS.1 u 38 647 63.424 -31.810 28.640
+69.89.207.199 212.215.1.1572 u 36 647 37.729 -37.207 28.643
remote local st t when poll reach delay offsetdisp
==
+66.228.48.38172.28.204.722 u 39 647 70.236 -35.592 0.913
+208.75.89.4 172.28.204.722 u 37 647 64.246 -20.776 0.921
*96.126.122.39 172.28.204.721 u 38 647 63.424 -31.810 0.906
+69.89.207.199 172.28.204.722 u 36 647 37.729 -37.207 0.914
ntpq 4.2.6p5@1.2349-o Wed Apr 12 21:24:08 UTC 2017 (1)
stderr:
***Command `sysinfo' unknown
E0116 18:57:29.333056 30799 system_ntp.cc:102] /sbin/ntpdc -n -c timeout 1000
-c peers -c sysinfo -c sysstats -c version
--
stdout:
remote local st poll reach delay offsetdisp
===
*96.126.122.39 172.28.204.721 647 0.06342 -0.031810 0.03009
=208.75.89.4 172.28.204.722 647 0.06424 -0.020776 0.03033
=69.89.207.199 172.28.204.722 647 0.03772 -0.037207 0.03023
=66.228.48.38172.28.204.722 647 0.07022 -0.035592 0.03021
system peer: 96.126.122.39
system peer mode: client
leap indicator: 11
stratum: 2
precision:-24
root distance:0.06543 s
root dispersion: 0.04346 s
reference ID: [96.126.122.39]
reference time: dfea6d72.e0d7d2b9 Wed, Jan 16 2019 18:56:50.878
system flags: auth ntp kernel stats
jitter: 0.011215 s
stability:0.000 ppm
broadcastdelay: 0.00 s
authdelay:0.00 s
time since restart: 1459
time since reset: 1459
packets received: 180
packets processed: 124
current version:124
previous version: 0
declined: 0
access denied: 0
bad length or format: 0
bad authentication: 0
rate exceeded: 0
ntpdc 4.2.6p5@1.2349-o Wed Apr 12 21:24:08 UTC 2017 (1)
E0116 18:57:29.336165 30799 system_ntp.cc:102] /usr/bin/chronyc -n tracking
--
stdout:
506 Cannot talk to daemon
E0116 18:57:29.339103 30799 system_ntp.cc:102] /usr/bin/chronyc -n sources
--
stdout:
506 Cannot talk to daemon
F0116 18:57:29.339143 30799 master_main.cc:81] Check failed: _s.ok() Bad
status: Runtime error: Cannot initialize clock: failed to wait for clock sync
using command '/usr/bin/chronyc waitsync 60 0 0 1': /usr/bin/chronyc: process
exited
[jira] [Updated] (IMPALA-8091) minicluster kudu failure: Cannot initialize clock: failed to wait for clock sync
[
https://issues.apache.org/jira/browse/IMPALA-8091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Brown updated IMPALA-8091:
--
Component/s: (was: Backend)
Infrastructure
> minicluster kudu failure: Cannot initialize clock: failed to wait for clock
> sync
>
>
> Key: IMPALA-8091
> URL: https://issues.apache.org/jira/browse/IMPALA-8091
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
>Affects Versions: Impala 3.2.0
>Reporter: David Knupp
>Assignee: Thomas Tauber-Marshall
>Priority: Critical
> Labels: broken-build
>
> *Console log*:
> {noformat}
> 19:10:04 19:10:04 Error executing impala SQL:
> /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/logs/data_loading/sql/functional/create-functional-query-exhaustive-impala-generated-kudu-none-none.sql
> See:
> /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/logs/data_loading/sql/functional/create-functional-query-exhaustive-impala-generated-kudu-none-none.sql.log
> 19:10:04 Background task Loading functional-query data (pid 4831) failed.
> 19:10:04 Background task Loading TPC-H data (pid 4832) failed.
> 19:10:04 ERROR in
> /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/testdata/bin/create-load-data.sh
> at line 85: CM_HOST=${2-}
> [...]
> 19:10:05 2019-01-16 19:10:05,461 - archive_core_dumps - INFO - Found core
> files: ['./core.1547693849.30799.kudu-master',
> './core.1547693849.30783.kudu-tserver',
> './core.1547693849.30767.kudu-tserver',
> './core.1547693849.30808.kudu-tserver']
> 19:10:05 2019-01-16 19:10:05,649 - archive_core_dumps - INFO - [New LWP 30799]
> 19:10:05 [New LWP 30834]
> 19:10:05 [New LWP 30835]
> 19:10:05 [New LWP 30838]
> 19:10:05 [New LWP 30837]
> 19:10:05 [New LWP 30836]
> 19:10:05 Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
> 19:10:05 Program terminated with signal SIGABRT, Aborted.
> 19:10:05 #0 0x7f9a2cc611f7 in ?? ()
> 19:10:05
> 19:10:05 2019-01-16 19:10:05,650 - archive_core_dumps - INFO - Found binary
> path through GDB:
> /data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c
> 19:10:05 2019-01-16 19:10:05,893 - archive_core_dumps - WARNING - Failed to
> determine binary because multiple candidate binaries were found and none of
> their paths contained 'latest' to disambiguate:
> 19:10:05 Core:./core.1547693849.30799.kudu-master
> 19:10:05
> Binaries:['./testdata/cluster/node_templates/common/etc/init.d/kudu-master',
> './testdata/cluster/cdh6/node-1/etc/init.d/kudu-master']
> 19:10:05
> 19:10:05 2019-01-16 19:10:05,917 - archive_core_dumps - INFO - [New LWP 30783]
> 19:10:05 [New LWP 30810]
> 19:10:05 [New LWP 30812]
> 19:10:05 [New LWP 30811]
> 19:10:05 [New LWP 30824]
> 19:10:05 [New LWP 30820]
> 19:10:05 Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
> 19:10:05 Program terminated with signal SIGABRT, Aborted.
> 19:10:05 #0 0x7f0b81fb11f7 in ?? ()
> {noformat}
> *Backtraces*:
> {noformat}
> CORE: ./core.1547693849.30799.kudu-master
> BINARY: ./be/build/latest/service/impalad
> Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
> Program terminated with signal SIGABRT, Aborted.
> #0 0x7f9a2cc611f7 in ?? ()
> #0 0x7f9a2cc611f7 in ?? ()
> #1 0x7f9a2cc628e8 in ?? ()
> #2 0x0020 in ?? ()
> #3 0x in ?? ()
> CORE: ./core.1547693849.30783.kudu-tserver
> BINARY: ./be/build/latest/service/impalad
> Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
> Program terminated with signal SIGABRT, Aborted.
> #0 0x7f0b81fb11f7 in ?? ()
> #0 0x7f0b81fb11f7 in ?? ()
> #1 0x7f0b81fb28e8 in ?? ()
> #2 0x0020 in ?? ()
> #3 0x in ?? ()
> CORE: ./core.1547693849.30767.kudu-tserver
> BINARY: ./be/build/latest/service/impalad
> Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
> Program terminated with signal SIGABRT, Aborted.
> #0 0x7f60b1e2f1f7 in ?? ()
> #0 0x7f60b1e2f1f7 in ?? ()
> #1 0x7f60b1e308e8 in ?? ()
> #2 0x0020 in ?? ()
> #3 0x in ?? ()
> CORE: ./core.1547693849.30808.kudu-tserver
> BINARY: ./be/build/latest/service/impalad
> Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
> Program terminated with signal SIGABRT, Aborted.
> #0 0x7fa3cb5591f7 in ?? ()
> #0 0x7fa3cb5591f7 in ?? ()
> #1 0x7fa3cb55a8e8 in ?? ()
> #2
[jira] [Updated] (IMPALA-8091) minicluster kudu failure: Cannot initialize clock: failed to wait for clock sync
[
https://issues.apache.org/jira/browse/IMPALA-8091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Brown updated IMPALA-8091:
--
Summary: minicluster kudu failure: Cannot initialize clock: failed to wait
for clock sync (was: Kudu SIGABRT'ed during dataload on Impala release build)
> minicluster kudu failure: Cannot initialize clock: failed to wait for clock
> sync
>
>
> Key: IMPALA-8091
> URL: https://issues.apache.org/jira/browse/IMPALA-8091
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 3.2.0
>Reporter: David Knupp
>Assignee: Thomas Tauber-Marshall
>Priority: Critical
> Labels: broken-build
>
> *Console log*:
> {noformat}
> 19:10:04 19:10:04 Error executing impala SQL:
> /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/logs/data_loading/sql/functional/create-functional-query-exhaustive-impala-generated-kudu-none-none.sql
> See:
> /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/logs/data_loading/sql/functional/create-functional-query-exhaustive-impala-generated-kudu-none-none.sql.log
> 19:10:04 Background task Loading functional-query data (pid 4831) failed.
> 19:10:04 Background task Loading TPC-H data (pid 4832) failed.
> 19:10:04 ERROR in
> /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/testdata/bin/create-load-data.sh
> at line 85: CM_HOST=${2-}
> [...]
> 19:10:05 2019-01-16 19:10:05,461 - archive_core_dumps - INFO - Found core
> files: ['./core.1547693849.30799.kudu-master',
> './core.1547693849.30783.kudu-tserver',
> './core.1547693849.30767.kudu-tserver',
> './core.1547693849.30808.kudu-tserver']
> 19:10:05 2019-01-16 19:10:05,649 - archive_core_dumps - INFO - [New LWP 30799]
> 19:10:05 [New LWP 30834]
> 19:10:05 [New LWP 30835]
> 19:10:05 [New LWP 30838]
> 19:10:05 [New LWP 30837]
> 19:10:05 [New LWP 30836]
> 19:10:05 Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
> 19:10:05 Program terminated with signal SIGABRT, Aborted.
> 19:10:05 #0 0x7f9a2cc611f7 in ?? ()
> 19:10:05
> 19:10:05 2019-01-16 19:10:05,650 - archive_core_dumps - INFO - Found binary
> path through GDB:
> /data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c
> 19:10:05 2019-01-16 19:10:05,893 - archive_core_dumps - WARNING - Failed to
> determine binary because multiple candidate binaries were found and none of
> their paths contained 'latest' to disambiguate:
> 19:10:05 Core:./core.1547693849.30799.kudu-master
> 19:10:05
> Binaries:['./testdata/cluster/node_templates/common/etc/init.d/kudu-master',
> './testdata/cluster/cdh6/node-1/etc/init.d/kudu-master']
> 19:10:05
> 19:10:05 2019-01-16 19:10:05,917 - archive_core_dumps - INFO - [New LWP 30783]
> 19:10:05 [New LWP 30810]
> 19:10:05 [New LWP 30812]
> 19:10:05 [New LWP 30811]
> 19:10:05 [New LWP 30824]
> 19:10:05 [New LWP 30820]
> 19:10:05 Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
> 19:10:05 Program terminated with signal SIGABRT, Aborted.
> 19:10:05 #0 0x7f0b81fb11f7 in ?? ()
> {noformat}
> *Backtraces*:
> {noformat}
> CORE: ./core.1547693849.30799.kudu-master
> BINARY: ./be/build/latest/service/impalad
> Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
> Program terminated with signal SIGABRT, Aborted.
> #0 0x7f9a2cc611f7 in ?? ()
> #0 0x7f9a2cc611f7 in ?? ()
> #1 0x7f9a2cc628e8 in ?? ()
> #2 0x0020 in ?? ()
> #3 0x in ?? ()
> CORE: ./core.1547693849.30783.kudu-tserver
> BINARY: ./be/build/latest/service/impalad
> Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
> Program terminated with signal SIGABRT, Aborted.
> #0 0x7f0b81fb11f7 in ?? ()
> #0 0x7f0b81fb11f7 in ?? ()
> #1 0x7f0b81fb28e8 in ?? ()
> #2 0x0020 in ?? ()
> #3 0x in ?? ()
> CORE: ./core.1547693849.30767.kudu-tserver
> BINARY: ./be/build/latest/service/impalad
> Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
> Program terminated with signal SIGABRT, Aborted.
> #0 0x7f60b1e2f1f7 in ?? ()
> #0 0x7f60b1e2f1f7 in ?? ()
> #1 0x7f60b1e308e8 in ?? ()
> #2 0x0020 in ?? ()
> #3 0x in ?? ()
> CORE: ./core.1547693849.30808.kudu-tserver
> BINARY: ./be/build/latest/service/impalad
> Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
> Program terminated with signal SIGABRT, Aborted.
> #0 0x7fa3cb5591f7
[jira] [Commented] (IMPALA-8091) Kudu SIGABRT'ed during dataload on Impala release build
[
https://issues.apache.org/jira/browse/IMPALA-8091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16749143#comment-16749143
]
Michael Brown commented on IMPALA-8091:
---
The failure isn't easy to triage from a console log. There, it looks like:
{noformat}
11:40:19 11:40:19 Error executing impala SQL:
/data/jenkins/workspace/impala-asf-master-core/repos/Impala/logs/data_loading/sql/functional/create-functional-query-exhaustive-impala-generated-kudu-none-none.sql
See:
/data/jenkins/workspace/impala-asf-master-core/repos/Impala/logs/data_loading/sql/functional/create-functional-query-exhaustive-impala-generated-kudu-none-none.sql.log
{noformat}
That log looks like this:
{noformat}
Traceback (most recent call last):
File
"/data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/bin/load-data.py",
line 207, in exec_impala_query_from_file
result = impala_client.execute(query)
File
"/data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/tests/beeswax/impala_beeswax.py",
line 182, in execute
handle = self.__execute_query(query_string.strip(), user=user)
File
"/data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/tests/beeswax/impala_beeswax.py",
line 357, in __execute_query
handle = self.execute_query_async(query_string, user=user)
File
"/data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/tests/beeswax/impala_beeswax.py",
line 351, in execute_query_async
handle = self.__do_rpc(lambda: self.imp_service.query(query,))
File
"/data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/tests/beeswax/impala_beeswax.py",
line 509, in __do_rpc
raise ImpalaBeeswaxException(self.__build_error_message(b), b)
ImpalaBeeswaxException: ImpalaBeeswaxException:
INNER EXCEPTION:
MESSAGE: ImpalaRuntimeException: Error creating Kudu table
'impala::tpch_kudu.lineitem'
CAUSED BY: NonRecoverableException: too many attempts:
KuduRpc(method=ListTables, tablet=null, attempt=101,
DeadlineTracker(timeout=18, elapsed=177706), Traces: [0ms] querying master,
[45ms] Sub rpc
: ConnectToMaster sending RPC to server master-127.0.0.1:7051, [100ms] Sub rpc:
ConnectToMaster received from server master-127.0.0.1:7051 response Network
error: java.net.ConnectException: Connection r
efused: /127.0.0.1:7051, [107ms] delaying RPC due to Service unavailable:
Master config (127.0.0.1:7051) has no leader.
{noformat}
This is the kudu service in the minicluster dropping out from under Impala.
Note btw that Kudu initially started ok.
{noformat}
11:26:26 Starting kudu (Web UI - http://localhost:8051)
11:26:41 The cluster is running
{noformat}
Finally, there's evidence in the minicluster logs for Kudu that all 4 services
(1 master, 3 tablet servers) all failed the same dcheck as pasted above.
> Kudu SIGABRT'ed during dataload on Impala release build
> ---
>
> Key: IMPALA-8091
> URL: https://issues.apache.org/jira/browse/IMPALA-8091
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 3.2.0
>Reporter: David Knupp
>Assignee: Thomas Tauber-Marshall
>Priority: Critical
> Labels: broken-build
>
> *Console log*:
> {noformat}
> 19:10:04 19:10:04 Error executing impala SQL:
> /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/logs/data_loading/sql/functional/create-functional-query-exhaustive-impala-generated-kudu-none-none.sql
> See:
> /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/logs/data_loading/sql/functional/create-functional-query-exhaustive-impala-generated-kudu-none-none.sql.log
> 19:10:04 Background task Loading functional-query data (pid 4831) failed.
> 19:10:04 Background task Loading TPC-H data (pid 4832) failed.
> 19:10:04 ERROR in
> /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/testdata/bin/create-load-data.sh
> at line 85: CM_HOST=${2-}
> [...]
> 19:10:05 2019-01-16 19:10:05,461 - archive_core_dumps - INFO - Found core
> files: ['./core.1547693849.30799.kudu-master',
> './core.1547693849.30783.kudu-tserver',
> './core.1547693849.30767.kudu-tserver',
> './core.1547693849.30808.kudu-tserver']
> 19:10:05 2019-01-16 19:10:05,649 - archive_core_dumps - INFO - [New LWP 30799]
> 19:10:05 [New LWP 30834]
> 19:10:05 [New LWP 30835]
> 19:10:05 [New LWP 30838]
> 19:10:05 [New LWP 30837]
> 19:10:05 [New LWP 30836]
> 19:10:05 Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
> 19:10:05 Program terminated with signal SIGABRT, Aborted.
> 19:10:05 #0 0x7f9a2cc611f7 in ?? ()
> 19:10:05
> 19:10:05 2019-01-16 19:10:05,650 - archive_core_dumps - INFO - Found binary
> path through GDB:
>
[jira] [Commented] (IMPALA-8091) Kudu SIGABRT'ed during dataload on Impala release build
[
https://issues.apache.org/jira/browse/IMPALA-8091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16749130#comment-16749130
]
Michael Brown commented on IMPALA-8091:
---
I've seen this too, and I think this is likely an environmental problem.
{noformat}
F0116 18:57:29.339143 30799 master_main.cc:81] Check failed: _s.ok() Bad
status: Runtime error: Cannot initialize clock: failed to wait for clock sync
using command '/usr/bin/chronyc waitsync 60 0 0 1': /usr/bin/chronyc: process
exited with non-zero status 1
{noformat}
> Kudu SIGABRT'ed during dataload on Impala release build
> ---
>
> Key: IMPALA-8091
> URL: https://issues.apache.org/jira/browse/IMPALA-8091
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 3.2.0
>Reporter: David Knupp
>Assignee: Thomas Tauber-Marshall
>Priority: Critical
> Labels: broken-build
>
> *Console log*:
> {noformat}
> 19:10:04 19:10:04 Error executing impala SQL:
> /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/logs/data_loading/sql/functional/create-functional-query-exhaustive-impala-generated-kudu-none-none.sql
> See:
> /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/logs/data_loading/sql/functional/create-functional-query-exhaustive-impala-generated-kudu-none-none.sql.log
> 19:10:04 Background task Loading functional-query data (pid 4831) failed.
> 19:10:04 Background task Loading TPC-H data (pid 4832) failed.
> 19:10:04 ERROR in
> /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/testdata/bin/create-load-data.sh
> at line 85: CM_HOST=${2-}
> [...]
> 19:10:05 2019-01-16 19:10:05,461 - archive_core_dumps - INFO - Found core
> files: ['./core.1547693849.30799.kudu-master',
> './core.1547693849.30783.kudu-tserver',
> './core.1547693849.30767.kudu-tserver',
> './core.1547693849.30808.kudu-tserver']
> 19:10:05 2019-01-16 19:10:05,649 - archive_core_dumps - INFO - [New LWP 30799]
> 19:10:05 [New LWP 30834]
> 19:10:05 [New LWP 30835]
> 19:10:05 [New LWP 30838]
> 19:10:05 [New LWP 30837]
> 19:10:05 [New LWP 30836]
> 19:10:05 Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
> 19:10:05 Program terminated with signal SIGABRT, Aborted.
> 19:10:05 #0 0x7f9a2cc611f7 in ?? ()
> 19:10:05
> 19:10:05 2019-01-16 19:10:05,650 - archive_core_dumps - INFO - Found binary
> path through GDB:
> /data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c
> 19:10:05 2019-01-16 19:10:05,893 - archive_core_dumps - WARNING - Failed to
> determine binary because multiple candidate binaries were found and none of
> their paths contained 'latest' to disambiguate:
> 19:10:05 Core:./core.1547693849.30799.kudu-master
> 19:10:05
> Binaries:['./testdata/cluster/node_templates/common/etc/init.d/kudu-master',
> './testdata/cluster/cdh6/node-1/etc/init.d/kudu-master']
> 19:10:05
> 19:10:05 2019-01-16 19:10:05,917 - archive_core_dumps - INFO - [New LWP 30783]
> 19:10:05 [New LWP 30810]
> 19:10:05 [New LWP 30812]
> 19:10:05 [New LWP 30811]
> 19:10:05 [New LWP 30824]
> 19:10:05 [New LWP 30820]
> 19:10:05 Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
> 19:10:05 Program terminated with signal SIGABRT, Aborted.
> 19:10:05 #0 0x7f0b81fb11f7 in ?? ()
> {noformat}
> *Backtraces*:
> {noformat}
> CORE: ./core.1547693849.30799.kudu-master
> BINARY: ./be/build/latest/service/impalad
> Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
> Program terminated with signal SIGABRT, Aborted.
> #0 0x7f9a2cc611f7 in ?? ()
> #0 0x7f9a2cc611f7 in ?? ()
> #1 0x7f9a2cc628e8 in ?? ()
> #2 0x0020 in ?? ()
> #3 0x in ?? ()
> CORE: ./core.1547693849.30783.kudu-tserver
> BINARY: ./be/build/latest/service/impalad
> Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
> Program terminated with signal SIGABRT, Aborted.
> #0 0x7f0b81fb11f7 in ?? ()
> #0 0x7f0b81fb11f7 in ?? ()
> #1 0x7f0b81fb28e8 in ?? ()
> #2 0x0020 in ?? ()
> #3 0x in ?? ()
> CORE: ./core.1547693849.30767.kudu-tserver
> BINARY: ./be/build/latest/service/impalad
> Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
> Program terminated with signal SIGABRT, Aborted.
> #0 0x7f60b1e2f1f7 in ?? ()
> #0 0x7f60b1e2f1f7 in ?? ()
> #1 0x7f60b1e308e8 in ?? ()
> #2 0x0020 in ?? ()
> #3 0x in ?? ()
> CORE: ./core.1547693849.30808.kudu-tserver
> BINARY: ./be/build/latest/service/impalad
[jira] [Commented] (IMPALA-8064) test_min_max_filters is flaky
[
https://issues.apache.org/jira/browse/IMPALA-8064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16746539#comment-16746539
]
Michael Brown commented on IMPALA-8064:
---
That's how I read that, too. My point is I didn't see evidence of waiting
100,000 ms according to my pastes above. Instead, according to the profile and
the way I read it, the scan nodes waited 11,337 ms and 36,768 ms.
> test_min_max_filters is flaky
> --
>
> Key: IMPALA-8064
> URL: https://issues.apache.org/jira/browse/IMPALA-8064
> Project: IMPALA
> Issue Type: Bug
>Reporter: Pooja Nilangekar
>Assignee: Janaki Lahorani
>Priority: Blocker
> Labels: broken-build, flaky-test
> Attachments: profile.txt
>
>
> The following configuration of the test_min_max_filters:
> {code:java}
> query_test.test_runtime_filters.TestMinMaxFilters.test_min_max_filters[protocol:
> beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0,
> 'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
> 'abort_on_error': 1, 'debug_action': None, 'exec_single_node_rows_threshold':
> 0} | table_format: kudu/none]{code}
> It produces a higher aggregation of sum over the proberows than expected:
> {code:java}
> query_test/test_runtime_filters.py:113: in test_min_max_filters
> self.run_test_case('QueryTest/min_max_filters', vector)
> common/impala_test_suite.py:518: in run_test_case
> update_section=pytest.config.option.update_results)
> common/test_result_verifier.py:612: in verify_runtime_profile %
> (function, field, expected_value, actual_value, actual))
> E AssertionError: Aggregation of SUM over ProbeRows did not match expected
> results.
> E EXPECTED VALUE: E 619
> EACTUAL VALUE: E 652
> {code}
> This test was introduced in the patch for IMPALA-6533. The failure occurred
> during an ASAN build.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-8064) test_min_max_filters is flaky
[
https://issues.apache.org/jira/browse/IMPALA-8064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16746525#comment-16746525
]
Michael Brown commented on IMPALA-8064:
---
Something curious here as someone who doesn't know all the intricacies of
profiles: If I'm setting RUNTIME_FILTER_WAIT_TIME_MS to 100,000 but then see
that the waits for those runtime filters weren't that long, I'm going to be
confused. Is the profile confusing and I'm misreading? Is there a bug in the
reporting or a bug in actually waiting that long?
> test_min_max_filters is flaky
> --
>
> Key: IMPALA-8064
> URL: https://issues.apache.org/jira/browse/IMPALA-8064
> Project: IMPALA
> Issue Type: Bug
>Reporter: Pooja Nilangekar
>Assignee: Janaki Lahorani
>Priority: Blocker
> Labels: broken-build, flaky-test
> Attachments: profile.txt
>
>
> The following configuration of the test_min_max_filters:
> {code:java}
> query_test.test_runtime_filters.TestMinMaxFilters.test_min_max_filters[protocol:
> beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0,
> 'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
> 'abort_on_error': 1, 'debug_action': None, 'exec_single_node_rows_threshold':
> 0} | table_format: kudu/none]{code}
> It produces a higher aggregation of sum over the proberows than expected:
> {code:java}
> query_test/test_runtime_filters.py:113: in test_min_max_filters
> self.run_test_case('QueryTest/min_max_filters', vector)
> common/impala_test_suite.py:518: in run_test_case
> update_section=pytest.config.option.update_results)
> common/test_result_verifier.py:612: in verify_runtime_profile %
> (function, field, expected_value, actual_value, actual))
> E AssertionError: Aggregation of SUM over ProbeRows did not match expected
> results.
> E EXPECTED VALUE: E 619
> EACTUAL VALUE: E 652
> {code}
> This test was introduced in the patch for IMPALA-6533. The failure occurred
> during an ASAN build.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-8064) test_min_max_filters is flaky
[
https://issues.apache.org/jira/browse/IMPALA-8064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16746521#comment-16746521
]
Michael Brown commented on IMPALA-8064:
---
Profile for above uploaded as {{profile.txt}}.
> test_min_max_filters is flaky
> --
>
> Key: IMPALA-8064
> URL: https://issues.apache.org/jira/browse/IMPALA-8064
> Project: IMPALA
> Issue Type: Bug
>Reporter: Pooja Nilangekar
>Assignee: Janaki Lahorani
>Priority: Blocker
> Labels: broken-build, flaky-test
> Attachments: profile.txt
>
>
> The following configuration of the test_min_max_filters:
> {code:java}
> query_test.test_runtime_filters.TestMinMaxFilters.test_min_max_filters[protocol:
> beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0,
> 'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
> 'abort_on_error': 1, 'debug_action': None, 'exec_single_node_rows_threshold':
> 0} | table_format: kudu/none]{code}
> It produces a higher aggregation of sum over the proberows than expected:
> {code:java}
> query_test/test_runtime_filters.py:113: in test_min_max_filters
> self.run_test_case('QueryTest/min_max_filters', vector)
> common/impala_test_suite.py:518: in run_test_case
> update_section=pytest.config.option.update_results)
> common/test_result_verifier.py:612: in verify_runtime_profile %
> (function, field, expected_value, actual_value, actual))
> E AssertionError: Aggregation of SUM over ProbeRows did not match expected
> results.
> E EXPECTED VALUE: E 619
> EACTUAL VALUE: E 652
> {code}
> This test was introduced in the patch for IMPALA-6533. The failure occurred
> during an ASAN build.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-8064) test_min_max_filters is flaky
[
https://issues.apache.org/jira/browse/IMPALA-8064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Brown updated IMPALA-8064:
--
Attachment: profile.txt
> test_min_max_filters is flaky
> --
>
> Key: IMPALA-8064
> URL: https://issues.apache.org/jira/browse/IMPALA-8064
> Project: IMPALA
> Issue Type: Bug
>Reporter: Pooja Nilangekar
>Assignee: Janaki Lahorani
>Priority: Blocker
> Labels: broken-build, flaky-test
> Attachments: profile.txt
>
>
> The following configuration of the test_min_max_filters:
> {code:java}
> query_test.test_runtime_filters.TestMinMaxFilters.test_min_max_filters[protocol:
> beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0,
> 'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
> 'abort_on_error': 1, 'debug_action': None, 'exec_single_node_rows_threshold':
> 0} | table_format: kudu/none]{code}
> It produces a higher aggregation of sum over the proberows than expected:
> {code:java}
> query_test/test_runtime_filters.py:113: in test_min_max_filters
> self.run_test_case('QueryTest/min_max_filters', vector)
> common/impala_test_suite.py:518: in run_test_case
> update_section=pytest.config.option.update_results)
> common/test_result_verifier.py:612: in verify_runtime_profile %
> (function, field, expected_value, actual_value, actual))
> E AssertionError: Aggregation of SUM over ProbeRows did not match expected
> results.
> E EXPECTED VALUE: E 619
> EACTUAL VALUE: E 652
> {code}
> This test was introduced in the patch for IMPALA-6533. The failure occurred
> during an ASAN build.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-8064) test_min_max_filters is flaky
[
https://issues.apache.org/jira/browse/IMPALA-8064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16746518#comment-16746518
]
Michael Brown commented on IMPALA-8064:
---
[~janulatha] This is still happening on master in a downstream ASAN build+test.
I confirmed it was at a Git hash that had the fix above and of course the
correct {{RUNTIME_FILTER_WAIT_TIME_MS}} is evident.
{noformat}
query_test/test_runtime_filters.py:114: in test_min_max_filters
test_file_vars={'$RUNTIME_FILTER_WAIT_TIME_MS': str(WAIT_TIME_MS)})
common/impala_test_suite.py:518: in run_test_case
update_section=pytest.config.option.update_results)
common/test_result_verifier.py:612: in verify_runtime_profile
% (function, field, expected_value, actual_value, actual))
E AssertionError: Aggregation of SUM over ProbeRows did not match expected
results.
E EXPECTED VALUE:
E 643
E
E ACTUAL VALUE:
E 661
E
{noformat}
The query in question:
{noformat}
USE functional_kudu;
SET RUNTIME_FILTER_WAIT_TIME_MS=10;
select STRAIGHT_JOIN count(*) from decimal_rtf_tbl a
join [BROADCAST] decimal_rtf_tiny_tbl b
where a.d38_38 = b.d38_38 and b.d38_38 != 0;
{noformat}
Snippets from the profile which I'll upload in full show that 2/3 scan nodes
did not receive the runtime filter in time.
{noformat}
Instance 2b4a14d9933eb108:7565b8650006
(host=downstream-jenkins-host:22001):(Total: 1m47s, non-child: 3.999ms, %
non-child: 0.00%)
HASH_JOIN_NODE (id=2):(Total: 172.996ms, non-child: 9.999ms, %
non-child: 5.78%)
Runtime filters: 1 of 1 Runtime Filter Published
KUDU_SCAN_NODE (id=0):(Total: 162.996ms, non-child: 162.996ms, %
non-child: 100.00%)
Runtime filters: All filters arrived. Waited 0
Instance 2b4a14d9933eb108:7565b8650004
(host=downstream-jenkins-host:22000):(Total: 1m42s, non-child: 0.000ns, %
non-child: 0.00%)
HASH_JOIN_NODE (id=2):(Total: 13s819ms, non-child: 25.999ms, %
non-child: 0.10%)
Runtime filters: 1 of 1 Runtime Filter Published
KUDU_SCAN_NODE (id=0):(Total: 11s359ms, non-child: 11s359ms, %
non-child: 100.00%)
Runtime filters: Not all filters arrived (arrived: [], missing [1]),
waited for 11s337ms
Instance 2b4a14d9933eb108:7565b8650005
(host=downstream-jenkins-host:22002):(Total: 1m42s, non-child: 0.000ns, %
non-child: 0.00%)
HASH_JOIN_NODE (id=2):(Total: 39s251ms, non-child: 6.999ms, %
non-child: 0.01%)
Runtime filters: 1 of 1 Runtime Filter Published
KUDU_SCAN_NODE (id=0):(Total: 36s812ms, non-child: 36s812ms, %
non-child: 100.00%)
Runtime filters: Not all filters arrived (arrived: [], missing [1]),
waited for 36s768ms
{noformat}
> test_min_max_filters is flaky
> --
>
> Key: IMPALA-8064
> URL: https://issues.apache.org/jira/browse/IMPALA-8064
> Project: IMPALA
> Issue Type: Bug
>Reporter: Pooja Nilangekar
>Assignee: Janaki Lahorani
>Priority: Blocker
> Labels: broken-build, flaky-test
>
> The following configuration of the test_min_max_filters:
> {code:java}
> query_test.test_runtime_filters.TestMinMaxFilters.test_min_max_filters[protocol:
> beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0,
> 'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
> 'abort_on_error': 1, 'debug_action': None, 'exec_single_node_rows_threshold':
> 0} | table_format: kudu/none]{code}
> It produces a higher aggregation of sum over the proberows than expected:
> {code:java}
> query_test/test_runtime_filters.py:113: in test_min_max_filters
> self.run_test_case('QueryTest/min_max_filters', vector)
> common/impala_test_suite.py:518: in run_test_case
> update_section=pytest.config.option.update_results)
> common/test_result_verifier.py:612: in verify_runtime_profile %
> (function, field, expected_value, actual_value, actual))
> E AssertionError: Aggregation of SUM over ProbeRows did not match expected
> results.
> E EXPECTED VALUE: E 619
> EACTUAL VALUE: E 652
> {code}
> This test was introduced in the patch for IMPALA-6533. The failure occurred
> during an ASAN build.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Comment Edited] (IMPALA-8069) crash in impala::Sorter::Run::Run
[
https://issues.apache.org/jira/browse/IMPALA-8069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16740721#comment-16740721
]
Michael Brown edited comment on IMPALA-8069 at 1/11/19 7:47 PM:
I'm also going to consider this a regression, because a query like this has
progressed from {{AnalysisException}} to crash.
was (Author: mikesbrown):
I'm also going to consider this a regression, because a query like this has
progress from {{AnalysisException}} to crash.
> crash in impala::Sorter::Run::Run
> -
>
> Key: IMPALA-8069
> URL: https://issues.apache.org/jira/browse/IMPALA-8069
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 3.2.0
>Reporter: Michael Brown
>Priority: Blocker
> Labels: crash, query_generator, regression
>
> {noformat}
> 0 0x7fd37c9b8428 in raise () from /lib/x86_64-linux-gnu/libc.so.6
> #1 0x7fd37c9ba02a in abort () from /lib/x86_64-linux-gnu/libc.so.6
> #2 0x7fd37fb7fe49 in ?? () from
> /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so
> #3 0x7fd37fd35387 in ?? () from
> /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so
> #4 0x7fd37fb8995f in JVM_handle_linux_signal () from
> /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so
> #5 0x7fd37fb7cf78 in ?? () from
> /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so
> #6
> #7 0x02723007 in impala::Sorter::Run::Run (this=0x13485200,
> parent=0xd3d8b00, sort_tuple_desc=0xf6dd320, initial_run=true) at
> /home/mikeb/Impala/be/src/runtime/sorter.cc:609
> #8 0x0272a3cb in impala::Sorter::Open (this=0xd3d8b00) at
> /home/mikeb/Impala/be/src/runtime/sorter.cc:1514
> #9 0x0237b186 in impala::SortNode::Open (this=0xd3d8840,
> state=0xffa1ba0) at /home/mikeb/Impala/be/src/exec/sort-node.cc:84
> #10 0x0239b44b in impala::AnalyticEvalNode::Open (this=0x100aba80,
> state=0xffa1ba0) at /home/mikeb/Impala/be/src/exec/analytic-eval-node.cc:182
> #11 0x01f20cba in impala::FragmentInstanceState::Open
> (this=0xf5afd80) at
> /home/mikeb/Impala/be/src/runtime/fragment-instance-state.cc:304
> #12 0x01f1deee in impala::FragmentInstanceState::Exec
> (this=0xf5afd80) at
> /home/mikeb/Impala/be/src/runtime/fragment-instance-state.cc:84
> #13 0x01f2ea39 in impala::QueryState::ExecFInstance (this=0xea2,
> fis=0xf5afd80) at /home/mikeb/Impala/be/src/runtime/query-state.cc:584
> #14 0x01f2cd42 in impala::QueryStateoperator()(void)
> const (__closure=0x7fd2b1869ca8) at
> /home/mikeb/Impala/be/src/runtime/query-state.cc:493
> #15 0x01f2f7e1 in
> boost::detail::function::void_function_obj_invoker0,
> void>::invoke(boost::detail::function::function_buffer &) (
> function_obj_ptr=...) at
> /home/mikeb/Impala/toolchain/boost-1.57.0-p3/include/boost/function/function_template.hpp:153
> #16 0x01d46c96 in boost::function0::operator()
> (this=0x7fd2b1869ca0) at
> /home/mikeb/Impala/toolchain/boost-1.57.0-p3/include/boost/function/function_template.hpp:767
> #17 0x021d4897 in impala::Thread::SuperviseThread(std::string const&,
> std::string const&, boost::function, impala::ThreadDebugInfo const*,
> impala::Promise*) (name=..., category=...,
> functor=..., parent_thread_info=0x7fd2b206a950,
> thread_started=0x7fd2b20698f0) at /home/mikeb/Impala/be/src/util/thread.cc:359
> #18 0x021dcbb7 in boost::_bi::list5,
> boost::_bi::value, boost::_bi::value >,
> boost::_bi::value,
> boost::_bi::value*>
> >::operator() boost::function, impala::ThreadDebugInfo const*,
> impala::Promise*),
> boost::_bi::list0>(boost::_bi::type, void (*&)(std::string const&,
> std::string const&, boost::function, impala::ThreadDebugInfo const*,
> impala::Promise*), boost::_bi::list0&, int)
> (this=0xed511c0,
> f=@0xed511b8: 0x21d4530 const&, std::string const&, boost::function, impala::ThreadDebugInfo
> const*, impala::Promise*)>, a=...) at
> /home/mikeb/Impala/toolchain/boost-1.57.0-p3/include/boost/bind/bind.hpp:525
> #19 0x021dcadb in boost::_bi::bind_t const&, std::string const&, boost::function, impala::ThreadDebugInfo
> const*, impala::Promise*),
> boost::_bi::list5,
> boost::_bi::value, boost::_bi::value >,
> boost::_bi::value,
> boost::_bi::value*> >
> >::operator()() (this=0xed511b8) at
> /home/mikeb/Impala/toolchain/boost-1.57.0-p3/include/boost/bind/bind_template.hpp:20
> #20 0x021dca9e in boost::detail::thread_data void (*)(std::string const&, std::string const&, boost::function,
> impala::ThreadDebugInfo const*, impala::Promise (impala::PromiseMode)0>*), boost::_bi::list5,
> boost::_bi::value, boost::_bi::value >,
> boost::_bi::value,
> boost::_bi::value*> > >
> >::run()
[jira] [Updated] (IMPALA-8069) crash in impala::Sorter::Run::Run
[
https://issues.apache.org/jira/browse/IMPALA-8069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Brown updated IMPALA-8069:
--
Labels: crash query_generator regression (was: crash query_generator)
> crash in impala::Sorter::Run::Run
> -
>
> Key: IMPALA-8069
> URL: https://issues.apache.org/jira/browse/IMPALA-8069
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 3.2.0
>Reporter: Michael Brown
>Priority: Blocker
> Labels: crash, query_generator, regression
>
> {noformat}
> 0 0x7fd37c9b8428 in raise () from /lib/x86_64-linux-gnu/libc.so.6
> #1 0x7fd37c9ba02a in abort () from /lib/x86_64-linux-gnu/libc.so.6
> #2 0x7fd37fb7fe49 in ?? () from
> /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so
> #3 0x7fd37fd35387 in ?? () from
> /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so
> #4 0x7fd37fb8995f in JVM_handle_linux_signal () from
> /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so
> #5 0x7fd37fb7cf78 in ?? () from
> /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so
> #6
> #7 0x02723007 in impala::Sorter::Run::Run (this=0x13485200,
> parent=0xd3d8b00, sort_tuple_desc=0xf6dd320, initial_run=true) at
> /home/mikeb/Impala/be/src/runtime/sorter.cc:609
> #8 0x0272a3cb in impala::Sorter::Open (this=0xd3d8b00) at
> /home/mikeb/Impala/be/src/runtime/sorter.cc:1514
> #9 0x0237b186 in impala::SortNode::Open (this=0xd3d8840,
> state=0xffa1ba0) at /home/mikeb/Impala/be/src/exec/sort-node.cc:84
> #10 0x0239b44b in impala::AnalyticEvalNode::Open (this=0x100aba80,
> state=0xffa1ba0) at /home/mikeb/Impala/be/src/exec/analytic-eval-node.cc:182
> #11 0x01f20cba in impala::FragmentInstanceState::Open
> (this=0xf5afd80) at
> /home/mikeb/Impala/be/src/runtime/fragment-instance-state.cc:304
> #12 0x01f1deee in impala::FragmentInstanceState::Exec
> (this=0xf5afd80) at
> /home/mikeb/Impala/be/src/runtime/fragment-instance-state.cc:84
> #13 0x01f2ea39 in impala::QueryState::ExecFInstance (this=0xea2,
> fis=0xf5afd80) at /home/mikeb/Impala/be/src/runtime/query-state.cc:584
> #14 0x01f2cd42 in impala::QueryStateoperator()(void)
> const (__closure=0x7fd2b1869ca8) at
> /home/mikeb/Impala/be/src/runtime/query-state.cc:493
> #15 0x01f2f7e1 in
> boost::detail::function::void_function_obj_invoker0,
> void>::invoke(boost::detail::function::function_buffer &) (
> function_obj_ptr=...) at
> /home/mikeb/Impala/toolchain/boost-1.57.0-p3/include/boost/function/function_template.hpp:153
> #16 0x01d46c96 in boost::function0::operator()
> (this=0x7fd2b1869ca0) at
> /home/mikeb/Impala/toolchain/boost-1.57.0-p3/include/boost/function/function_template.hpp:767
> #17 0x021d4897 in impala::Thread::SuperviseThread(std::string const&,
> std::string const&, boost::function, impala::ThreadDebugInfo const*,
> impala::Promise*) (name=..., category=...,
> functor=..., parent_thread_info=0x7fd2b206a950,
> thread_started=0x7fd2b20698f0) at /home/mikeb/Impala/be/src/util/thread.cc:359
> #18 0x021dcbb7 in boost::_bi::list5,
> boost::_bi::value, boost::_bi::value >,
> boost::_bi::value,
> boost::_bi::value*>
> >::operator() boost::function, impala::ThreadDebugInfo const*,
> impala::Promise*),
> boost::_bi::list0>(boost::_bi::type, void (*&)(std::string const&,
> std::string const&, boost::function, impala::ThreadDebugInfo const*,
> impala::Promise*), boost::_bi::list0&, int)
> (this=0xed511c0,
> f=@0xed511b8: 0x21d4530 const&, std::string const&, boost::function, impala::ThreadDebugInfo
> const*, impala::Promise*)>, a=...) at
> /home/mikeb/Impala/toolchain/boost-1.57.0-p3/include/boost/bind/bind.hpp:525
> #19 0x021dcadb in boost::_bi::bind_t const&, std::string const&, boost::function, impala::ThreadDebugInfo
> const*, impala::Promise*),
> boost::_bi::list5,
> boost::_bi::value, boost::_bi::value >,
> boost::_bi::value,
> boost::_bi::value*> >
> >::operator()() (this=0xed511b8) at
> /home/mikeb/Impala/toolchain/boost-1.57.0-p3/include/boost/bind/bind_template.hpp:20
> #20 0x021dca9e in boost::detail::thread_data void (*)(std::string const&, std::string const&, boost::function,
> impala::ThreadDebugInfo const*, impala::Promise (impala::PromiseMode)0>*), boost::_bi::list5,
> boost::_bi::value, boost::_bi::value >,
> boost::_bi::value,
> boost::_bi::value*> > >
> >::run() (this=0xed51000)
> at
> /home/mikeb/Impala/toolchain/boost-1.57.0-p3/include/boost/thread/detail/thread.hpp:116
> #21 0x036c17fa in thread_proxy ()
> #22 0x7fd37cd546ba in start_thread () from
> /lib/x86_64-linux-gnu/libpthread.so.0
> #23 0x7fd37ca8a41d in clone () from
[jira] [Commented] (IMPALA-8069) crash in impala::Sorter::Run::Run
[
https://issues.apache.org/jira/browse/IMPALA-8069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16740721#comment-16740721
]
Michael Brown commented on IMPALA-8069:
---
I'm also going to consider this a regression, because a query like this has
progress from {{AnalysisException}} to crash.
> crash in impala::Sorter::Run::Run
> -
>
> Key: IMPALA-8069
> URL: https://issues.apache.org/jira/browse/IMPALA-8069
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 3.2.0
>Reporter: Michael Brown
>Priority: Blocker
> Labels: crash, query_generator
>
> {noformat}
> 0 0x7fd37c9b8428 in raise () from /lib/x86_64-linux-gnu/libc.so.6
> #1 0x7fd37c9ba02a in abort () from /lib/x86_64-linux-gnu/libc.so.6
> #2 0x7fd37fb7fe49 in ?? () from
> /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so
> #3 0x7fd37fd35387 in ?? () from
> /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so
> #4 0x7fd37fb8995f in JVM_handle_linux_signal () from
> /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so
> #5 0x7fd37fb7cf78 in ?? () from
> /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so
> #6
> #7 0x02723007 in impala::Sorter::Run::Run (this=0x13485200,
> parent=0xd3d8b00, sort_tuple_desc=0xf6dd320, initial_run=true) at
> /home/mikeb/Impala/be/src/runtime/sorter.cc:609
> #8 0x0272a3cb in impala::Sorter::Open (this=0xd3d8b00) at
> /home/mikeb/Impala/be/src/runtime/sorter.cc:1514
> #9 0x0237b186 in impala::SortNode::Open (this=0xd3d8840,
> state=0xffa1ba0) at /home/mikeb/Impala/be/src/exec/sort-node.cc:84
> #10 0x0239b44b in impala::AnalyticEvalNode::Open (this=0x100aba80,
> state=0xffa1ba0) at /home/mikeb/Impala/be/src/exec/analytic-eval-node.cc:182
> #11 0x01f20cba in impala::FragmentInstanceState::Open
> (this=0xf5afd80) at
> /home/mikeb/Impala/be/src/runtime/fragment-instance-state.cc:304
> #12 0x01f1deee in impala::FragmentInstanceState::Exec
> (this=0xf5afd80) at
> /home/mikeb/Impala/be/src/runtime/fragment-instance-state.cc:84
> #13 0x01f2ea39 in impala::QueryState::ExecFInstance (this=0xea2,
> fis=0xf5afd80) at /home/mikeb/Impala/be/src/runtime/query-state.cc:584
> #14 0x01f2cd42 in impala::QueryStateoperator()(void)
> const (__closure=0x7fd2b1869ca8) at
> /home/mikeb/Impala/be/src/runtime/query-state.cc:493
> #15 0x01f2f7e1 in
> boost::detail::function::void_function_obj_invoker0,
> void>::invoke(boost::detail::function::function_buffer &) (
> function_obj_ptr=...) at
> /home/mikeb/Impala/toolchain/boost-1.57.0-p3/include/boost/function/function_template.hpp:153
> #16 0x01d46c96 in boost::function0::operator()
> (this=0x7fd2b1869ca0) at
> /home/mikeb/Impala/toolchain/boost-1.57.0-p3/include/boost/function/function_template.hpp:767
> #17 0x021d4897 in impala::Thread::SuperviseThread(std::string const&,
> std::string const&, boost::function, impala::ThreadDebugInfo const*,
> impala::Promise*) (name=..., category=...,
> functor=..., parent_thread_info=0x7fd2b206a950,
> thread_started=0x7fd2b20698f0) at /home/mikeb/Impala/be/src/util/thread.cc:359
> #18 0x021dcbb7 in boost::_bi::list5,
> boost::_bi::value, boost::_bi::value >,
> boost::_bi::value,
> boost::_bi::value*>
> >::operator() boost::function, impala::ThreadDebugInfo const*,
> impala::Promise*),
> boost::_bi::list0>(boost::_bi::type, void (*&)(std::string const&,
> std::string const&, boost::function, impala::ThreadDebugInfo const*,
> impala::Promise*), boost::_bi::list0&, int)
> (this=0xed511c0,
> f=@0xed511b8: 0x21d4530 const&, std::string const&, boost::function, impala::ThreadDebugInfo
> const*, impala::Promise*)>, a=...) at
> /home/mikeb/Impala/toolchain/boost-1.57.0-p3/include/boost/bind/bind.hpp:525
> #19 0x021dcadb in boost::_bi::bind_t const&, std::string const&, boost::function, impala::ThreadDebugInfo
> const*, impala::Promise*),
> boost::_bi::list5,
> boost::_bi::value, boost::_bi::value >,
> boost::_bi::value,
> boost::_bi::value*> >
> >::operator()() (this=0xed511b8) at
> /home/mikeb/Impala/toolchain/boost-1.57.0-p3/include/boost/bind/bind_template.hpp:20
> #20 0x021dca9e in boost::detail::thread_data void (*)(std::string const&, std::string const&, boost::function,
> impala::ThreadDebugInfo const*, impala::Promise (impala::PromiseMode)0>*), boost::_bi::list5,
> boost::_bi::value, boost::_bi::value >,
> boost::_bi::value,
> boost::_bi::value*> > >
> >::run() (this=0xed51000)
> at
> /home/mikeb/Impala/toolchain/boost-1.57.0-p3/include/boost/thread/detail/thread.hpp:116
> #21 0x036c17fa in thread_proxy ()
> #22 0x7fd37cd546ba in start_thread () from
>
[jira] [Commented] (IMPALA-8069) crash in impala::Sorter::Run::Run
[
https://issues.apache.org/jira/browse/IMPALA-8069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16740713#comment-16740713
]
Michael Brown commented on IMPALA-8069:
---
Not sure if [~mostrows] is available to work on this or not. Cc [~Paul.Rogers]
as well.
> crash in impala::Sorter::Run::Run
> -
>
> Key: IMPALA-8069
> URL: https://issues.apache.org/jira/browse/IMPALA-8069
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 3.2.0
>Reporter: Michael Brown
>Priority: Blocker
> Labels: crash, query_generator
>
> {noformat}
> 0 0x7fd37c9b8428 in raise () from /lib/x86_64-linux-gnu/libc.so.6
> #1 0x7fd37c9ba02a in abort () from /lib/x86_64-linux-gnu/libc.so.6
> #2 0x7fd37fb7fe49 in ?? () from
> /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so
> #3 0x7fd37fd35387 in ?? () from
> /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so
> #4 0x7fd37fb8995f in JVM_handle_linux_signal () from
> /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so
> #5 0x7fd37fb7cf78 in ?? () from
> /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so
> #6
> #7 0x02723007 in impala::Sorter::Run::Run (this=0x13485200,
> parent=0xd3d8b00, sort_tuple_desc=0xf6dd320, initial_run=true) at
> /home/mikeb/Impala/be/src/runtime/sorter.cc:609
> #8 0x0272a3cb in impala::Sorter::Open (this=0xd3d8b00) at
> /home/mikeb/Impala/be/src/runtime/sorter.cc:1514
> #9 0x0237b186 in impala::SortNode::Open (this=0xd3d8840,
> state=0xffa1ba0) at /home/mikeb/Impala/be/src/exec/sort-node.cc:84
> #10 0x0239b44b in impala::AnalyticEvalNode::Open (this=0x100aba80,
> state=0xffa1ba0) at /home/mikeb/Impala/be/src/exec/analytic-eval-node.cc:182
> #11 0x01f20cba in impala::FragmentInstanceState::Open
> (this=0xf5afd80) at
> /home/mikeb/Impala/be/src/runtime/fragment-instance-state.cc:304
> #12 0x01f1deee in impala::FragmentInstanceState::Exec
> (this=0xf5afd80) at
> /home/mikeb/Impala/be/src/runtime/fragment-instance-state.cc:84
> #13 0x01f2ea39 in impala::QueryState::ExecFInstance (this=0xea2,
> fis=0xf5afd80) at /home/mikeb/Impala/be/src/runtime/query-state.cc:584
> #14 0x01f2cd42 in impala::QueryStateoperator()(void)
> const (__closure=0x7fd2b1869ca8) at
> /home/mikeb/Impala/be/src/runtime/query-state.cc:493
> #15 0x01f2f7e1 in
> boost::detail::function::void_function_obj_invoker0,
> void>::invoke(boost::detail::function::function_buffer &) (
> function_obj_ptr=...) at
> /home/mikeb/Impala/toolchain/boost-1.57.0-p3/include/boost/function/function_template.hpp:153
> #16 0x01d46c96 in boost::function0::operator()
> (this=0x7fd2b1869ca0) at
> /home/mikeb/Impala/toolchain/boost-1.57.0-p3/include/boost/function/function_template.hpp:767
> #17 0x021d4897 in impala::Thread::SuperviseThread(std::string const&,
> std::string const&, boost::function, impala::ThreadDebugInfo const*,
> impala::Promise*) (name=..., category=...,
> functor=..., parent_thread_info=0x7fd2b206a950,
> thread_started=0x7fd2b20698f0) at /home/mikeb/Impala/be/src/util/thread.cc:359
> #18 0x021dcbb7 in boost::_bi::list5,
> boost::_bi::value, boost::_bi::value >,
> boost::_bi::value,
> boost::_bi::value*>
> >::operator() boost::function, impala::ThreadDebugInfo const*,
> impala::Promise*),
> boost::_bi::list0>(boost::_bi::type, void (*&)(std::string const&,
> std::string const&, boost::function, impala::ThreadDebugInfo const*,
> impala::Promise*), boost::_bi::list0&, int)
> (this=0xed511c0,
> f=@0xed511b8: 0x21d4530 const&, std::string const&, boost::function, impala::ThreadDebugInfo
> const*, impala::Promise*)>, a=...) at
> /home/mikeb/Impala/toolchain/boost-1.57.0-p3/include/boost/bind/bind.hpp:525
> #19 0x021dcadb in boost::_bi::bind_t const&, std::string const&, boost::function, impala::ThreadDebugInfo
> const*, impala::Promise*),
> boost::_bi::list5,
> boost::_bi::value, boost::_bi::value >,
> boost::_bi::value,
> boost::_bi::value*> >
> >::operator()() (this=0xed511b8) at
> /home/mikeb/Impala/toolchain/boost-1.57.0-p3/include/boost/bind/bind_template.hpp:20
> #20 0x021dca9e in boost::detail::thread_data void (*)(std::string const&, std::string const&, boost::function,
> impala::ThreadDebugInfo const*, impala::Promise (impala::PromiseMode)0>*), boost::_bi::list5,
> boost::_bi::value, boost::_bi::value >,
> boost::_bi::value,
> boost::_bi::value*> > >
> >::run() (this=0xed51000)
> at
> /home/mikeb/Impala/toolchain/boost-1.57.0-p3/include/boost/thread/detail/thread.hpp:116
> #21 0x036c17fa in thread_proxy ()
> #22 0x7fd37cd546ba in start_thread () from
> /lib/x86_64-linux-gnu/libpthread.so.0
[jira] [Updated] (IMPALA-8069) crash in impala::Sorter::Run::Run
[
https://issues.apache.org/jira/browse/IMPALA-8069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Brown updated IMPALA-8069:
--
Labels: crash query_generator (was: query_generator)
> crash in impala::Sorter::Run::Run
> -
>
> Key: IMPALA-8069
> URL: https://issues.apache.org/jira/browse/IMPALA-8069
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 3.2.0
>Reporter: Michael Brown
>Priority: Blocker
> Labels: crash, query_generator
>
> {noformat}
> 0 0x7fd37c9b8428 in raise () from /lib/x86_64-linux-gnu/libc.so.6
> #1 0x7fd37c9ba02a in abort () from /lib/x86_64-linux-gnu/libc.so.6
> #2 0x7fd37fb7fe49 in ?? () from
> /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so
> #3 0x7fd37fd35387 in ?? () from
> /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so
> #4 0x7fd37fb8995f in JVM_handle_linux_signal () from
> /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so
> #5 0x7fd37fb7cf78 in ?? () from
> /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so
> #6
> #7 0x02723007 in impala::Sorter::Run::Run (this=0x13485200,
> parent=0xd3d8b00, sort_tuple_desc=0xf6dd320, initial_run=true) at
> /home/mikeb/Impala/be/src/runtime/sorter.cc:609
> #8 0x0272a3cb in impala::Sorter::Open (this=0xd3d8b00) at
> /home/mikeb/Impala/be/src/runtime/sorter.cc:1514
> #9 0x0237b186 in impala::SortNode::Open (this=0xd3d8840,
> state=0xffa1ba0) at /home/mikeb/Impala/be/src/exec/sort-node.cc:84
> #10 0x0239b44b in impala::AnalyticEvalNode::Open (this=0x100aba80,
> state=0xffa1ba0) at /home/mikeb/Impala/be/src/exec/analytic-eval-node.cc:182
> #11 0x01f20cba in impala::FragmentInstanceState::Open
> (this=0xf5afd80) at
> /home/mikeb/Impala/be/src/runtime/fragment-instance-state.cc:304
> #12 0x01f1deee in impala::FragmentInstanceState::Exec
> (this=0xf5afd80) at
> /home/mikeb/Impala/be/src/runtime/fragment-instance-state.cc:84
> #13 0x01f2ea39 in impala::QueryState::ExecFInstance (this=0xea2,
> fis=0xf5afd80) at /home/mikeb/Impala/be/src/runtime/query-state.cc:584
> #14 0x01f2cd42 in impala::QueryStateoperator()(void)
> const (__closure=0x7fd2b1869ca8) at
> /home/mikeb/Impala/be/src/runtime/query-state.cc:493
> #15 0x01f2f7e1 in
> boost::detail::function::void_function_obj_invoker0,
> void>::invoke(boost::detail::function::function_buffer &) (
> function_obj_ptr=...) at
> /home/mikeb/Impala/toolchain/boost-1.57.0-p3/include/boost/function/function_template.hpp:153
> #16 0x01d46c96 in boost::function0::operator()
> (this=0x7fd2b1869ca0) at
> /home/mikeb/Impala/toolchain/boost-1.57.0-p3/include/boost/function/function_template.hpp:767
> #17 0x021d4897 in impala::Thread::SuperviseThread(std::string const&,
> std::string const&, boost::function, impala::ThreadDebugInfo const*,
> impala::Promise*) (name=..., category=...,
> functor=..., parent_thread_info=0x7fd2b206a950,
> thread_started=0x7fd2b20698f0) at /home/mikeb/Impala/be/src/util/thread.cc:359
> #18 0x021dcbb7 in boost::_bi::list5,
> boost::_bi::value, boost::_bi::value >,
> boost::_bi::value,
> boost::_bi::value*>
> >::operator() boost::function, impala::ThreadDebugInfo const*,
> impala::Promise*),
> boost::_bi::list0>(boost::_bi::type, void (*&)(std::string const&,
> std::string const&, boost::function, impala::ThreadDebugInfo const*,
> impala::Promise*), boost::_bi::list0&, int)
> (this=0xed511c0,
> f=@0xed511b8: 0x21d4530 const&, std::string const&, boost::function, impala::ThreadDebugInfo
> const*, impala::Promise*)>, a=...) at
> /home/mikeb/Impala/toolchain/boost-1.57.0-p3/include/boost/bind/bind.hpp:525
> #19 0x021dcadb in boost::_bi::bind_t const&, std::string const&, boost::function, impala::ThreadDebugInfo
> const*, impala::Promise*),
> boost::_bi::list5,
> boost::_bi::value, boost::_bi::value >,
> boost::_bi::value,
> boost::_bi::value*> >
> >::operator()() (this=0xed511b8) at
> /home/mikeb/Impala/toolchain/boost-1.57.0-p3/include/boost/bind/bind_template.hpp:20
> #20 0x021dca9e in boost::detail::thread_data void (*)(std::string const&, std::string const&, boost::function,
> impala::ThreadDebugInfo const*, impala::Promise (impala::PromiseMode)0>*), boost::_bi::list5,
> boost::_bi::value, boost::_bi::value >,
> boost::_bi::value,
> boost::_bi::value*> > >
> >::run() (this=0xed51000)
> at
> /home/mikeb/Impala/toolchain/boost-1.57.0-p3/include/boost/thread/detail/thread.hpp:116
> #21 0x036c17fa in thread_proxy ()
> #22 0x7fd37cd546ba in start_thread () from
> /lib/x86_64-linux-gnu/libpthread.so.0
> #23 0x7fd37ca8a41d in clone () from
[jira] [Commented] (IMPALA-8069) crash in impala::Sorter::Run::Run
[
https://issues.apache.org/jira/browse/IMPALA-8069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16740708#comment-16740708
]
Michael Brown commented on IMPALA-8069:
---
Yep, that's it. Analysis doesn't permit me even to run that query before commit
hash {{cf89c73485c119d5068be1cdddf1acbd3c8acfd5}}, and on that commit hash, I
get the same crash.
> crash in impala::Sorter::Run::Run
> -
>
> Key: IMPALA-8069
> URL: https://issues.apache.org/jira/browse/IMPALA-8069
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 3.2.0
>Reporter: Michael Brown
>Priority: Blocker
> Labels: query_generator
>
> {noformat}
> 0 0x7fd37c9b8428 in raise () from /lib/x86_64-linux-gnu/libc.so.6
> #1 0x7fd37c9ba02a in abort () from /lib/x86_64-linux-gnu/libc.so.6
> #2 0x7fd37fb7fe49 in ?? () from
> /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so
> #3 0x7fd37fd35387 in ?? () from
> /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so
> #4 0x7fd37fb8995f in JVM_handle_linux_signal () from
> /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so
> #5 0x7fd37fb7cf78 in ?? () from
> /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so
> #6
> #7 0x02723007 in impala::Sorter::Run::Run (this=0x13485200,
> parent=0xd3d8b00, sort_tuple_desc=0xf6dd320, initial_run=true) at
> /home/mikeb/Impala/be/src/runtime/sorter.cc:609
> #8 0x0272a3cb in impala::Sorter::Open (this=0xd3d8b00) at
> /home/mikeb/Impala/be/src/runtime/sorter.cc:1514
> #9 0x0237b186 in impala::SortNode::Open (this=0xd3d8840,
> state=0xffa1ba0) at /home/mikeb/Impala/be/src/exec/sort-node.cc:84
> #10 0x0239b44b in impala::AnalyticEvalNode::Open (this=0x100aba80,
> state=0xffa1ba0) at /home/mikeb/Impala/be/src/exec/analytic-eval-node.cc:182
> #11 0x01f20cba in impala::FragmentInstanceState::Open
> (this=0xf5afd80) at
> /home/mikeb/Impala/be/src/runtime/fragment-instance-state.cc:304
> #12 0x01f1deee in impala::FragmentInstanceState::Exec
> (this=0xf5afd80) at
> /home/mikeb/Impala/be/src/runtime/fragment-instance-state.cc:84
> #13 0x01f2ea39 in impala::QueryState::ExecFInstance (this=0xea2,
> fis=0xf5afd80) at /home/mikeb/Impala/be/src/runtime/query-state.cc:584
> #14 0x01f2cd42 in impala::QueryStateoperator()(void)
> const (__closure=0x7fd2b1869ca8) at
> /home/mikeb/Impala/be/src/runtime/query-state.cc:493
> #15 0x01f2f7e1 in
> boost::detail::function::void_function_obj_invoker0,
> void>::invoke(boost::detail::function::function_buffer &) (
> function_obj_ptr=...) at
> /home/mikeb/Impala/toolchain/boost-1.57.0-p3/include/boost/function/function_template.hpp:153
> #16 0x01d46c96 in boost::function0::operator()
> (this=0x7fd2b1869ca0) at
> /home/mikeb/Impala/toolchain/boost-1.57.0-p3/include/boost/function/function_template.hpp:767
> #17 0x021d4897 in impala::Thread::SuperviseThread(std::string const&,
> std::string const&, boost::function, impala::ThreadDebugInfo const*,
> impala::Promise*) (name=..., category=...,
> functor=..., parent_thread_info=0x7fd2b206a950,
> thread_started=0x7fd2b20698f0) at /home/mikeb/Impala/be/src/util/thread.cc:359
> #18 0x021dcbb7 in boost::_bi::list5,
> boost::_bi::value, boost::_bi::value >,
> boost::_bi::value,
> boost::_bi::value*>
> >::operator() boost::function, impala::ThreadDebugInfo const*,
> impala::Promise*),
> boost::_bi::list0>(boost::_bi::type, void (*&)(std::string const&,
> std::string const&, boost::function, impala::ThreadDebugInfo const*,
> impala::Promise*), boost::_bi::list0&, int)
> (this=0xed511c0,
> f=@0xed511b8: 0x21d4530 const&, std::string const&, boost::function, impala::ThreadDebugInfo
> const*, impala::Promise*)>, a=...) at
> /home/mikeb/Impala/toolchain/boost-1.57.0-p3/include/boost/bind/bind.hpp:525
> #19 0x021dcadb in boost::_bi::bind_t const&, std::string const&, boost::function, impala::ThreadDebugInfo
> const*, impala::Promise*),
> boost::_bi::list5,
> boost::_bi::value, boost::_bi::value >,
> boost::_bi::value,
> boost::_bi::value*> >
> >::operator()() (this=0xed511b8) at
> /home/mikeb/Impala/toolchain/boost-1.57.0-p3/include/boost/bind/bind_template.hpp:20
> #20 0x021dca9e in boost::detail::thread_data void (*)(std::string const&, std::string const&, boost::function,
> impala::ThreadDebugInfo const*, impala::Promise (impala::PromiseMode)0>*), boost::_bi::list5,
> boost::_bi::value, boost::_bi::value >,
> boost::_bi::value,
> boost::_bi::value*> > >
> >::run() (this=0xed51000)
> at
> /home/mikeb/Impala/toolchain/boost-1.57.0-p3/include/boost/thread/detail/thread.hpp:116
> #21 0x036c17fa in thread_proxy ()
> #22
[jira] [Commented] (IMPALA-8069) crash in impala::Sorter::Run::Run
[
https://issues.apache.org/jira/browse/IMPALA-8069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16740700#comment-16740700
]
Michael Brown commented on IMPALA-8069:
---
Oh, I think I know why this happens now: IMPALA-6323 Allow constant analytic
window expressions.
> crash in impala::Sorter::Run::Run
> -
>
> Key: IMPALA-8069
> URL: https://issues.apache.org/jira/browse/IMPALA-8069
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 3.2.0
>Reporter: Michael Brown
>Priority: Blocker
> Labels: query_generator
>
> {noformat}
> 0 0x7fd37c9b8428 in raise () from /lib/x86_64-linux-gnu/libc.so.6
> #1 0x7fd37c9ba02a in abort () from /lib/x86_64-linux-gnu/libc.so.6
> #2 0x7fd37fb7fe49 in ?? () from
> /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so
> #3 0x7fd37fd35387 in ?? () from
> /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so
> #4 0x7fd37fb8995f in JVM_handle_linux_signal () from
> /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so
> #5 0x7fd37fb7cf78 in ?? () from
> /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so
> #6
> #7 0x02723007 in impala::Sorter::Run::Run (this=0x13485200,
> parent=0xd3d8b00, sort_tuple_desc=0xf6dd320, initial_run=true) at
> /home/mikeb/Impala/be/src/runtime/sorter.cc:609
> #8 0x0272a3cb in impala::Sorter::Open (this=0xd3d8b00) at
> /home/mikeb/Impala/be/src/runtime/sorter.cc:1514
> #9 0x0237b186 in impala::SortNode::Open (this=0xd3d8840,
> state=0xffa1ba0) at /home/mikeb/Impala/be/src/exec/sort-node.cc:84
> #10 0x0239b44b in impala::AnalyticEvalNode::Open (this=0x100aba80,
> state=0xffa1ba0) at /home/mikeb/Impala/be/src/exec/analytic-eval-node.cc:182
> #11 0x01f20cba in impala::FragmentInstanceState::Open
> (this=0xf5afd80) at
> /home/mikeb/Impala/be/src/runtime/fragment-instance-state.cc:304
> #12 0x01f1deee in impala::FragmentInstanceState::Exec
> (this=0xf5afd80) at
> /home/mikeb/Impala/be/src/runtime/fragment-instance-state.cc:84
> #13 0x01f2ea39 in impala::QueryState::ExecFInstance (this=0xea2,
> fis=0xf5afd80) at /home/mikeb/Impala/be/src/runtime/query-state.cc:584
> #14 0x01f2cd42 in impala::QueryStateoperator()(void)
> const (__closure=0x7fd2b1869ca8) at
> /home/mikeb/Impala/be/src/runtime/query-state.cc:493
> #15 0x01f2f7e1 in
> boost::detail::function::void_function_obj_invoker0,
> void>::invoke(boost::detail::function::function_buffer &) (
> function_obj_ptr=...) at
> /home/mikeb/Impala/toolchain/boost-1.57.0-p3/include/boost/function/function_template.hpp:153
> #16 0x01d46c96 in boost::function0::operator()
> (this=0x7fd2b1869ca0) at
> /home/mikeb/Impala/toolchain/boost-1.57.0-p3/include/boost/function/function_template.hpp:767
> #17 0x021d4897 in impala::Thread::SuperviseThread(std::string const&,
> std::string const&, boost::function, impala::ThreadDebugInfo const*,
> impala::Promise*) (name=..., category=...,
> functor=..., parent_thread_info=0x7fd2b206a950,
> thread_started=0x7fd2b20698f0) at /home/mikeb/Impala/be/src/util/thread.cc:359
> #18 0x021dcbb7 in boost::_bi::list5,
> boost::_bi::value, boost::_bi::value >,
> boost::_bi::value,
> boost::_bi::value*>
> >::operator() boost::function, impala::ThreadDebugInfo const*,
> impala::Promise*),
> boost::_bi::list0>(boost::_bi::type, void (*&)(std::string const&,
> std::string const&, boost::function, impala::ThreadDebugInfo const*,
> impala::Promise*), boost::_bi::list0&, int)
> (this=0xed511c0,
> f=@0xed511b8: 0x21d4530 const&, std::string const&, boost::function, impala::ThreadDebugInfo
> const*, impala::Promise*)>, a=...) at
> /home/mikeb/Impala/toolchain/boost-1.57.0-p3/include/boost/bind/bind.hpp:525
> #19 0x021dcadb in boost::_bi::bind_t const&, std::string const&, boost::function, impala::ThreadDebugInfo
> const*, impala::Promise*),
> boost::_bi::list5,
> boost::_bi::value, boost::_bi::value >,
> boost::_bi::value,
> boost::_bi::value*> >
> >::operator()() (this=0xed511b8) at
> /home/mikeb/Impala/toolchain/boost-1.57.0-p3/include/boost/bind/bind_template.hpp:20
> #20 0x021dca9e in boost::detail::thread_data void (*)(std::string const&, std::string const&, boost::function,
> impala::ThreadDebugInfo const*, impala::Promise (impala::PromiseMode)0>*), boost::_bi::list5,
> boost::_bi::value, boost::_bi::value >,
> boost::_bi::value,
> boost::_bi::value*> > >
> >::run() (this=0xed51000)
> at
> /home/mikeb/Impala/toolchain/boost-1.57.0-p3/include/boost/thread/detail/thread.hpp:116
> #21 0x036c17fa in thread_proxy ()
> #22 0x7fd37cd546ba in start_thread () from
>
[jira] [Commented] (IMPALA-8069) crash in impala::Sorter::Run::Run
[
https://issues.apache.org/jira/browse/IMPALA-8069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16740610#comment-16740610
]
Michael Brown commented on IMPALA-8069:
---
If my Git is right this may have been around since
{{37ec25396f2964c386655ce3408b32a73f71edf6}} ? Cc [~tarmstrong]. This assumes
the bug really lies here, and a contract isn't broken elsewhere.
> crash in impala::Sorter::Run::Run
> -
>
> Key: IMPALA-8069
> URL: https://issues.apache.org/jira/browse/IMPALA-8069
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 3.2.0
>Reporter: Michael Brown
>Priority: Blocker
> Labels: query_generator
>
> {noformat}
> 0 0x7fd37c9b8428 in raise () from /lib/x86_64-linux-gnu/libc.so.6
> #1 0x7fd37c9ba02a in abort () from /lib/x86_64-linux-gnu/libc.so.6
> #2 0x7fd37fb7fe49 in ?? () from
> /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so
> #3 0x7fd37fd35387 in ?? () from
> /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so
> #4 0x7fd37fb8995f in JVM_handle_linux_signal () from
> /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so
> #5 0x7fd37fb7cf78 in ?? () from
> /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so
> #6
> #7 0x02723007 in impala::Sorter::Run::Run (this=0x13485200,
> parent=0xd3d8b00, sort_tuple_desc=0xf6dd320, initial_run=true) at
> /home/mikeb/Impala/be/src/runtime/sorter.cc:609
> #8 0x0272a3cb in impala::Sorter::Open (this=0xd3d8b00) at
> /home/mikeb/Impala/be/src/runtime/sorter.cc:1514
> #9 0x0237b186 in impala::SortNode::Open (this=0xd3d8840,
> state=0xffa1ba0) at /home/mikeb/Impala/be/src/exec/sort-node.cc:84
> #10 0x0239b44b in impala::AnalyticEvalNode::Open (this=0x100aba80,
> state=0xffa1ba0) at /home/mikeb/Impala/be/src/exec/analytic-eval-node.cc:182
> #11 0x01f20cba in impala::FragmentInstanceState::Open
> (this=0xf5afd80) at
> /home/mikeb/Impala/be/src/runtime/fragment-instance-state.cc:304
> #12 0x01f1deee in impala::FragmentInstanceState::Exec
> (this=0xf5afd80) at
> /home/mikeb/Impala/be/src/runtime/fragment-instance-state.cc:84
> #13 0x01f2ea39 in impala::QueryState::ExecFInstance (this=0xea2,
> fis=0xf5afd80) at /home/mikeb/Impala/be/src/runtime/query-state.cc:584
> #14 0x01f2cd42 in impala::QueryStateoperator()(void)
> const (__closure=0x7fd2b1869ca8) at
> /home/mikeb/Impala/be/src/runtime/query-state.cc:493
> #15 0x01f2f7e1 in
> boost::detail::function::void_function_obj_invoker0,
> void>::invoke(boost::detail::function::function_buffer &) (
> function_obj_ptr=...) at
> /home/mikeb/Impala/toolchain/boost-1.57.0-p3/include/boost/function/function_template.hpp:153
> #16 0x01d46c96 in boost::function0::operator()
> (this=0x7fd2b1869ca0) at
> /home/mikeb/Impala/toolchain/boost-1.57.0-p3/include/boost/function/function_template.hpp:767
> #17 0x021d4897 in impala::Thread::SuperviseThread(std::string const&,
> std::string const&, boost::function, impala::ThreadDebugInfo const*,
> impala::Promise*) (name=..., category=...,
> functor=..., parent_thread_info=0x7fd2b206a950,
> thread_started=0x7fd2b20698f0) at /home/mikeb/Impala/be/src/util/thread.cc:359
> #18 0x021dcbb7 in boost::_bi::list5,
> boost::_bi::value, boost::_bi::value >,
> boost::_bi::value,
> boost::_bi::value*>
> >::operator() boost::function, impala::ThreadDebugInfo const*,
> impala::Promise*),
> boost::_bi::list0>(boost::_bi::type, void (*&)(std::string const&,
> std::string const&, boost::function, impala::ThreadDebugInfo const*,
> impala::Promise*), boost::_bi::list0&, int)
> (this=0xed511c0,
> f=@0xed511b8: 0x21d4530 const&, std::string const&, boost::function, impala::ThreadDebugInfo
> const*, impala::Promise*)>, a=...) at
> /home/mikeb/Impala/toolchain/boost-1.57.0-p3/include/boost/bind/bind.hpp:525
> #19 0x021dcadb in boost::_bi::bind_t const&, std::string const&, boost::function, impala::ThreadDebugInfo
> const*, impala::Promise*),
> boost::_bi::list5,
> boost::_bi::value, boost::_bi::value >,
> boost::_bi::value,
> boost::_bi::value*> >
> >::operator()() (this=0xed511b8) at
> /home/mikeb/Impala/toolchain/boost-1.57.0-p3/include/boost/bind/bind_template.hpp:20
> #20 0x021dca9e in boost::detail::thread_data void (*)(std::string const&, std::string const&, boost::function,
> impala::ThreadDebugInfo const*, impala::Promise (impala::PromiseMode)0>*), boost::_bi::list5,
> boost::_bi::value, boost::_bi::value >,
> boost::_bi::value,
> boost::_bi::value*> > >
> >::run() (this=0xed51000)
> at
> /home/mikeb/Impala/toolchain/boost-1.57.0-p3/include/boost/thread/detail/thread.hpp:116
> #21 0x036c17fa in
[jira] [Created] (IMPALA-8069) crash in impala::Sorter::Run::Run
Michael Brown created IMPALA-8069:
-
Summary: crash in impala::Sorter::Run::Run
Key: IMPALA-8069
URL: https://issues.apache.org/jira/browse/IMPALA-8069
Project: IMPALA
Issue Type: Bug
Components: Backend
Affects Versions: Impala 3.2.0
Reporter: Michael Brown
{noformat}
0 0x7fd37c9b8428 in raise () from /lib/x86_64-linux-gnu/libc.so.6
#1 0x7fd37c9ba02a in abort () from /lib/x86_64-linux-gnu/libc.so.6
#2 0x7fd37fb7fe49 in ?? () from
/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so
#3 0x7fd37fd35387 in ?? () from
/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so
#4 0x7fd37fb8995f in JVM_handle_linux_signal () from
/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so
#5 0x7fd37fb7cf78 in ?? () from
/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so
#6
#7 0x02723007 in impala::Sorter::Run::Run (this=0x13485200,
parent=0xd3d8b00, sort_tuple_desc=0xf6dd320, initial_run=true) at
/home/mikeb/Impala/be/src/runtime/sorter.cc:609
#8 0x0272a3cb in impala::Sorter::Open (this=0xd3d8b00) at
/home/mikeb/Impala/be/src/runtime/sorter.cc:1514
#9 0x0237b186 in impala::SortNode::Open (this=0xd3d8840,
state=0xffa1ba0) at /home/mikeb/Impala/be/src/exec/sort-node.cc:84
#10 0x0239b44b in impala::AnalyticEvalNode::Open (this=0x100aba80,
state=0xffa1ba0) at /home/mikeb/Impala/be/src/exec/analytic-eval-node.cc:182
#11 0x01f20cba in impala::FragmentInstanceState::Open (this=0xf5afd80)
at /home/mikeb/Impala/be/src/runtime/fragment-instance-state.cc:304
#12 0x01f1deee in impala::FragmentInstanceState::Exec (this=0xf5afd80)
at /home/mikeb/Impala/be/src/runtime/fragment-instance-state.cc:84
#13 0x01f2ea39 in impala::QueryState::ExecFInstance (this=0xea2,
fis=0xf5afd80) at /home/mikeb/Impala/be/src/runtime/query-state.cc:584
#14 0x01f2cd42 in impala::QueryStateoperator()(void)
const (__closure=0x7fd2b1869ca8) at
/home/mikeb/Impala/be/src/runtime/query-state.cc:493
#15 0x01f2f7e1 in
boost::detail::function::void_function_obj_invoker0,
void>::invoke(boost::detail::function::function_buffer &) (
function_obj_ptr=...) at
/home/mikeb/Impala/toolchain/boost-1.57.0-p3/include/boost/function/function_template.hpp:153
#16 0x01d46c96 in boost::function0::operator()
(this=0x7fd2b1869ca0) at
/home/mikeb/Impala/toolchain/boost-1.57.0-p3/include/boost/function/function_template.hpp:767
#17 0x021d4897 in impala::Thread::SuperviseThread(std::string const&,
std::string const&, boost::function, impala::ThreadDebugInfo const*,
impala::Promise*) (name=..., category=...,
functor=..., parent_thread_info=0x7fd2b206a950, thread_started=0x7fd2b20698f0)
at /home/mikeb/Impala/be/src/util/thread.cc:359
#18 0x021dcbb7 in boost::_bi::list5,
boost::_bi::value, boost::_bi::value >,
boost::_bi::value,
boost::_bi::value*>
>::operator(), impala::ThreadDebugInfo const*, impala::Promise*), boost::_bi::list0>(boost::_bi::type, void
(*&)(std::string const&, std::string const&, boost::function,
impala::ThreadDebugInfo const*, impala::Promise*), boost::_bi::list0&, int) (this=0xed511c0,
f=@0xed511b8: 0x21d4530 , impala::ThreadDebugInfo
const*, impala::Promise*)>, a=...) at
/home/mikeb/Impala/toolchain/boost-1.57.0-p3/include/boost/bind/bind.hpp:525
#19 0x021dcadb in boost::_bi::bind_t, impala::ThreadDebugInfo const*,
impala::Promise*),
boost::_bi::list5,
boost::_bi::value, boost::_bi::value >,
boost::_bi::value,
boost::_bi::value*> >
>::operator()() (this=0xed511b8) at
/home/mikeb/Impala/toolchain/boost-1.57.0-p3/include/boost/bind/bind_template.hpp:20
#20 0x021dca9e in boost::detail::thread_data,
impala::ThreadDebugInfo const*, impala::Promise*), boost::_bi::list5,
boost::_bi::value, boost::_bi::value >,
boost::_bi::value,
boost::_bi::value*> > > >::run()
(this=0xed51000)
at
/home/mikeb/Impala/toolchain/boost-1.57.0-p3/include/boost/thread/detail/thread.hpp:116
#21 0x036c17fa in thread_proxy ()
#22 0x7fd37cd546ba in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#23 0x7fd37ca8a41d in clone () from /lib/x86_64-linux-gnu/libc.so.6
{noformat}
{noformat}
605 Sorter::Run::Run(Sorter* parent, TupleDescriptor* sort_tuple_desc, bool
initial_run)
606 : sorter_(parent),
607 sort_tuple_desc_(sort_tuple_desc),
608 sort_tuple_size_(sort_tuple_desc->byte_size()),
609 page_capacity_(parent->page_len_ / sort_tuple_size_),
610 has_var_len_slots_(sort_tuple_desc->HasVarlenSlots()),
611 initial_run_(initial_run),
612 is_pinned_(initial_run),
613 is_finalized_(false),
614 is_sorted_(!initial_run),
615 num_tuples_(0) {}
{noformat}
{{sort_tuple_size_}} is 0:
{noformat}
(gdb) f
#7 0x02723007 in
[jira] [Created] (IMPALA-8069) crash in impala::Sorter::Run::Run
Michael Brown created IMPALA-8069:
-
Summary: crash in impala::Sorter::Run::Run
Key: IMPALA-8069
URL: https://issues.apache.org/jira/browse/IMPALA-8069
Project: IMPALA
Issue Type: Bug
Components: Backend
Affects Versions: Impala 3.2.0
Reporter: Michael Brown
{noformat}
0 0x7fd37c9b8428 in raise () from /lib/x86_64-linux-gnu/libc.so.6
#1 0x7fd37c9ba02a in abort () from /lib/x86_64-linux-gnu/libc.so.6
#2 0x7fd37fb7fe49 in ?? () from
/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so
#3 0x7fd37fd35387 in ?? () from
/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so
#4 0x7fd37fb8995f in JVM_handle_linux_signal () from
/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so
#5 0x7fd37fb7cf78 in ?? () from
/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so
#6
#7 0x02723007 in impala::Sorter::Run::Run (this=0x13485200,
parent=0xd3d8b00, sort_tuple_desc=0xf6dd320, initial_run=true) at
/home/mikeb/Impala/be/src/runtime/sorter.cc:609
#8 0x0272a3cb in impala::Sorter::Open (this=0xd3d8b00) at
/home/mikeb/Impala/be/src/runtime/sorter.cc:1514
#9 0x0237b186 in impala::SortNode::Open (this=0xd3d8840,
state=0xffa1ba0) at /home/mikeb/Impala/be/src/exec/sort-node.cc:84
#10 0x0239b44b in impala::AnalyticEvalNode::Open (this=0x100aba80,
state=0xffa1ba0) at /home/mikeb/Impala/be/src/exec/analytic-eval-node.cc:182
#11 0x01f20cba in impala::FragmentInstanceState::Open (this=0xf5afd80)
at /home/mikeb/Impala/be/src/runtime/fragment-instance-state.cc:304
#12 0x01f1deee in impala::FragmentInstanceState::Exec (this=0xf5afd80)
at /home/mikeb/Impala/be/src/runtime/fragment-instance-state.cc:84
#13 0x01f2ea39 in impala::QueryState::ExecFInstance (this=0xea2,
fis=0xf5afd80) at /home/mikeb/Impala/be/src/runtime/query-state.cc:584
#14 0x01f2cd42 in impala::QueryStateoperator()(void)
const (__closure=0x7fd2b1869ca8) at
/home/mikeb/Impala/be/src/runtime/query-state.cc:493
#15 0x01f2f7e1 in
boost::detail::function::void_function_obj_invoker0,
void>::invoke(boost::detail::function::function_buffer &) (
function_obj_ptr=...) at
/home/mikeb/Impala/toolchain/boost-1.57.0-p3/include/boost/function/function_template.hpp:153
#16 0x01d46c96 in boost::function0::operator()
(this=0x7fd2b1869ca0) at
/home/mikeb/Impala/toolchain/boost-1.57.0-p3/include/boost/function/function_template.hpp:767
#17 0x021d4897 in impala::Thread::SuperviseThread(std::string const&,
std::string const&, boost::function, impala::ThreadDebugInfo const*,
impala::Promise*) (name=..., category=...,
functor=..., parent_thread_info=0x7fd2b206a950, thread_started=0x7fd2b20698f0)
at /home/mikeb/Impala/be/src/util/thread.cc:359
#18 0x021dcbb7 in boost::_bi::list5,
boost::_bi::value, boost::_bi::value >,
boost::_bi::value,
boost::_bi::value*>
>::operator(), impala::ThreadDebugInfo const*, impala::Promise*), boost::_bi::list0>(boost::_bi::type, void
(*&)(std::string const&, std::string const&, boost::function,
impala::ThreadDebugInfo const*, impala::Promise*), boost::_bi::list0&, int) (this=0xed511c0,
f=@0xed511b8: 0x21d4530 , impala::ThreadDebugInfo
const*, impala::Promise*)>, a=...) at
/home/mikeb/Impala/toolchain/boost-1.57.0-p3/include/boost/bind/bind.hpp:525
#19 0x021dcadb in boost::_bi::bind_t, impala::ThreadDebugInfo const*,
impala::Promise*),
boost::_bi::list5,
boost::_bi::value, boost::_bi::value >,
boost::_bi::value,
boost::_bi::value*> >
>::operator()() (this=0xed511b8) at
/home/mikeb/Impala/toolchain/boost-1.57.0-p3/include/boost/bind/bind_template.hpp:20
#20 0x021dca9e in boost::detail::thread_data,
impala::ThreadDebugInfo const*, impala::Promise*), boost::_bi::list5,
boost::_bi::value, boost::_bi::value >,
boost::_bi::value,
boost::_bi::value*> > > >::run()
(this=0xed51000)
at
/home/mikeb/Impala/toolchain/boost-1.57.0-p3/include/boost/thread/detail/thread.hpp:116
#21 0x036c17fa in thread_proxy ()
#22 0x7fd37cd546ba in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#23 0x7fd37ca8a41d in clone () from /lib/x86_64-linux-gnu/libc.so.6
{noformat}
{noformat}
605 Sorter::Run::Run(Sorter* parent, TupleDescriptor* sort_tuple_desc, bool
initial_run)
606 : sorter_(parent),
607 sort_tuple_desc_(sort_tuple_desc),
608 sort_tuple_size_(sort_tuple_desc->byte_size()),
609 page_capacity_(parent->page_len_ / sort_tuple_size_),
610 has_var_len_slots_(sort_tuple_desc->HasVarlenSlots()),
611 initial_run_(initial_run),
612 is_pinned_(initial_run),
613 is_finalized_(false),
614 is_sorted_(!initial_run),
615 num_tuples_(0) {}
{noformat}
{{sort_tuple_size_}} is 0:
{noformat}
(gdb) f
#7 0x02723007 in
[jira] [Resolved] (IMPALA-7736) stress: AssertionError: The number of executing queries is negative
[
https://issues.apache.org/jira/browse/IMPALA-7736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Brown resolved IMPALA-7736.
---
Resolution: Won't Fix
For me this happens when you run concurrent_select.py from master against a
pre-3.0 Impala. I think it should be fine to expect people to run earlier
versions of the stress test on pre-3.0 Impala clusters. People will have to
start doing that anyway now that we include queries with multiple DISTINCT
aggregations.
> stress: AssertionError: The number of executing queries is negative
> ---
>
> Key: IMPALA-7736
> URL: https://issues.apache.org/jira/browse/IMPALA-7736
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
>Affects Versions: Impala 3.1.0
>Reporter: Michael Brown
>Assignee: Michael Brown
>Priority: Critical
> Labels: stress
>
> I'd heard about this anecdotally but haven't seen it until now:
> {noformat}
> Done | Active | Executing | Mem Lmt Ex | AC Reject | AC Timeout | Time Out |
> Cancel | Err | Incorrect | Next Qry Mem Lmt | Tot Qry Mem Lmt | Tracked Mem |
> RSS Mem
> 263 | 6 | 3 | 20 | 0 | 0 |0 |
>17 | 0 | 0 | 9818 | 26802 | 15656 |
> 19329
> 263 | 6 | 3 | 20 | 0 | 0 |0 |
>17 | 0 | 0 | 9818 | 26802 | 15534 |
> 19341
> 263 | 6 | 3 | 20 | 0 | 0 |0 |
>17 | 0 | 0 | 9818 | 26802 | 15462 |
> 19190
> 265 | 4 | 1 | 20 | 0 | 0 |0 |
>17 | 0 | 0 | 9818 | 22938 | 15482 |
> 19565
> Traceback (most recent call last):
> File "tests/stress/concurrent_select.py", line 2320, in
> main()
> File "tests/stress/concurrent_select.py", line 2316, in main
> select_probability=args.select_probability)
> File "tests/stress/concurrent_select.py", line 529, in run_queries
> self._wait_for_test_to_finish(impala, should_print_status)
> File "tests/stress/concurrent_select.py", line 943, in
> _wait_for_test_to_finish
> self._print_status(print_header=(lines_printed == 0))
> File "tests/stress/concurrent_select.py", line 837, in _print_status
> self._num_queries_executing,
> File "tests/stress/concurrent_select.py", line 690, in
> _num_queries_executing
> assert num_executing >= 0, "The number of executing queries is negative"
> AssertionError: The number of executing queries is negative
> {noformat}
> I've run many thousand-query stress test runs against 3.1 without hitting
> this. Where this finally happened was against a 2.12-based build, if that
> matters.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (IMPALA-7736) stress: AssertionError: The number of executing queries is negative
[
https://issues.apache.org/jira/browse/IMPALA-7736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Brown resolved IMPALA-7736.
---
Resolution: Won't Fix
For me this happens when you run concurrent_select.py from master against a
pre-3.0 Impala. I think it should be fine to expect people to run earlier
versions of the stress test on pre-3.0 Impala clusters. People will have to
start doing that anyway now that we include queries with multiple DISTINCT
aggregations.
> stress: AssertionError: The number of executing queries is negative
> ---
>
> Key: IMPALA-7736
> URL: https://issues.apache.org/jira/browse/IMPALA-7736
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
>Affects Versions: Impala 3.1.0
>Reporter: Michael Brown
>Assignee: Michael Brown
>Priority: Critical
> Labels: stress
>
> I'd heard about this anecdotally but haven't seen it until now:
> {noformat}
> Done | Active | Executing | Mem Lmt Ex | AC Reject | AC Timeout | Time Out |
> Cancel | Err | Incorrect | Next Qry Mem Lmt | Tot Qry Mem Lmt | Tracked Mem |
> RSS Mem
> 263 | 6 | 3 | 20 | 0 | 0 |0 |
>17 | 0 | 0 | 9818 | 26802 | 15656 |
> 19329
> 263 | 6 | 3 | 20 | 0 | 0 |0 |
>17 | 0 | 0 | 9818 | 26802 | 15534 |
> 19341
> 263 | 6 | 3 | 20 | 0 | 0 |0 |
>17 | 0 | 0 | 9818 | 26802 | 15462 |
> 19190
> 265 | 4 | 1 | 20 | 0 | 0 |0 |
>17 | 0 | 0 | 9818 | 22938 | 15482 |
> 19565
> Traceback (most recent call last):
> File "tests/stress/concurrent_select.py", line 2320, in
> main()
> File "tests/stress/concurrent_select.py", line 2316, in main
> select_probability=args.select_probability)
> File "tests/stress/concurrent_select.py", line 529, in run_queries
> self._wait_for_test_to_finish(impala, should_print_status)
> File "tests/stress/concurrent_select.py", line 943, in
> _wait_for_test_to_finish
> self._print_status(print_header=(lines_printed == 0))
> File "tests/stress/concurrent_select.py", line 837, in _print_status
> self._num_queries_executing,
> File "tests/stress/concurrent_select.py", line 690, in
> _num_queries_executing
> assert num_executing >= 0, "The number of executing queries is negative"
> AssertionError: The number of executing queries is negative
> {noformat}
> I've run many thousand-query stress test runs against 3.1 without hitting
> this. Where this finally happened was against a 2.12-based build, if that
> matters.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-7910) COMPUTE STATS does an unnecessary REFRESH after writing to the Metastore
Michael Brown created IMPALA-7910:
-
Summary: COMPUTE STATS does an unnecessary REFRESH after writing
to the Metastore
Key: IMPALA-7910
URL: https://issues.apache.org/jira/browse/IMPALA-7910
Project: IMPALA
Issue Type: Bug
Components: Catalog
Affects Versions: Impala 2.12.0, Impala 2.11.0, Impala 2.9.0
Reporter: Michael Brown
Assignee: Tianyi Wang
COMPUTE STATS and possibly other DDL operations unnecessarily do the equivalent
of a REFRESH after writing to the Hive Metastore. This unnecessary operation
can be very expensive, so should be avoided.
The behavior can be confirmed from the catalogd logs:
{code}
compute stats functional_parquet.alltypes;
+---+
| summary |
+---+
| Updated 24 partition(s) and 11 column(s). |
+---+
Relevant catalogd.INFO snippet
I0413 14:40:24.210749 27295 HdfsTable.java:1263] Incrementally loading table
metadata for: functional_parquet.alltypes
I0413 14:40:24.242122 27295 HdfsTable.java:555] Refreshed file metadata for
functional_parquet.alltypes Path:
hdfs://localhost:20500/test-warehouse/alltypes_parquet/year=2009/month=1:
Loaded files: 1 Hidden files: 0 Skipped files: 0 Unknown diskIDs: 0
I0413 14:40:24.244634 27295 HdfsTable.java:555] Refreshed file metadata for
functional_parquet.alltypes Path:
hdfs://localhost:20500/test-warehouse/alltypes_parquet/year=2009/month=10:
Loaded files: 1 Hidden files: 0 Skipped files: 0 Unknown diskIDs: 0
I0413 14:40:24.247174 27295 HdfsTable.java:555] Refreshed file metadata for
functional_parquet.alltypes Path:
hdfs://localhost:20500/test-warehouse/alltypes_parquet/year=2009/month=11:
Loaded files: 1 Hidden files: 0 Skipped files: 0 Unknown diskIDs: 0
I0413 14:40:24.249713 27295 HdfsTable.java:555] Refreshed file metadata for
functional_parquet.alltypes Path:
hdfs://localhost:20500/test-warehouse/alltypes_parquet/year=2009/month=12:
Loaded files: 1 Hidden files: 0 Skipped files: 0 Unknown diskIDs: 0
I0413 14:40:24.252288 27295 HdfsTable.java:555] Refreshed file metadata for
functional_parquet.alltypes Path:
hdfs://localhost:20500/test-warehouse/alltypes_parquet/year=2009/month=2:
Loaded files: 1 Hidden files: 0 Skipped files: 0 Unknown diskIDs: 0
I0413 14:40:24.254629 27295 HdfsTable.java:555] Refreshed file metadata for
functional_parquet.alltypes Path:
hdfs://localhost:20500/test-warehouse/alltypes_parquet/year=2009/month=3:
Loaded files: 1 Hidden files: 0 Skipped files: 0 Unknown diskIDs: 0
I0413 14:40:24.256991 27295 HdfsTable.java:555] Refreshed file metadata for
functional_parquet.alltypes Path:
hdfs://localhost:20500/test-warehouse/alltypes_parquet/year=2009/month=4:
Loaded files: 1 Hidden files: 0 Skipped files: 0 Unknown diskIDs: 0
I0413 14:40:24.259464 27295 HdfsTable.java:555] Refreshed file metadata for
functional_parquet.alltypes Path:
hdfs://localhost:20500/test-warehouse/alltypes_parquet/year=2009/month=5:
Loaded files: 1 Hidden files: 0 Skipped files: 0 Unknown diskIDs: 0
I0413 14:40:24.262197 27295 HdfsTable.java:555] Refreshed file metadata for
functional_parquet.alltypes Path:
hdfs://localhost:20500/test-warehouse/alltypes_parquet/year=2009/month=6:
Loaded files: 1 Hidden files: 0 Skipped files: 0 Unknown diskIDs: 0
I0413 14:40:24.264463 27295 HdfsTable.java:555] Refreshed file metadata for
functional_parquet.alltypes Path:
hdfs://localhost:20500/test-warehouse/alltypes_parquet/year=2009/month=7:
Loaded files: 1 Hidden files: 0 Skipped files: 0 Unknown diskIDs: 0
I0413 14:40:24.266736 27295 HdfsTable.java:555] Refreshed file metadata for
functional_parquet.alltypes Path:
hdfs://localhost:20500/test-warehouse/alltypes_parquet/year=2009/month=8:
Loaded files: 1 Hidden files: 0 Skipped files: 0 Unknown diskIDs: 0
I0413 14:40:24.269210 27295 HdfsTable.java:555] Refreshed file metadata for
functional_parquet.alltypes Path:
hdfs://localhost:20500/test-warehouse/alltypes_parquet/year=2009/month=9:
Loaded files: 1 Hidden files: 0 Skipped files: 0 Unknown diskIDs: 0
I0413 14:40:24.271800 27295 HdfsTable.java:555] Refreshed file metadata for
functional_parquet.alltypes Path:
hdfs://localhost:20500/test-warehouse/alltypes_parquet/year=2010/month=1:
Loaded files: 1 Hidden files: 0 Skipped files: 0 Unknown diskIDs: 0
I0413 14:40:24.274348 27295 HdfsTable.java:555] Refreshed file metadata for
functional_parquet.alltypes Path:
hdfs://localhost:20500/test-warehouse/alltypes_parquet/year=2010/month=10:
Loaded files: 1 Hidden files: 0 Skipped files: 0 Unknown diskIDs: 0
I0413 14:40:24.277053 27295 HdfsTable.java:555] Refreshed file metadata for
functional_parquet.alltypes Path:
hdfs://localhost:20500/test-warehouse/alltypes_parquet/year=2010/month=11:
Loaded files: 1 Hidden files: 0 Skipped files:
[jira] [Created] (IMPALA-7910) COMPUTE STATS does an unnecessary REFRESH after writing to the Metastore
Michael Brown created IMPALA-7910:
-
Summary: COMPUTE STATS does an unnecessary REFRESH after writing
to the Metastore
Key: IMPALA-7910
URL: https://issues.apache.org/jira/browse/IMPALA-7910
Project: IMPALA
Issue Type: Bug
Components: Catalog
Affects Versions: Impala 2.12.0, Impala 2.11.0, Impala 2.9.0
Reporter: Michael Brown
Assignee: Tianyi Wang
COMPUTE STATS and possibly other DDL operations unnecessarily do the equivalent
of a REFRESH after writing to the Hive Metastore. This unnecessary operation
can be very expensive, so should be avoided.
The behavior can be confirmed from the catalogd logs:
{code}
compute stats functional_parquet.alltypes;
+---+
| summary |
+---+
| Updated 24 partition(s) and 11 column(s). |
+---+
Relevant catalogd.INFO snippet
I0413 14:40:24.210749 27295 HdfsTable.java:1263] Incrementally loading table
metadata for: functional_parquet.alltypes
I0413 14:40:24.242122 27295 HdfsTable.java:555] Refreshed file metadata for
functional_parquet.alltypes Path:
hdfs://localhost:20500/test-warehouse/alltypes_parquet/year=2009/month=1:
Loaded files: 1 Hidden files: 0 Skipped files: 0 Unknown diskIDs: 0
I0413 14:40:24.244634 27295 HdfsTable.java:555] Refreshed file metadata for
functional_parquet.alltypes Path:
hdfs://localhost:20500/test-warehouse/alltypes_parquet/year=2009/month=10:
Loaded files: 1 Hidden files: 0 Skipped files: 0 Unknown diskIDs: 0
I0413 14:40:24.247174 27295 HdfsTable.java:555] Refreshed file metadata for
functional_parquet.alltypes Path:
hdfs://localhost:20500/test-warehouse/alltypes_parquet/year=2009/month=11:
Loaded files: 1 Hidden files: 0 Skipped files: 0 Unknown diskIDs: 0
I0413 14:40:24.249713 27295 HdfsTable.java:555] Refreshed file metadata for
functional_parquet.alltypes Path:
hdfs://localhost:20500/test-warehouse/alltypes_parquet/year=2009/month=12:
Loaded files: 1 Hidden files: 0 Skipped files: 0 Unknown diskIDs: 0
I0413 14:40:24.252288 27295 HdfsTable.java:555] Refreshed file metadata for
functional_parquet.alltypes Path:
hdfs://localhost:20500/test-warehouse/alltypes_parquet/year=2009/month=2:
Loaded files: 1 Hidden files: 0 Skipped files: 0 Unknown diskIDs: 0
I0413 14:40:24.254629 27295 HdfsTable.java:555] Refreshed file metadata for
functional_parquet.alltypes Path:
hdfs://localhost:20500/test-warehouse/alltypes_parquet/year=2009/month=3:
Loaded files: 1 Hidden files: 0 Skipped files: 0 Unknown diskIDs: 0
I0413 14:40:24.256991 27295 HdfsTable.java:555] Refreshed file metadata for
functional_parquet.alltypes Path:
hdfs://localhost:20500/test-warehouse/alltypes_parquet/year=2009/month=4:
Loaded files: 1 Hidden files: 0 Skipped files: 0 Unknown diskIDs: 0
I0413 14:40:24.259464 27295 HdfsTable.java:555] Refreshed file metadata for
functional_parquet.alltypes Path:
hdfs://localhost:20500/test-warehouse/alltypes_parquet/year=2009/month=5:
Loaded files: 1 Hidden files: 0 Skipped files: 0 Unknown diskIDs: 0
I0413 14:40:24.262197 27295 HdfsTable.java:555] Refreshed file metadata for
functional_parquet.alltypes Path:
hdfs://localhost:20500/test-warehouse/alltypes_parquet/year=2009/month=6:
Loaded files: 1 Hidden files: 0 Skipped files: 0 Unknown diskIDs: 0
I0413 14:40:24.264463 27295 HdfsTable.java:555] Refreshed file metadata for
functional_parquet.alltypes Path:
hdfs://localhost:20500/test-warehouse/alltypes_parquet/year=2009/month=7:
Loaded files: 1 Hidden files: 0 Skipped files: 0 Unknown diskIDs: 0
I0413 14:40:24.266736 27295 HdfsTable.java:555] Refreshed file metadata for
functional_parquet.alltypes Path:
hdfs://localhost:20500/test-warehouse/alltypes_parquet/year=2009/month=8:
Loaded files: 1 Hidden files: 0 Skipped files: 0 Unknown diskIDs: 0
I0413 14:40:24.269210 27295 HdfsTable.java:555] Refreshed file metadata for
functional_parquet.alltypes Path:
hdfs://localhost:20500/test-warehouse/alltypes_parquet/year=2009/month=9:
Loaded files: 1 Hidden files: 0 Skipped files: 0 Unknown diskIDs: 0
I0413 14:40:24.271800 27295 HdfsTable.java:555] Refreshed file metadata for
functional_parquet.alltypes Path:
hdfs://localhost:20500/test-warehouse/alltypes_parquet/year=2010/month=1:
Loaded files: 1 Hidden files: 0 Skipped files: 0 Unknown diskIDs: 0
I0413 14:40:24.274348 27295 HdfsTable.java:555] Refreshed file metadata for
functional_parquet.alltypes Path:
hdfs://localhost:20500/test-warehouse/alltypes_parquet/year=2010/month=10:
Loaded files: 1 Hidden files: 0 Skipped files: 0 Unknown diskIDs: 0
I0413 14:40:24.277053 27295 HdfsTable.java:555] Refreshed file metadata for
functional_parquet.alltypes Path:
hdfs://localhost:20500/test-warehouse/alltypes_parquet/year=2010/month=11:
Loaded files: 1 Hidden files: 0 Skipped files:
[jira] [Deleted] (IMPALA-6853) COMPUTE STATS does an unnecessary REFRESH after writing to the Metastore
[
https://issues.apache.org/jira/browse/IMPALA-6853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Brown deleted IMPALA-6853:
--
> COMPUTE STATS does an unnecessary REFRESH after writing to the Metastore
>
>
> Key: IMPALA-6853
> URL: https://issues.apache.org/jira/browse/IMPALA-6853
> Project: IMPALA
> Issue Type: Bug
>Reporter: Alexander Behm
>Assignee: Tianyi Wang
>Priority: Critical
> Labels: compute-stats, perfomance
>
> COMPUTE STATS and possibly other DDL operations unnecessarily do the
> equivalent of a REFRESH after writing to the Hive Metastore. This unnecessary
> operation can be very expensive, so should be avoided.
> The behavior can be confirmed from the catalogd logs:
> {code}
> compute stats functional_parquet.alltypes;
> +---+
> | summary |
> +---+
> | Updated 24 partition(s) and 11 column(s). |
> +---+
> Relevant catalogd.INFO snippet
> I0413 14:40:24.210749 27295 HdfsTable.java:1263] Incrementally loading table
> metadata for: functional_parquet.alltypes
> I0413 14:40:24.242122 27295 HdfsTable.java:555] Refreshed file metadata for
> functional_parquet.alltypes Path:
> hdfs://localhost:20500/test-warehouse/alltypes_parquet/year=2009/month=1:
> Loaded files: 1 Hidden files: 0 Skipped files: 0 Unknown diskIDs: 0
> I0413 14:40:24.244634 27295 HdfsTable.java:555] Refreshed file metadata for
> functional_parquet.alltypes Path:
> hdfs://localhost:20500/test-warehouse/alltypes_parquet/year=2009/month=10:
> Loaded files: 1 Hidden files: 0 Skipped files: 0 Unknown diskIDs: 0
> I0413 14:40:24.247174 27295 HdfsTable.java:555] Refreshed file metadata for
> functional_parquet.alltypes Path:
> hdfs://localhost:20500/test-warehouse/alltypes_parquet/year=2009/month=11:
> Loaded files: 1 Hidden files: 0 Skipped files: 0 Unknown diskIDs: 0
> I0413 14:40:24.249713 27295 HdfsTable.java:555] Refreshed file metadata for
> functional_parquet.alltypes Path:
> hdfs://localhost:20500/test-warehouse/alltypes_parquet/year=2009/month=12:
> Loaded files: 1 Hidden files: 0 Skipped files: 0 Unknown diskIDs: 0
> I0413 14:40:24.252288 27295 HdfsTable.java:555] Refreshed file metadata for
> functional_parquet.alltypes Path:
> hdfs://localhost:20500/test-warehouse/alltypes_parquet/year=2009/month=2:
> Loaded files: 1 Hidden files: 0 Skipped files: 0 Unknown diskIDs: 0
> I0413 14:40:24.254629 27295 HdfsTable.java:555] Refreshed file metadata for
> functional_parquet.alltypes Path:
> hdfs://localhost:20500/test-warehouse/alltypes_parquet/year=2009/month=3:
> Loaded files: 1 Hidden files: 0 Skipped files: 0 Unknown diskIDs: 0
> I0413 14:40:24.256991 27295 HdfsTable.java:555] Refreshed file metadata for
> functional_parquet.alltypes Path:
> hdfs://localhost:20500/test-warehouse/alltypes_parquet/year=2009/month=4:
> Loaded files: 1 Hidden files: 0 Skipped files: 0 Unknown diskIDs: 0
> I0413 14:40:24.259464 27295 HdfsTable.java:555] Refreshed file metadata for
> functional_parquet.alltypes Path:
> hdfs://localhost:20500/test-warehouse/alltypes_parquet/year=2009/month=5:
> Loaded files: 1 Hidden files: 0 Skipped files: 0 Unknown diskIDs: 0
> I0413 14:40:24.262197 27295 HdfsTable.java:555] Refreshed file metadata for
> functional_parquet.alltypes Path:
> hdfs://localhost:20500/test-warehouse/alltypes_parquet/year=2009/month=6:
> Loaded files: 1 Hidden files: 0 Skipped files: 0 Unknown diskIDs: 0
> I0413 14:40:24.264463 27295 HdfsTable.java:555] Refreshed file metadata for
> functional_parquet.alltypes Path:
> hdfs://localhost:20500/test-warehouse/alltypes_parquet/year=2009/month=7:
> Loaded files: 1 Hidden files: 0 Skipped files: 0 Unknown diskIDs: 0
> I0413 14:40:24.266736 27295 HdfsTable.java:555] Refreshed file metadata for
> functional_parquet.alltypes Path:
> hdfs://localhost:20500/test-warehouse/alltypes_parquet/year=2009/month=8:
> Loaded files: 1 Hidden files: 0 Skipped files: 0 Unknown diskIDs: 0
> I0413 14:40:24.269210 27295 HdfsTable.java:555] Refreshed file metadata for
> functional_parquet.alltypes Path:
> hdfs://localhost:20500/test-warehouse/alltypes_parquet/year=2009/month=9:
> Loaded files: 1 Hidden files: 0 Skipped files: 0 Unknown diskIDs: 0
> I0413 14:40:24.271800 27295 HdfsTable.java:555] Refreshed file metadata for
> functional_parquet.alltypes Path:
> hdfs://localhost:20500/test-warehouse/alltypes_parquet/year=2010/month=1:
> Loaded files: 1 Hidden files: 0 Skipped files: 0 Unknown diskIDs: 0
> I0413 14:40:24.274348 27295 HdfsTable.java:555] Refreshed file metadata for
> functional_parquet.alltypes Path:
>
[jira] [Deleted] (IMPALA-6853) COMPUTE STATS does an unnecessary REFRESH after writing to the Metastore
[
https://issues.apache.org/jira/browse/IMPALA-6853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Brown deleted IMPALA-6853:
--
> COMPUTE STATS does an unnecessary REFRESH after writing to the Metastore
>
>
> Key: IMPALA-6853
> URL: https://issues.apache.org/jira/browse/IMPALA-6853
> Project: IMPALA
> Issue Type: Bug
>Reporter: Alexander Behm
>Assignee: Tianyi Wang
>Priority: Critical
> Labels: compute-stats, perfomance
>
> COMPUTE STATS and possibly other DDL operations unnecessarily do the
> equivalent of a REFRESH after writing to the Hive Metastore. This unnecessary
> operation can be very expensive, so should be avoided.
> The behavior can be confirmed from the catalogd logs:
> {code}
> compute stats functional_parquet.alltypes;
> +---+
> | summary |
> +---+
> | Updated 24 partition(s) and 11 column(s). |
> +---+
> Relevant catalogd.INFO snippet
> I0413 14:40:24.210749 27295 HdfsTable.java:1263] Incrementally loading table
> metadata for: functional_parquet.alltypes
> I0413 14:40:24.242122 27295 HdfsTable.java:555] Refreshed file metadata for
> functional_parquet.alltypes Path:
> hdfs://localhost:20500/test-warehouse/alltypes_parquet/year=2009/month=1:
> Loaded files: 1 Hidden files: 0 Skipped files: 0 Unknown diskIDs: 0
> I0413 14:40:24.244634 27295 HdfsTable.java:555] Refreshed file metadata for
> functional_parquet.alltypes Path:
> hdfs://localhost:20500/test-warehouse/alltypes_parquet/year=2009/month=10:
> Loaded files: 1 Hidden files: 0 Skipped files: 0 Unknown diskIDs: 0
> I0413 14:40:24.247174 27295 HdfsTable.java:555] Refreshed file metadata for
> functional_parquet.alltypes Path:
> hdfs://localhost:20500/test-warehouse/alltypes_parquet/year=2009/month=11:
> Loaded files: 1 Hidden files: 0 Skipped files: 0 Unknown diskIDs: 0
> I0413 14:40:24.249713 27295 HdfsTable.java:555] Refreshed file metadata for
> functional_parquet.alltypes Path:
> hdfs://localhost:20500/test-warehouse/alltypes_parquet/year=2009/month=12:
> Loaded files: 1 Hidden files: 0 Skipped files: 0 Unknown diskIDs: 0
> I0413 14:40:24.252288 27295 HdfsTable.java:555] Refreshed file metadata for
> functional_parquet.alltypes Path:
> hdfs://localhost:20500/test-warehouse/alltypes_parquet/year=2009/month=2:
> Loaded files: 1 Hidden files: 0 Skipped files: 0 Unknown diskIDs: 0
> I0413 14:40:24.254629 27295 HdfsTable.java:555] Refreshed file metadata for
> functional_parquet.alltypes Path:
> hdfs://localhost:20500/test-warehouse/alltypes_parquet/year=2009/month=3:
> Loaded files: 1 Hidden files: 0 Skipped files: 0 Unknown diskIDs: 0
> I0413 14:40:24.256991 27295 HdfsTable.java:555] Refreshed file metadata for
> functional_parquet.alltypes Path:
> hdfs://localhost:20500/test-warehouse/alltypes_parquet/year=2009/month=4:
> Loaded files: 1 Hidden files: 0 Skipped files: 0 Unknown diskIDs: 0
> I0413 14:40:24.259464 27295 HdfsTable.java:555] Refreshed file metadata for
> functional_parquet.alltypes Path:
> hdfs://localhost:20500/test-warehouse/alltypes_parquet/year=2009/month=5:
> Loaded files: 1 Hidden files: 0 Skipped files: 0 Unknown diskIDs: 0
> I0413 14:40:24.262197 27295 HdfsTable.java:555] Refreshed file metadata for
> functional_parquet.alltypes Path:
> hdfs://localhost:20500/test-warehouse/alltypes_parquet/year=2009/month=6:
> Loaded files: 1 Hidden files: 0 Skipped files: 0 Unknown diskIDs: 0
> I0413 14:40:24.264463 27295 HdfsTable.java:555] Refreshed file metadata for
> functional_parquet.alltypes Path:
> hdfs://localhost:20500/test-warehouse/alltypes_parquet/year=2009/month=7:
> Loaded files: 1 Hidden files: 0 Skipped files: 0 Unknown diskIDs: 0
> I0413 14:40:24.266736 27295 HdfsTable.java:555] Refreshed file metadata for
> functional_parquet.alltypes Path:
> hdfs://localhost:20500/test-warehouse/alltypes_parquet/year=2009/month=8:
> Loaded files: 1 Hidden files: 0 Skipped files: 0 Unknown diskIDs: 0
> I0413 14:40:24.269210 27295 HdfsTable.java:555] Refreshed file metadata for
> functional_parquet.alltypes Path:
> hdfs://localhost:20500/test-warehouse/alltypes_parquet/year=2009/month=9:
> Loaded files: 1 Hidden files: 0 Skipped files: 0 Unknown diskIDs: 0
> I0413 14:40:24.271800 27295 HdfsTable.java:555] Refreshed file metadata for
> functional_parquet.alltypes Path:
> hdfs://localhost:20500/test-warehouse/alltypes_parquet/year=2010/month=1:
> Loaded files: 1 Hidden files: 0 Skipped files: 0 Unknown diskIDs: 0
> I0413 14:40:24.274348 27295 HdfsTable.java:555] Refreshed file metadata for
> functional_parquet.alltypes Path:
>
[jira] [Issue Comment Deleted] (IMPALA-6853) COMPUTE STATS does an unnecessary REFRESH after writing to the Metastore
[
https://issues.apache.org/jira/browse/IMPALA-6853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Brown updated IMPALA-6853:
--
Comment: was deleted
(was: Bell Canada seems to be facing this issue,)
> COMPUTE STATS does an unnecessary REFRESH after writing to the Metastore
>
>
> Key: IMPALA-6853
> URL: https://issues.apache.org/jira/browse/IMPALA-6853
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
>Affects Versions: Impala 2.9.0, Impala 2.10.0, Impala 2.11.0, Impala 2.12.0
>Reporter: Alexander Behm
>Assignee: Tianyi Wang
>Priority: Critical
> Labels: compute-stats, perfomance
>
> COMPUTE STATS and possibly other DDL operations unnecessarily do the
> equivalent of a REFRESH after writing to the Hive Metastore. This unnecessary
> operation can be very expensive, so should be avoided.
> The behavior can be confirmed from the catalogd logs:
> {code}
> compute stats functional_parquet.alltypes;
> +---+
> | summary |
> +---+
> | Updated 24 partition(s) and 11 column(s). |
> +---+
> Relevant catalogd.INFO snippet
> I0413 14:40:24.210749 27295 HdfsTable.java:1263] Incrementally loading table
> metadata for: functional_parquet.alltypes
> I0413 14:40:24.242122 27295 HdfsTable.java:555] Refreshed file metadata for
> functional_parquet.alltypes Path:
> hdfs://localhost:20500/test-warehouse/alltypes_parquet/year=2009/month=1:
> Loaded files: 1 Hidden files: 0 Skipped files: 0 Unknown diskIDs: 0
> I0413 14:40:24.244634 27295 HdfsTable.java:555] Refreshed file metadata for
> functional_parquet.alltypes Path:
> hdfs://localhost:20500/test-warehouse/alltypes_parquet/year=2009/month=10:
> Loaded files: 1 Hidden files: 0 Skipped files: 0 Unknown diskIDs: 0
> I0413 14:40:24.247174 27295 HdfsTable.java:555] Refreshed file metadata for
> functional_parquet.alltypes Path:
> hdfs://localhost:20500/test-warehouse/alltypes_parquet/year=2009/month=11:
> Loaded files: 1 Hidden files: 0 Skipped files: 0 Unknown diskIDs: 0
> I0413 14:40:24.249713 27295 HdfsTable.java:555] Refreshed file metadata for
> functional_parquet.alltypes Path:
> hdfs://localhost:20500/test-warehouse/alltypes_parquet/year=2009/month=12:
> Loaded files: 1 Hidden files: 0 Skipped files: 0 Unknown diskIDs: 0
> I0413 14:40:24.252288 27295 HdfsTable.java:555] Refreshed file metadata for
> functional_parquet.alltypes Path:
> hdfs://localhost:20500/test-warehouse/alltypes_parquet/year=2009/month=2:
> Loaded files: 1 Hidden files: 0 Skipped files: 0 Unknown diskIDs: 0
> I0413 14:40:24.254629 27295 HdfsTable.java:555] Refreshed file metadata for
> functional_parquet.alltypes Path:
> hdfs://localhost:20500/test-warehouse/alltypes_parquet/year=2009/month=3:
> Loaded files: 1 Hidden files: 0 Skipped files: 0 Unknown diskIDs: 0
> I0413 14:40:24.256991 27295 HdfsTable.java:555] Refreshed file metadata for
> functional_parquet.alltypes Path:
> hdfs://localhost:20500/test-warehouse/alltypes_parquet/year=2009/month=4:
> Loaded files: 1 Hidden files: 0 Skipped files: 0 Unknown diskIDs: 0
> I0413 14:40:24.259464 27295 HdfsTable.java:555] Refreshed file metadata for
> functional_parquet.alltypes Path:
> hdfs://localhost:20500/test-warehouse/alltypes_parquet/year=2009/month=5:
> Loaded files: 1 Hidden files: 0 Skipped files: 0 Unknown diskIDs: 0
> I0413 14:40:24.262197 27295 HdfsTable.java:555] Refreshed file metadata for
> functional_parquet.alltypes Path:
> hdfs://localhost:20500/test-warehouse/alltypes_parquet/year=2009/month=6:
> Loaded files: 1 Hidden files: 0 Skipped files: 0 Unknown diskIDs: 0
> I0413 14:40:24.264463 27295 HdfsTable.java:555] Refreshed file metadata for
> functional_parquet.alltypes Path:
> hdfs://localhost:20500/test-warehouse/alltypes_parquet/year=2009/month=7:
> Loaded files: 1 Hidden files: 0 Skipped files: 0 Unknown diskIDs: 0
> I0413 14:40:24.266736 27295 HdfsTable.java:555] Refreshed file metadata for
> functional_parquet.alltypes Path:
> hdfs://localhost:20500/test-warehouse/alltypes_parquet/year=2009/month=8:
> Loaded files: 1 Hidden files: 0 Skipped files: 0 Unknown diskIDs: 0
> I0413 14:40:24.269210 27295 HdfsTable.java:555] Refreshed file metadata for
> functional_parquet.alltypes Path:
> hdfs://localhost:20500/test-warehouse/alltypes_parquet/year=2009/month=9:
> Loaded files: 1 Hidden files: 0 Skipped files: 0 Unknown diskIDs: 0
> I0413 14:40:24.271800 27295 HdfsTable.java:555] Refreshed file metadata for
> functional_parquet.alltypes Path:
> hdfs://localhost:20500/test-warehouse/alltypes_parquet/year=2010/month=1:
> Loaded files: 1 Hidden files: 0 Skipped files: 0 Unknown diskIDs:
[jira] [Assigned] (IMPALA-7901) Docs problem describing Kudu partitioning syntax
[
https://issues.apache.org/jira/browse/IMPALA-7901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Brown reassigned IMPALA-7901:
-
Assignee: Alex Rodoni
> Docs problem describing Kudu partitioning syntax
>
>
> Key: IMPALA-7901
> URL: https://issues.apache.org/jira/browse/IMPALA-7901
> Project: IMPALA
> Issue Type: Bug
> Components: Docs
>Affects Versions: Impala 3.0
>Reporter: Mike Percy
>Assignee: Alex Rodoni
>Priority: Major
>
> The Impala partitioning syntax @
> [https://impala.apache.org/docs/build/html/topics/impala_create_table.html]
> has the following in the Kudu section:
> *kudu_partition_clause:*
> {noformat}
> kudu_partition_clause ::= [hash_clause] [, range_clause [ , range_clause ]
> ]{noformat}
> That is wrong: the hash clause may appear any number of times, while the
> range_clause may only appear 0 or 1 times.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-7863) To solve Gmail Server Error 007
[ https://issues.apache.org/jira/browse/IMPALA-7863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Brown updated IMPALA-7863: -- Attachment: (was: 007.jpg) > To solve Gmail Server Error 007 > --- > > Key: IMPALA-7863 > URL: https://issues.apache.org/jira/browse/IMPALA-7863 > Project: IMPALA > Issue Type: Bug > Components: Catalog >Affects Versions: 2.2.0 >Reporter: smith_john >Priority: Major > Labels: GmailServerError007 > Fix For: Impala 3.2.0 > > > The Gmail Problem is quite common among Gmail users. All of the Gmail users > have faced this issue at least once. Like that one most common issue which is > faced by the users is [Gmail Server Error > 007|https://www.gmailtechnicalsupportnumbers.com/blog/gmail-server-error-007/] > In this issue an error message "oops... a server error occurred and your > email was not sent. (#007)" comes on the screen again and again. Various > methods like history and cookies cleaning, chrome cleaner kits, registry > checkups, and repairs checking extensions are not works for that situation. > one of the most popular methods is *Disable Email Signature from Avast.* > Follow the steps to solve this issue:- > In AVG, you can find this setting listed under Settings > General. Once > you’re there, deactivate the checkbox associated with Email Signature and hit > Save. > In Avast, go to Settings > General and disable the checkbox next to Enable > Avast email signature and hit Save. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-7863) To solve Gmail Server Error 007
[ https://issues.apache.org/jira/browse/IMPALA-7863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Brown updated IMPALA-7863: -- Description: The Gmail Problem is quite common among Gmail users. All of the Gmail users have faced this issue at least once. Like that one most common issue which is faced by the users is In this issue an error message "oops... a server error occurred and your email was not sent. (#007)" comes on the screen again and again. Various methods like history and cookies cleaning, chrome cleaner kits, registry checkups, and repairs checking extensions are not works for that situation. one of the most popular methods is *Disable Email Signature from Avast.* Follow the steps to solve this issue:- In AVG, you can find this setting listed under Settings > General. Once you’re there, deactivate the checkbox associated with Email Signature and hit Save. In Avast, go to Settings > General and disable the checkbox next to Enable Avast email signature and hit Save. was: The Gmail Problem is quite common among Gmail users. All of the Gmail users have faced this issue at least once. Like that one most common issue which is faced by the users is [Gmail Server Error 007|https://www.gmailtechnicalsupportnumbers.com/blog/gmail-server-error-007/] In this issue an error message "oops... a server error occurred and your email was not sent. (#007)" comes on the screen again and again. Various methods like history and cookies cleaning, chrome cleaner kits, registry checkups, and repairs checking extensions are not works for that situation. one of the most popular methods is *Disable Email Signature from Avast.* Follow the steps to solve this issue:- In AVG, you can find this setting listed under Settings > General. Once you’re there, deactivate the checkbox associated with Email Signature and hit Save. In Avast, go to Settings > General and disable the checkbox next to Enable Avast email signature and hit Save. > To solve Gmail Server Error 007 > --- > > Key: IMPALA-7863 > URL: https://issues.apache.org/jira/browse/IMPALA-7863 > Project: IMPALA > Issue Type: Bug > Components: Catalog >Affects Versions: 2.2.0 >Reporter: smith_john >Priority: Major > Labels: GmailServerError007 > Fix For: Impala 3.2.0 > > > The Gmail Problem is quite common among Gmail users. All of the Gmail users > have faced this issue at least once. Like that one most common issue which is > faced by the users is In this issue an error message "oops... a server error > occurred and your email was not sent. (#007)" comes on the screen again and > again. Various methods like history and cookies cleaning, chrome cleaner > kits, registry checkups, and repairs checking extensions are not works for > that situation. one of the most popular methods is *Disable Email Signature > from Avast.* Follow the steps to solve this issue:- > In AVG, you can find this setting listed under Settings > General. Once > you’re there, deactivate the checkbox associated with Email Signature and hit > Save. > In Avast, go to Settings > General and disable the checkbox next to Enable > Avast email signature and hit Save. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-7804) Various scanner tests intermittently failing on S3 on different runs
[
https://issues.apache.org/jira/browse/IMPALA-7804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Brown updated IMPALA-7804:
--
Target Version: Impala 3.2.0
> Various scanner tests intermittently failing on S3 on different runs
>
>
> Key: IMPALA-7804
> URL: https://issues.apache.org/jira/browse/IMPALA-7804
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 3.2.0
>Reporter: David Knupp
>Priority: Critical
> Labels: S3
>
> The failures have to do with getting AWS client credentials.
> *query_test/test_scanners.py:696: in test_decimal_encodings*
> _Stacktrace_
> {noformat}
> query_test/test_scanners.py:696: in test_decimal_encodings
> self.run_test_case('QueryTest/parquet-decimal-formats', vector,
> unique_database)
> common/impala_test_suite.py:496: in run_test_case
> self.__verify_results_and_errors(vector, test_section, result, use_db)
> common/impala_test_suite.py:358: in __verify_results_and_errors
> replace_filenames_with_placeholder)
> common/test_result_verifier.py:438: in verify_raw_results
> VERIFIER_MAP[verifier](expected, actual)
> common/test_result_verifier.py:260: in verify_query_result_is_equal
> assert expected_results == actual_results
> E assert Comparing QueryTestResults (expected vs actual):
> E -255.00,-255.00,-255.00 == -255.00,-255.00,-255.00
> E -255.00,-255.00,-255.00 != -65535.00,-65535.00,-65535.00
> E -65535.00,-65535.00,-65535.00 != -999.99,-999.99,-999.99
> E -65535.00,-65535.00,-65535.00 !=
> 0.00,-.99,-.99
> E -999.99,-999.99,-999.99 != 0.00,0.00,0.00
> E -999.99,-999.99,-999.99 !=
> 0.00,.99,.99
> E 0.00,-.99,-.99 !=
> 255.00,255.00,255.00
> E 0.00,-.99,-.99 !=
> 65535.00,65535.00,65535.00
> E 0.00,0.00,0.00 != 999.99,999.99,999.99
> E 0.00,0.00,0.00 != None
> E 0.00,.99,.99 != None
> E 0.00,.99,.99 != None
> E 255.00,255.00,255.00 != None
> E 255.00,255.00,255.00 != None
> E 65535.00,65535.00,65535.00 != None
> E 65535.00,65535.00,65535.00 != None
> E 999.99,999.99,999.99 != None
> E 999.99,999.99,999.99 != None
> E Number of rows returned (expected vs actual): 18 != 9
> {noformat}
> _Standard Error_
> {noformat}
> SET sync_ddl=False;
> -- executing against localhost:21000
> DROP DATABASE IF EXISTS `test_huge_num_rows_76a09ef1` CASCADE;
> -- 2018-11-01 09:42:41,140 INFO MainThread: Started query
> 4c4bc0e7b69d7641:130ffe73
> SET sync_ddl=False;
> -- executing against localhost:21000
> CREATE DATABASE `test_huge_num_rows_76a09ef1`;
> -- 2018-11-01 09:42:42,402 INFO MainThread: Started query
> e34d714d6a62cba1:2a8544d0
> -- 2018-11-01 09:42:42,405 INFO MainThread: Created database
> "test_huge_num_rows_76a09ef1" for test ID
> "query_test/test_scanners.py::TestParquet::()::test_huge_num_rows[protocol:
> beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0,
> 'disable_codegen_rows_threshold': 0, 'disable_codegen': True,
> 'abort_on_error': 1, 'debug_action':
> '-1:OPEN:SET_DENY_RESERVATION_PROBABILITY@1.0',
> 'exec_single_node_rows_threshold': 0} | table_format: parquet/none]"
> 18/11/01 09:42:43 DEBUG s3a.S3AFileSystem: Initializing S3AFileSystem for
> impala-test-uswest2-1
> 18/11/01 09:42:43 DEBUG s3a.S3AUtils: Propagating entries under
> fs.s3a.bucket.impala-test-uswest2-1.
> 18/11/01 09:42:43 WARN impl.MetricsConfig: Cannot locate configuration: tried
> hadoop-metrics2-s3a-file-system.properties,hadoop-metrics2.properties
> 18/11/01 09:42:43 INFO impl.MetricsSystemImpl: Scheduled Metric snapshot
> period at 10 second(s).
> 18/11/01 09:42:43 INFO impl.MetricsSystemImpl: s3a-file-system metrics system
> started
> 18/11/01 09:42:43 DEBUG s3a.S3AUtils: For URI s3a://impala-test-uswest2-1/,
> using credentials AWSCredentialProviderList: BasicAWSCredentialsProvider
> EnvironmentVariableCredentialsProvider
> com.amazonaws.auth.InstanceProfileCredentialsProvider@15bbf42f
> 18/11/01 09:42:43 DEBUG s3a.S3AUtils: Value of fs.s3a.connection.maximum is
> 1500
> 18/11/01 09:42:43 DEBUG s3a.S3AUtils: Value of fs.s3a.attempts.maximum is 20
> 18/11/01 09:42:43 DEBUG s3a.S3AUtils: Value of
> fs.s3a.connection.establish.timeout is 5000
> 18/11/01 09:42:43 DEBUG s3a.S3AUtils: Value of fs.s3a.connection.timeout is
> 20
> 18/11/01 09:42:43 DEBUG s3a.S3AUtils: Value of
[jira] [Updated] (IMPALA-7804) Various scanner tests intermittently failing on S3 on different runs
[
https://issues.apache.org/jira/browse/IMPALA-7804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Brown updated IMPALA-7804:
--
Affects Version/s: (was: Impala 3.1.0)
Impala 3.2.0
> Various scanner tests intermittently failing on S3 on different runs
>
>
> Key: IMPALA-7804
> URL: https://issues.apache.org/jira/browse/IMPALA-7804
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 3.2.0
>Reporter: David Knupp
>Priority: Critical
> Labels: S3
>
> The failures have to do with getting AWS client credentials.
> *query_test/test_scanners.py:696: in test_decimal_encodings*
> _Stacktrace_
> {noformat}
> query_test/test_scanners.py:696: in test_decimal_encodings
> self.run_test_case('QueryTest/parquet-decimal-formats', vector,
> unique_database)
> common/impala_test_suite.py:496: in run_test_case
> self.__verify_results_and_errors(vector, test_section, result, use_db)
> common/impala_test_suite.py:358: in __verify_results_and_errors
> replace_filenames_with_placeholder)
> common/test_result_verifier.py:438: in verify_raw_results
> VERIFIER_MAP[verifier](expected, actual)
> common/test_result_verifier.py:260: in verify_query_result_is_equal
> assert expected_results == actual_results
> E assert Comparing QueryTestResults (expected vs actual):
> E -255.00,-255.00,-255.00 == -255.00,-255.00,-255.00
> E -255.00,-255.00,-255.00 != -65535.00,-65535.00,-65535.00
> E -65535.00,-65535.00,-65535.00 != -999.99,-999.99,-999.99
> E -65535.00,-65535.00,-65535.00 !=
> 0.00,-.99,-.99
> E -999.99,-999.99,-999.99 != 0.00,0.00,0.00
> E -999.99,-999.99,-999.99 !=
> 0.00,.99,.99
> E 0.00,-.99,-.99 !=
> 255.00,255.00,255.00
> E 0.00,-.99,-.99 !=
> 65535.00,65535.00,65535.00
> E 0.00,0.00,0.00 != 999.99,999.99,999.99
> E 0.00,0.00,0.00 != None
> E 0.00,.99,.99 != None
> E 0.00,.99,.99 != None
> E 255.00,255.00,255.00 != None
> E 255.00,255.00,255.00 != None
> E 65535.00,65535.00,65535.00 != None
> E 65535.00,65535.00,65535.00 != None
> E 999.99,999.99,999.99 != None
> E 999.99,999.99,999.99 != None
> E Number of rows returned (expected vs actual): 18 != 9
> {noformat}
> _Standard Error_
> {noformat}
> SET sync_ddl=False;
> -- executing against localhost:21000
> DROP DATABASE IF EXISTS `test_huge_num_rows_76a09ef1` CASCADE;
> -- 2018-11-01 09:42:41,140 INFO MainThread: Started query
> 4c4bc0e7b69d7641:130ffe73
> SET sync_ddl=False;
> -- executing against localhost:21000
> CREATE DATABASE `test_huge_num_rows_76a09ef1`;
> -- 2018-11-01 09:42:42,402 INFO MainThread: Started query
> e34d714d6a62cba1:2a8544d0
> -- 2018-11-01 09:42:42,405 INFO MainThread: Created database
> "test_huge_num_rows_76a09ef1" for test ID
> "query_test/test_scanners.py::TestParquet::()::test_huge_num_rows[protocol:
> beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0,
> 'disable_codegen_rows_threshold': 0, 'disable_codegen': True,
> 'abort_on_error': 1, 'debug_action':
> '-1:OPEN:SET_DENY_RESERVATION_PROBABILITY@1.0',
> 'exec_single_node_rows_threshold': 0} | table_format: parquet/none]"
> 18/11/01 09:42:43 DEBUG s3a.S3AFileSystem: Initializing S3AFileSystem for
> impala-test-uswest2-1
> 18/11/01 09:42:43 DEBUG s3a.S3AUtils: Propagating entries under
> fs.s3a.bucket.impala-test-uswest2-1.
> 18/11/01 09:42:43 WARN impl.MetricsConfig: Cannot locate configuration: tried
> hadoop-metrics2-s3a-file-system.properties,hadoop-metrics2.properties
> 18/11/01 09:42:43 INFO impl.MetricsSystemImpl: Scheduled Metric snapshot
> period at 10 second(s).
> 18/11/01 09:42:43 INFO impl.MetricsSystemImpl: s3a-file-system metrics system
> started
> 18/11/01 09:42:43 DEBUG s3a.S3AUtils: For URI s3a://impala-test-uswest2-1/,
> using credentials AWSCredentialProviderList: BasicAWSCredentialsProvider
> EnvironmentVariableCredentialsProvider
> com.amazonaws.auth.InstanceProfileCredentialsProvider@15bbf42f
> 18/11/01 09:42:43 DEBUG s3a.S3AUtils: Value of fs.s3a.connection.maximum is
> 1500
> 18/11/01 09:42:43 DEBUG s3a.S3AUtils: Value of fs.s3a.attempts.maximum is 20
> 18/11/01 09:42:43 DEBUG s3a.S3AUtils: Value of
> fs.s3a.connection.establish.timeout is 5000
> 18/11/01 09:42:43 DEBUG s3a.S3AUtils: Value of fs.s3a.connection.timeout is
> 20
> 18/11/01
[jira] [Updated] (IMPALA-7804) Various scanner tests intermittently failing on S3 on different runs
[
https://issues.apache.org/jira/browse/IMPALA-7804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Brown updated IMPALA-7804:
--
Priority: Blocker (was: Critical)
> Various scanner tests intermittently failing on S3 on different runs
>
>
> Key: IMPALA-7804
> URL: https://issues.apache.org/jira/browse/IMPALA-7804
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 3.2.0
>Reporter: David Knupp
>Priority: Blocker
> Labels: S3
>
> The failures have to do with getting AWS client credentials.
> *query_test/test_scanners.py:696: in test_decimal_encodings*
> _Stacktrace_
> {noformat}
> query_test/test_scanners.py:696: in test_decimal_encodings
> self.run_test_case('QueryTest/parquet-decimal-formats', vector,
> unique_database)
> common/impala_test_suite.py:496: in run_test_case
> self.__verify_results_and_errors(vector, test_section, result, use_db)
> common/impala_test_suite.py:358: in __verify_results_and_errors
> replace_filenames_with_placeholder)
> common/test_result_verifier.py:438: in verify_raw_results
> VERIFIER_MAP[verifier](expected, actual)
> common/test_result_verifier.py:260: in verify_query_result_is_equal
> assert expected_results == actual_results
> E assert Comparing QueryTestResults (expected vs actual):
> E -255.00,-255.00,-255.00 == -255.00,-255.00,-255.00
> E -255.00,-255.00,-255.00 != -65535.00,-65535.00,-65535.00
> E -65535.00,-65535.00,-65535.00 != -999.99,-999.99,-999.99
> E -65535.00,-65535.00,-65535.00 !=
> 0.00,-.99,-.99
> E -999.99,-999.99,-999.99 != 0.00,0.00,0.00
> E -999.99,-999.99,-999.99 !=
> 0.00,.99,.99
> E 0.00,-.99,-.99 !=
> 255.00,255.00,255.00
> E 0.00,-.99,-.99 !=
> 65535.00,65535.00,65535.00
> E 0.00,0.00,0.00 != 999.99,999.99,999.99
> E 0.00,0.00,0.00 != None
> E 0.00,.99,.99 != None
> E 0.00,.99,.99 != None
> E 255.00,255.00,255.00 != None
> E 255.00,255.00,255.00 != None
> E 65535.00,65535.00,65535.00 != None
> E 65535.00,65535.00,65535.00 != None
> E 999.99,999.99,999.99 != None
> E 999.99,999.99,999.99 != None
> E Number of rows returned (expected vs actual): 18 != 9
> {noformat}
> _Standard Error_
> {noformat}
> SET sync_ddl=False;
> -- executing against localhost:21000
> DROP DATABASE IF EXISTS `test_huge_num_rows_76a09ef1` CASCADE;
> -- 2018-11-01 09:42:41,140 INFO MainThread: Started query
> 4c4bc0e7b69d7641:130ffe73
> SET sync_ddl=False;
> -- executing against localhost:21000
> CREATE DATABASE `test_huge_num_rows_76a09ef1`;
> -- 2018-11-01 09:42:42,402 INFO MainThread: Started query
> e34d714d6a62cba1:2a8544d0
> -- 2018-11-01 09:42:42,405 INFO MainThread: Created database
> "test_huge_num_rows_76a09ef1" for test ID
> "query_test/test_scanners.py::TestParquet::()::test_huge_num_rows[protocol:
> beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0,
> 'disable_codegen_rows_threshold': 0, 'disable_codegen': True,
> 'abort_on_error': 1, 'debug_action':
> '-1:OPEN:SET_DENY_RESERVATION_PROBABILITY@1.0',
> 'exec_single_node_rows_threshold': 0} | table_format: parquet/none]"
> 18/11/01 09:42:43 DEBUG s3a.S3AFileSystem: Initializing S3AFileSystem for
> impala-test-uswest2-1
> 18/11/01 09:42:43 DEBUG s3a.S3AUtils: Propagating entries under
> fs.s3a.bucket.impala-test-uswest2-1.
> 18/11/01 09:42:43 WARN impl.MetricsConfig: Cannot locate configuration: tried
> hadoop-metrics2-s3a-file-system.properties,hadoop-metrics2.properties
> 18/11/01 09:42:43 INFO impl.MetricsSystemImpl: Scheduled Metric snapshot
> period at 10 second(s).
> 18/11/01 09:42:43 INFO impl.MetricsSystemImpl: s3a-file-system metrics system
> started
> 18/11/01 09:42:43 DEBUG s3a.S3AUtils: For URI s3a://impala-test-uswest2-1/,
> using credentials AWSCredentialProviderList: BasicAWSCredentialsProvider
> EnvironmentVariableCredentialsProvider
> com.amazonaws.auth.InstanceProfileCredentialsProvider@15bbf42f
> 18/11/01 09:42:43 DEBUG s3a.S3AUtils: Value of fs.s3a.connection.maximum is
> 1500
> 18/11/01 09:42:43 DEBUG s3a.S3AUtils: Value of fs.s3a.attempts.maximum is 20
> 18/11/01 09:42:43 DEBUG s3a.S3AUtils: Value of
> fs.s3a.connection.establish.timeout is 5000
> 18/11/01 09:42:43 DEBUG s3a.S3AUtils: Value of fs.s3a.connection.timeout is
> 20
> 18/11/01 09:42:43 DEBUG s3a.S3AUtils: Value of
[jira] [Updated] (IMPALA-7837) SCAN_BYTES_LIMIT="100M" test failing to raise exception in release build
[
https://issues.apache.org/jira/browse/IMPALA-7837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Brown updated IMPALA-7837:
--
Target Version: Impala 3.2.0 (was: Impala 3.1.0)
> SCAN_BYTES_LIMIT="100M" test failing to raise exception in release build
>
>
> Key: IMPALA-7837
> URL: https://issues.apache.org/jira/browse/IMPALA-7837
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 3.2.0
>Reporter: Michael Brown
>Assignee: Bikramjeet Vig
>Priority: Blocker
> Attachments: impala-logs.tar.gz
>
>
> This test is not raising the expected exception on a *release build*:
> {noformat}
> QUERY
> # Query should fail due to exceeding scan bytes limit.
> set SCAN_BYTES_LIMIT="100M";
> select count(*) from tpch.lineitem l1,tpch.lineitem l2, tpch.lineitem l3 where
> l1.l_suppkey = l2.l_linenumber and l1.l_orderkey = l2.l_orderkey and
> l1.l_orderkey = l3.l_orderkey group by l1.l_comment, l2.l_comment
> having count(*) = 99
> CATCH
> row_regex:.*terminated due to scan bytes limit of 100.00 M.
> {noformat}
> {noformat}
> Stacktrace
> query_test/test_resource_limits.py:46: in test_resource_limits
> self.run_test_case('QueryTest/query-resource-limits', vector)
> common/impala_test_suite.py:482: in run_test_case
> assert False, "Expected exception: %s" % expected_str
> E AssertionError: Expected exception: row_regex:.*terminated due to scan
> bytes limit of 100.00 M.*
> {noformat}
> It fails deterministically in CI (3 times in a row). I can't find a query
> profile matching the query ID for some reason, but I've attached logs.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-7837) SCAN_BYTES_LIMIT="100M" test failing to raise exception in release build
[
https://issues.apache.org/jira/browse/IMPALA-7837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Brown updated IMPALA-7837:
--
Affects Version/s: (was: Impala 3.1.0)
Impala 3.2.0
> SCAN_BYTES_LIMIT="100M" test failing to raise exception in release build
>
>
> Key: IMPALA-7837
> URL: https://issues.apache.org/jira/browse/IMPALA-7837
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 3.2.0
>Reporter: Michael Brown
>Assignee: Bikramjeet Vig
>Priority: Blocker
> Attachments: impala-logs.tar.gz
>
>
> This test is not raising the expected exception on a *release build*:
> {noformat}
> QUERY
> # Query should fail due to exceeding scan bytes limit.
> set SCAN_BYTES_LIMIT="100M";
> select count(*) from tpch.lineitem l1,tpch.lineitem l2, tpch.lineitem l3 where
> l1.l_suppkey = l2.l_linenumber and l1.l_orderkey = l2.l_orderkey and
> l1.l_orderkey = l3.l_orderkey group by l1.l_comment, l2.l_comment
> having count(*) = 99
> CATCH
> row_regex:.*terminated due to scan bytes limit of 100.00 M.
> {noformat}
> {noformat}
> Stacktrace
> query_test/test_resource_limits.py:46: in test_resource_limits
> self.run_test_case('QueryTest/query-resource-limits', vector)
> common/impala_test_suite.py:482: in run_test_case
> assert False, "Expected exception: %s" % expected_str
> E AssertionError: Expected exception: row_regex:.*terminated due to scan
> bytes limit of 100.00 M.*
> {noformat}
> It fails deterministically in CI (3 times in a row). I can't find a query
> profile matching the query ID for some reason, but I've attached logs.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-7837) SCAN_BYTES_LIMIT="100M" test failing to raise exception in release build
Michael Brown created IMPALA-7837:
-
Summary: SCAN_BYTES_LIMIT="100M" test failing to raise exception
in release build
Key: IMPALA-7837
URL: https://issues.apache.org/jira/browse/IMPALA-7837
Project: IMPALA
Issue Type: Bug
Components: Backend
Affects Versions: Impala 3.1.0
Reporter: Michael Brown
Assignee: Bikramjeet Vig
Attachments: impala-logs.tar.gz
This test is not raising the expected exception on a *release build*:
{noformat}
QUERY
# Query should fail due to exceeding scan bytes limit.
set SCAN_BYTES_LIMIT="100M";
select count(*) from tpch.lineitem l1,tpch.lineitem l2, tpch.lineitem l3 where
l1.l_suppkey = l2.l_linenumber and l1.l_orderkey = l2.l_orderkey and
l1.l_orderkey = l3.l_orderkey group by l1.l_comment, l2.l_comment
having count(*) = 99
CATCH
row_regex:.*terminated due to scan bytes limit of 100.00 M.
{noformat}
{noformat}
Stacktrace
query_test/test_resource_limits.py:46: in test_resource_limits
self.run_test_case('QueryTest/query-resource-limits', vector)
common/impala_test_suite.py:482: in run_test_case
assert False, "Expected exception: %s" % expected_str
E AssertionError: Expected exception: row_regex:.*terminated due to scan
bytes limit of 100.00 M.*
{noformat}
It fails deterministically in CI (3 times in a row). I can't find a query
profile matching the query ID for some reason, but I've attached logs.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (IMPALA-7837) SCAN_BYTES_LIMIT="100M" test failing to raise exception in release build
Michael Brown created IMPALA-7837:
-
Summary: SCAN_BYTES_LIMIT="100M" test failing to raise exception
in release build
Key: IMPALA-7837
URL: https://issues.apache.org/jira/browse/IMPALA-7837
Project: IMPALA
Issue Type: Bug
Components: Backend
Affects Versions: Impala 3.1.0
Reporter: Michael Brown
Assignee: Bikramjeet Vig
Attachments: impala-logs.tar.gz
This test is not raising the expected exception on a *release build*:
{noformat}
QUERY
# Query should fail due to exceeding scan bytes limit.
set SCAN_BYTES_LIMIT="100M";
select count(*) from tpch.lineitem l1,tpch.lineitem l2, tpch.lineitem l3 where
l1.l_suppkey = l2.l_linenumber and l1.l_orderkey = l2.l_orderkey and
l1.l_orderkey = l3.l_orderkey group by l1.l_comment, l2.l_comment
having count(*) = 99
CATCH
row_regex:.*terminated due to scan bytes limit of 100.00 M.
{noformat}
{noformat}
Stacktrace
query_test/test_resource_limits.py:46: in test_resource_limits
self.run_test_case('QueryTest/query-resource-limits', vector)
common/impala_test_suite.py:482: in run_test_case
assert False, "Expected exception: %s" % expected_str
E AssertionError: Expected exception: row_regex:.*terminated due to scan
bytes limit of 100.00 M.*
{noformat}
It fails deterministically in CI (3 times in a row). I can't find a query
profile matching the query ID for some reason, but I've attached logs.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-7837) SCAN_BYTES_LIMIT="100M" test failing to raise exception in release build
[
https://issues.apache.org/jira/browse/IMPALA-7837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16680092#comment-16680092
]
Michael Brown commented on IMPALA-7837:
---
[~bikramjeet.vig] Gave this to you to look at, please reassign if necessary.
> SCAN_BYTES_LIMIT="100M" test failing to raise exception in release build
>
>
> Key: IMPALA-7837
> URL: https://issues.apache.org/jira/browse/IMPALA-7837
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 3.1.0
>Reporter: Michael Brown
>Assignee: Bikramjeet Vig
>Priority: Blocker
> Attachments: impala-logs.tar.gz
>
>
> This test is not raising the expected exception on a *release build*:
> {noformat}
> QUERY
> # Query should fail due to exceeding scan bytes limit.
> set SCAN_BYTES_LIMIT="100M";
> select count(*) from tpch.lineitem l1,tpch.lineitem l2, tpch.lineitem l3 where
> l1.l_suppkey = l2.l_linenumber and l1.l_orderkey = l2.l_orderkey and
> l1.l_orderkey = l3.l_orderkey group by l1.l_comment, l2.l_comment
> having count(*) = 99
> CATCH
> row_regex:.*terminated due to scan bytes limit of 100.00 M.
> {noformat}
> {noformat}
> Stacktrace
> query_test/test_resource_limits.py:46: in test_resource_limits
> self.run_test_case('QueryTest/query-resource-limits', vector)
> common/impala_test_suite.py:482: in run_test_case
> assert False, "Expected exception: %s" % expected_str
> E AssertionError: Expected exception: row_regex:.*terminated due to scan
> bytes limit of 100.00 M.*
> {noformat}
> It fails deterministically in CI (3 times in a row). I can't find a query
> profile matching the query ID for some reason, but I've attached logs.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-7809) test_concurrent_schema_change incompatible with Kudu 1.9
[
https://issues.apache.org/jira/browse/IMPALA-7809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Brown resolved IMPALA-7809.
---
Resolution: Fixed
Fix Version/s: Impala 3.2.0
> test_concurrent_schema_change incompatible with Kudu 1.9
>
>
> Key: IMPALA-7809
> URL: https://issues.apache.org/jira/browse/IMPALA-7809
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
>Affects Versions: Impala 3.2.0
>Reporter: Michael Brown
>Assignee: Michael Brown
>Priority: Blocker
> Labels: broken-build
> Fix For: Impala 3.2.0
>
>
> With Kudu 1.9, test_concurrent_schema_change incompatible fails because the
> message format is slightly different.
> {noformat}
> query_test/test_kudu.py:442: in test_concurrent_schema_change assert "has
> fewer columns (1) than the SELECT / VALUES clause returns (2)" in msg \ E
> assert ('has fewer columns (1) than the SELECT / VALUES clause returns (2)'
> in 'ImpalaBeeswaxException:\n Query aborted:Kudu error(s) reported, first
> error: Invalid argument: Failed to write batch ...redacted.com:31201): Client
> provided column col1 INT64 NULLABLE not present in tablet (1 of 2
> similar)\n\n' or "(type: TINYINT) is not compatible with column 'col1' (type:
> STRING)" in 'ImpalaBeeswaxException:\n Query aborted:Kudu error(s) reported,
> first error: Invalid argument: Failed to write batch ...redacted.com:31201):
> Client provided column col1 INT64 NULLABLE not present in tablet (1 of 2
> similar)\n\n' or 'has fewer columns than expected.' in
> 'ImpalaBeeswaxException:\n Query aborted:Kudu error(s) reported, first error:
> Invalid argument: Failed to write batch ...redacted.com:31201): Client
> provided column col1 INT64 NULLABLE not present in tablet (1 of 2
> similar)\n\n' or 'Column col1 has unexpected type.' in
> 'ImpalaBeeswaxException:\n Query aborted:Kudu error(s) reported, first error:
> Invalid argument: Failed to write batch ...redacted.com:31201): Client
> provided column col1 INT64 NULLABLE not present in tablet (1 of 2
> similar)\n\n' or 'Client provided column col1[int64 NULLABLE] not present in
> tablet' in 'ImpalaBeeswaxException:\n Query aborted:Kudu error(s) reported,
> first error: Invalid argument: Failed to write batch ...redacted.com:31201):
> Client provided column col1 INT64 NULLABLE not present in tablet (1 of 2
> similar)\n\n')
> Stacktrace
> query_test/test_kudu.py:442: in test_concurrent_schema_change
> assert "has fewer columns (1) than the SELECT / VALUES clause returns
> (2)" in msg \
> E assert ('has fewer columns (1) than the SELECT / VALUES clause returns
> (2)' in 'ImpalaBeeswaxException:\n Query aborted:Kudu error(s) reported,
> first error: Invalid argument: Failed to write batch ...redacted.com:31201):
> Client provided column col1 INT64 NULLABLE not present in tablet (1 of 2
> similar)\n\n' or "(type: TINYINT) is not compatible with column 'col1' (type:
> STRING)" in 'ImpalaBeeswaxException:\n Query aborted:Kudu error(s) reported,
> first error: Invalid argument: Failed to write batch ...redacted.com:31201):
> Client provided column col1 INT64 NULLABLE not present in tablet (1 of 2
> similar)\n\n' or 'has fewer columns than expected.' in
> 'ImpalaBeeswaxException:\n Query aborted:Kudu error(s) reported, first error:
> Invalid argument: Failed to write batch ...redacted.com:31201): Client
> provided column col1 INT64 NULLABLE not present in tablet (1 of 2
> similar)\n\n' or 'Column col1 has unexpected type.' in
> 'ImpalaBeeswaxException:\n Query aborted:Kudu error(s) reported, first error:
> Invalid argument: Failed to write batch ...redacted.com:31201): Client
> provided column col1 INT64 NULLABLE not present in tablet (1 of 2
> similar)\n\n' or 'Client provided column col1[int64 NULLABLE] not present in
> tablet' in 'ImpalaBeeswaxException:\n Query aborted:Kudu error(s) reported,
> first error: Invalid argument: Failed to write batch ...redacted.com:31201):
> Client provided column col1 INT64 NULLABLE not present in tablet (1 of 2
> similar)\n\n')
> {noformat}
> We see that the message returned includes {{Client provided column col1 INT64
> NULLABLE not present in tablet}} and the code is looking for:
> {noformat}
> 437 for error in insert_thread.errors:
> 438 msg = str(error)
> 439 # The first two are AnalysisExceptions, the next two come from
> KuduTableSink::Open()
> 440 # if the schema has changed since analysis, the last comes from
> the Kudu server if
> 441 # the schema changes between KuduTableSink::Open() and when the
> write ops are sent.
> 442 assert "has fewer columns (1) than the SELECT / VALUES clause
> returns (2)" in msg \
> 443 or "(type: TINYINT) is not
[jira] [Resolved] (IMPALA-7809) test_concurrent_schema_change incompatible with Kudu 1.9
[
https://issues.apache.org/jira/browse/IMPALA-7809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Brown resolved IMPALA-7809.
---
Resolution: Fixed
Fix Version/s: Impala 3.2.0
> test_concurrent_schema_change incompatible with Kudu 1.9
>
>
> Key: IMPALA-7809
> URL: https://issues.apache.org/jira/browse/IMPALA-7809
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
>Affects Versions: Impala 3.2.0
>Reporter: Michael Brown
>Assignee: Michael Brown
>Priority: Blocker
> Labels: broken-build
> Fix For: Impala 3.2.0
>
>
> With Kudu 1.9, test_concurrent_schema_change incompatible fails because the
> message format is slightly different.
> {noformat}
> query_test/test_kudu.py:442: in test_concurrent_schema_change assert "has
> fewer columns (1) than the SELECT / VALUES clause returns (2)" in msg \ E
> assert ('has fewer columns (1) than the SELECT / VALUES clause returns (2)'
> in 'ImpalaBeeswaxException:\n Query aborted:Kudu error(s) reported, first
> error: Invalid argument: Failed to write batch ...redacted.com:31201): Client
> provided column col1 INT64 NULLABLE not present in tablet (1 of 2
> similar)\n\n' or "(type: TINYINT) is not compatible with column 'col1' (type:
> STRING)" in 'ImpalaBeeswaxException:\n Query aborted:Kudu error(s) reported,
> first error: Invalid argument: Failed to write batch ...redacted.com:31201):
> Client provided column col1 INT64 NULLABLE not present in tablet (1 of 2
> similar)\n\n' or 'has fewer columns than expected.' in
> 'ImpalaBeeswaxException:\n Query aborted:Kudu error(s) reported, first error:
> Invalid argument: Failed to write batch ...redacted.com:31201): Client
> provided column col1 INT64 NULLABLE not present in tablet (1 of 2
> similar)\n\n' or 'Column col1 has unexpected type.' in
> 'ImpalaBeeswaxException:\n Query aborted:Kudu error(s) reported, first error:
> Invalid argument: Failed to write batch ...redacted.com:31201): Client
> provided column col1 INT64 NULLABLE not present in tablet (1 of 2
> similar)\n\n' or 'Client provided column col1[int64 NULLABLE] not present in
> tablet' in 'ImpalaBeeswaxException:\n Query aborted:Kudu error(s) reported,
> first error: Invalid argument: Failed to write batch ...redacted.com:31201):
> Client provided column col1 INT64 NULLABLE not present in tablet (1 of 2
> similar)\n\n')
> Stacktrace
> query_test/test_kudu.py:442: in test_concurrent_schema_change
> assert "has fewer columns (1) than the SELECT / VALUES clause returns
> (2)" in msg \
> E assert ('has fewer columns (1) than the SELECT / VALUES clause returns
> (2)' in 'ImpalaBeeswaxException:\n Query aborted:Kudu error(s) reported,
> first error: Invalid argument: Failed to write batch ...redacted.com:31201):
> Client provided column col1 INT64 NULLABLE not present in tablet (1 of 2
> similar)\n\n' or "(type: TINYINT) is not compatible with column 'col1' (type:
> STRING)" in 'ImpalaBeeswaxException:\n Query aborted:Kudu error(s) reported,
> first error: Invalid argument: Failed to write batch ...redacted.com:31201):
> Client provided column col1 INT64 NULLABLE not present in tablet (1 of 2
> similar)\n\n' or 'has fewer columns than expected.' in
> 'ImpalaBeeswaxException:\n Query aborted:Kudu error(s) reported, first error:
> Invalid argument: Failed to write batch ...redacted.com:31201): Client
> provided column col1 INT64 NULLABLE not present in tablet (1 of 2
> similar)\n\n' or 'Column col1 has unexpected type.' in
> 'ImpalaBeeswaxException:\n Query aborted:Kudu error(s) reported, first error:
> Invalid argument: Failed to write batch ...redacted.com:31201): Client
> provided column col1 INT64 NULLABLE not present in tablet (1 of 2
> similar)\n\n' or 'Client provided column col1[int64 NULLABLE] not present in
> tablet' in 'ImpalaBeeswaxException:\n Query aborted:Kudu error(s) reported,
> first error: Invalid argument: Failed to write batch ...redacted.com:31201):
> Client provided column col1 INT64 NULLABLE not present in tablet (1 of 2
> similar)\n\n')
> {noformat}
> We see that the message returned includes {{Client provided column col1 INT64
> NULLABLE not present in tablet}} and the code is looking for:
> {noformat}
> 437 for error in insert_thread.errors:
> 438 msg = str(error)
> 439 # The first two are AnalysisExceptions, the next two come from
> KuduTableSink::Open()
> 440 # if the schema has changed since analysis, the last comes from
> the Kudu server if
> 441 # the schema changes between KuduTableSink::Open() and when the
> write ops are sent.
> 442 assert "has fewer columns (1) than the SELECT / VALUES clause
> returns (2)" in msg \
> 443 or "(type: TINYINT) is not
[jira] [Work started] (IMPALA-7809) test_concurrent_schema_change incompatible with Kudu 1.9
[
https://issues.apache.org/jira/browse/IMPALA-7809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on IMPALA-7809 started by Michael Brown.
-
> test_concurrent_schema_change incompatible with Kudu 1.9
>
>
> Key: IMPALA-7809
> URL: https://issues.apache.org/jira/browse/IMPALA-7809
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
>Affects Versions: Impala 3.2.0
>Reporter: Michael Brown
>Assignee: Michael Brown
>Priority: Blocker
> Labels: broken-build
>
> With Kudu 1.9, test_concurrent_schema_change incompatible fails because the
> message format is slightly different.
> {noformat}
> query_test/test_kudu.py:442: in test_concurrent_schema_change assert "has
> fewer columns (1) than the SELECT / VALUES clause returns (2)" in msg \ E
> assert ('has fewer columns (1) than the SELECT / VALUES clause returns (2)'
> in 'ImpalaBeeswaxException:\n Query aborted:Kudu error(s) reported, first
> error: Invalid argument: Failed to write batch ...redacted.com:31201): Client
> provided column col1 INT64 NULLABLE not present in tablet (1 of 2
> similar)\n\n' or "(type: TINYINT) is not compatible with column 'col1' (type:
> STRING)" in 'ImpalaBeeswaxException:\n Query aborted:Kudu error(s) reported,
> first error: Invalid argument: Failed to write batch ...redacted.com:31201):
> Client provided column col1 INT64 NULLABLE not present in tablet (1 of 2
> similar)\n\n' or 'has fewer columns than expected.' in
> 'ImpalaBeeswaxException:\n Query aborted:Kudu error(s) reported, first error:
> Invalid argument: Failed to write batch ...redacted.com:31201): Client
> provided column col1 INT64 NULLABLE not present in tablet (1 of 2
> similar)\n\n' or 'Column col1 has unexpected type.' in
> 'ImpalaBeeswaxException:\n Query aborted:Kudu error(s) reported, first error:
> Invalid argument: Failed to write batch ...redacted.com:31201): Client
> provided column col1 INT64 NULLABLE not present in tablet (1 of 2
> similar)\n\n' or 'Client provided column col1[int64 NULLABLE] not present in
> tablet' in 'ImpalaBeeswaxException:\n Query aborted:Kudu error(s) reported,
> first error: Invalid argument: Failed to write batch ...redacted.com:31201):
> Client provided column col1 INT64 NULLABLE not present in tablet (1 of 2
> similar)\n\n')
> Stacktrace
> query_test/test_kudu.py:442: in test_concurrent_schema_change
> assert "has fewer columns (1) than the SELECT / VALUES clause returns
> (2)" in msg \
> E assert ('has fewer columns (1) than the SELECT / VALUES clause returns
> (2)' in 'ImpalaBeeswaxException:\n Query aborted:Kudu error(s) reported,
> first error: Invalid argument: Failed to write batch ...redacted.com:31201):
> Client provided column col1 INT64 NULLABLE not present in tablet (1 of 2
> similar)\n\n' or "(type: TINYINT) is not compatible with column 'col1' (type:
> STRING)" in 'ImpalaBeeswaxException:\n Query aborted:Kudu error(s) reported,
> first error: Invalid argument: Failed to write batch ...redacted.com:31201):
> Client provided column col1 INT64 NULLABLE not present in tablet (1 of 2
> similar)\n\n' or 'has fewer columns than expected.' in
> 'ImpalaBeeswaxException:\n Query aborted:Kudu error(s) reported, first error:
> Invalid argument: Failed to write batch ...redacted.com:31201): Client
> provided column col1 INT64 NULLABLE not present in tablet (1 of 2
> similar)\n\n' or 'Column col1 has unexpected type.' in
> 'ImpalaBeeswaxException:\n Query aborted:Kudu error(s) reported, first error:
> Invalid argument: Failed to write batch ...redacted.com:31201): Client
> provided column col1 INT64 NULLABLE not present in tablet (1 of 2
> similar)\n\n' or 'Client provided column col1[int64 NULLABLE] not present in
> tablet' in 'ImpalaBeeswaxException:\n Query aborted:Kudu error(s) reported,
> first error: Invalid argument: Failed to write batch ...redacted.com:31201):
> Client provided column col1 INT64 NULLABLE not present in tablet (1 of 2
> similar)\n\n')
> {noformat}
> We see that the message returned includes {{Client provided column col1 INT64
> NULLABLE not present in tablet}} and the code is looking for:
> {noformat}
> 437 for error in insert_thread.errors:
> 438 msg = str(error)
> 439 # The first two are AnalysisExceptions, the next two come from
> KuduTableSink::Open()
> 440 # if the schema has changed since analysis, the last comes from
> the Kudu server if
> 441 # the schema changes between KuduTableSink::Open() and when the
> write ops are sent.
> 442 assert "has fewer columns (1) than the SELECT / VALUES clause
> returns (2)" in msg \
> 443 or "(type: TINYINT) is not compatible with column 'col1' (type:
> STRING)" in msg \
> 444 or "has
[jira] [Created] (IMPALA-7809) test_concurrent_schema_change incompatible with Kudu 1.9
Michael Brown created IMPALA-7809:
-
Summary: test_concurrent_schema_change incompatible with Kudu 1.9
Key: IMPALA-7809
URL: https://issues.apache.org/jira/browse/IMPALA-7809
Project: IMPALA
Issue Type: Bug
Components: Infrastructure
Affects Versions: Impala 3.2.0
Reporter: Michael Brown
Assignee: Michael Brown
With Kudu 1.9, test_concurrent_schema_change incompatible fails because the
message format is slightly different.
{noformat}
query_test/test_kudu.py:442: in test_concurrent_schema_change assert "has
fewer columns (1) than the SELECT / VALUES clause returns (2)" in msg \ E
assert ('has fewer columns (1) than the SELECT / VALUES clause returns (2)' in
'ImpalaBeeswaxException:\n Query aborted:Kudu error(s) reported, first error:
Invalid argument: Failed to write batch ...redacted.com:31201): Client provided
column col1 INT64 NULLABLE not present in tablet (1 of 2 similar)\n\n' or
"(type: TINYINT) is not compatible with column 'col1' (type: STRING)" in
'ImpalaBeeswaxException:\n Query aborted:Kudu error(s) reported, first error:
Invalid argument: Failed to write batch ...redacted.com:31201): Client provided
column col1 INT64 NULLABLE not present in tablet (1 of 2 similar)\n\n' or 'has
fewer columns than expected.' in 'ImpalaBeeswaxException:\n Query aborted:Kudu
error(s) reported, first error: Invalid argument: Failed to write batch
...redacted.com:31201): Client provided column col1 INT64 NULLABLE not present
in tablet (1 of 2 similar)\n\n' or 'Column col1 has unexpected type.' in
'ImpalaBeeswaxException:\n Query aborted:Kudu error(s) reported, first error:
Invalid argument: Failed to write batch ...redacted.com:31201): Client provided
column col1 INT64 NULLABLE not present in tablet (1 of 2 similar)\n\n' or
'Client provided column col1[int64 NULLABLE] not present in tablet' in
'ImpalaBeeswaxException:\n Query aborted:Kudu error(s) reported, first error:
Invalid argument: Failed to write batch ...redacted.com:31201): Client provided
column col1 INT64 NULLABLE not present in tablet (1 of 2 similar)\n\n')
Stacktrace
query_test/test_kudu.py:442: in test_concurrent_schema_change
assert "has fewer columns (1) than the SELECT / VALUES clause returns (2)"
in msg \
E assert ('has fewer columns (1) than the SELECT / VALUES clause returns (2)'
in 'ImpalaBeeswaxException:\n Query aborted:Kudu error(s) reported, first
error: Invalid argument: Failed to write batch ...redacted.com:31201): Client
provided column col1 INT64 NULLABLE not present in tablet (1 of 2 similar)\n\n'
or "(type: TINYINT) is not compatible with column 'col1' (type: STRING)" in
'ImpalaBeeswaxException:\n Query aborted:Kudu error(s) reported, first error:
Invalid argument: Failed to write batch ...redacted.com:31201): Client provided
column col1 INT64 NULLABLE not present in tablet (1 of 2 similar)\n\n' or 'has
fewer columns than expected.' in 'ImpalaBeeswaxException:\n Query aborted:Kudu
error(s) reported, first error: Invalid argument: Failed to write batch
...redacted.com:31201): Client provided column col1 INT64 NULLABLE not present
in tablet (1 of 2 similar)\n\n' or 'Column col1 has unexpected type.' in
'ImpalaBeeswaxException:\n Query aborted:Kudu error(s) reported, first error:
Invalid argument: Failed to write batch ...redacted.com:31201): Client provided
column col1 INT64 NULLABLE not present in tablet (1 of 2 similar)\n\n' or
'Client provided column col1[int64 NULLABLE] not present in tablet' in
'ImpalaBeeswaxException:\n Query aborted:Kudu error(s) reported, first error:
Invalid argument: Failed to write batch ...redacted.com:31201): Client provided
column col1 INT64 NULLABLE not present in tablet (1 of 2 similar)\n\n')
{noformat}
We see that the message returned includes {{Client provided column col1 INT64
NULLABLE not present in tablet}} and the code is looking for:
{noformat}
437 for error in insert_thread.errors:
438 msg = str(error)
439 # The first two are AnalysisExceptions, the next two come from
KuduTableSink::Open()
440 # if the schema has changed since analysis, the last comes from the
Kudu server if
441 # the schema changes between KuduTableSink::Open() and when the
write ops are sent.
442 assert "has fewer columns (1) than the SELECT / VALUES clause
returns (2)" in msg \
443 or "(type: TINYINT) is not compatible with column 'col1' (type:
STRING)" in msg \
444 or "has fewer columns than expected." in msg \
445 or "Column col1 has unexpected type." in msg \
446 or "Client provided column col1[int64 NULLABLE] not present in
tablet" in msg
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For
[jira] [Assigned] (IMPALA-7736) stress: AssertionError: The number of executing queries is negative
[
https://issues.apache.org/jira/browse/IMPALA-7736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Brown reassigned IMPALA-7736:
-
Assignee: Michael Brown
> stress: AssertionError: The number of executing queries is negative
> ---
>
> Key: IMPALA-7736
> URL: https://issues.apache.org/jira/browse/IMPALA-7736
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
>Affects Versions: Impala 3.1.0
>Reporter: Michael Brown
>Assignee: Michael Brown
>Priority: Critical
> Labels: stress
>
> I'd heard about this anecdotally but haven't seen it until now:
> {noformat}
> Done | Active | Executing | Mem Lmt Ex | AC Reject | AC Timeout | Time Out |
> Cancel | Err | Incorrect | Next Qry Mem Lmt | Tot Qry Mem Lmt | Tracked Mem |
> RSS Mem
> 263 | 6 | 3 | 20 | 0 | 0 |0 |
>17 | 0 | 0 | 9818 | 26802 | 15656 |
> 19329
> 263 | 6 | 3 | 20 | 0 | 0 |0 |
>17 | 0 | 0 | 9818 | 26802 | 15534 |
> 19341
> 263 | 6 | 3 | 20 | 0 | 0 |0 |
>17 | 0 | 0 | 9818 | 26802 | 15462 |
> 19190
> 265 | 4 | 1 | 20 | 0 | 0 |0 |
>17 | 0 | 0 | 9818 | 22938 | 15482 |
> 19565
> Traceback (most recent call last):
> File "tests/stress/concurrent_select.py", line 2320, in
> main()
> File "tests/stress/concurrent_select.py", line 2316, in main
> select_probability=args.select_probability)
> File "tests/stress/concurrent_select.py", line 529, in run_queries
> self._wait_for_test_to_finish(impala, should_print_status)
> File "tests/stress/concurrent_select.py", line 943, in
> _wait_for_test_to_finish
> self._print_status(print_header=(lines_printed == 0))
> File "tests/stress/concurrent_select.py", line 837, in _print_status
> self._num_queries_executing,
> File "tests/stress/concurrent_select.py", line 690, in
> _num_queries_executing
> assert num_executing >= 0, "The number of executing queries is negative"
> AssertionError: The number of executing queries is negative
> {noformat}
> I've run many thousand-query stress test runs against 3.1 without hitting
> this. Where this finally happened was against a 2.12-based build, if that
> matters.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-7736) stress: AssertionError: The number of executing queries is negative
[
https://issues.apache.org/jira/browse/IMPALA-7736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16669410#comment-16669410
]
Michael Brown commented on IMPALA-7736:
---
I finally got time to come back to this, and I have a way to reproduce this
with some instrumented logging to see what's going on. Kind of a pain since I
have to wait for some time, but anyway
The bug is under counting of {{self._num_queries_started_running_or_cancelled}}.
{noformat}
(Pdb) p self._num_queries_started_running_or_cancelled.value
213
(Pdb) p self._num_queries_finished.value
218
(Pdb)
{noformat}
In my logfile, I found 5 queries that failed to increment
{{self._num_queries_started_running_or_cancelled}} and they all failed like
this:
{noformat}
ExecQueryFInstances rpc query_id=f4c264ac41953b9:f289dfd4 failed:
Failed to get minimum memory reservation of 245.25 MB on daemon redacted:22000
for query f4c264ac41953b9:f289dfd4 due to following error: Failed to
increase reservation by 245.25 MB because it would exceed the applicable
reservation limit for the "Process" ReservationTracker: reservation_limit=42.67
GB reservation=42.64 GB used_reservation=0 child_reservations=42.64 GB
{noformat}
I instrumented concurrent_select.py to log when it's incrementing counters and
did some post-processing on the logs to make it readable. A full query where
this is going on:
{noformat}
2018-10-30 10:46:14,269 1427 Query Consumer
DEBUG:concurrent_select[725]:Getting query_idx
2018-10-30 10:46:14,269 1427 Query Consumer
DEBUG:concurrent_select[742]:Waiting for other query runners to start their
queries
2018-10-30 11:00:54,832 1427 Query Consumer
DEBUG:concurrent_select[748]:Requesting memory reservation
2018-10-30 11:01:04,350 1427 Query Consumer
DEBUG:concurrent_select[382]:Reserved 636 MB; 53 MB available; 64198 MB
overcommitted
2018-10-30 11:01:04,350 1427 Query Consumer
DEBUG:concurrent_select[188]:Incrementing with id 140453288913680
2018-10-30 11:01:04,351 1427 Query Consumer
DEBUG:concurrent_select[750]:Received memory reservation
2018-10-30 11:01:04,351 1427 Query Consumer
DEBUG:concurrent_select[188]:Incrementing
with id self._num_queries_submitted <- new
DEBUG statement with post-processed logging for counters I care about
2018-10-30 11:01:04,363 1427 Query Consumer DEBUG:concurrent_select[1126]:Using
tpcds_300_decimal_parquet database
2018-10-30 11:01:04,363 1427 Query Consumer DEBUG:db_connection[203]:IMPALA:
USE tpcds_300_decimal_parquet
2018-10-30 11:01:04,394 1427 Query Consumer DEBUG:db_connection[203]:IMPALA:
SET ABORT_ON_ERROR=1
2018-10-30 11:01:04,440 1427 Query Consumer
DEBUG:concurrent_select[1143]:Setting mem limit to 636 MB
2018-10-30 11:01:04,440 1427 Query Consumer DEBUG:db_connection[203]:IMPALA:
SET MEM_LIMIT=636M
2018-10-30 11:01:04,477 1427 Query Consumer
DEBUG:concurrent_select[1147]:Running query with 636 MB mem limit at redacted5
with timeout secs 45943.8717842:
-- RESULT MISMATCH FROM ORIGINAL
-- ADD ROUND()s TO 4th, 5th, 9th, 10th COLUMNS, TAKE ACTUAL RESULTS AS EXPECTED.
with inv as
(select w_warehouse_name,w_warehouse_sk,i_item_sk,d_moy
,stdev,mean, case mean when 0 then null else stdev/mean end cov
from(select w_warehouse_name,w_warehouse_sk,i_item_sk,d_moy
,stddev_samp(inv_quantity_on_hand) stdev,avg(inv_quantity_on_hand)
mean
from inventory
,item
,warehouse
,date_dim
where inv_item_sk = i_item_sk
and inv_warehouse_sk = w_warehouse_sk
and inv_date_sk = d_date_sk
and d_year =2001
group by w_warehouse_name,w_warehouse_sk,i_item_sk,d_moy) foo
where case mean when 0 then 0 else stdev/mean end > 1)
select inv1.w_warehouse_sk,inv1.i_item_sk,inv1.d_moy
,round(inv1.mean, 2), round(inv1.cov, 8)
,inv2.w_warehouse_sk,inv2.i_item_sk,inv2.d_moy
,round(inv2.mean, 2), round(inv2.cov, 8)
from inv inv1,inv inv2
where inv1.i_item_sk = inv2.i_item_sk
and inv1.w_warehouse_sk = inv2.w_warehouse_sk
and inv1.d_moy=1
and inv2.d_moy=1+1
order by inv1.w_warehouse_sk,inv1.i_item_sk,inv1.d_moy,inv1.mean,inv1.cov
,inv2.d_moy,inv2.mean, inv2.cov
;
2018-10-30 11:01:04,803 1427 Query Consumer DEBUG:concurrent_select[1106]:Error
running query with id None: ExecQueryFInstances rpc
query_id=f4c264ac41953b9:f289dfd4 failed: Failed to get minimum memory
reservation of 245.25 MB on daemon redacted2:22000 for query
f4c264ac41953b9:f289dfd4 due to following error: Failed to increase
reservation by 245.25 MB because it would exceed the applicable reservation
limit for the "Process" ReservationTracker: reservation_limit=42.67 GB
reservation=42.64 GB used_reservation=0 child_reservations=42.64 GB
2018-10-30 11:01:04,805 1427 Query Consumer DEBUG:concurrent_select[760]:Got
execution report for query
2018-10-30 11:01:04,805 1427 Query Consumer
[jira] [Created] (IMPALA-7782) discrepancy in results with a subquery containing an agg that produces an empty set
Michael Brown created IMPALA-7782:
-
Summary: discrepancy in results with a subquery containing an agg
that produces an empty set
Key: IMPALA-7782
URL: https://issues.apache.org/jira/browse/IMPALA-7782
Project: IMPALA
Issue Type: Bug
Components: Frontend
Affects Versions: Impala 2.12.0, Impala 3.1.0
Reporter: Michael Brown
A discrepancy exists between Impala and Postgres when a subquery contains an
agg and results in an empty set, yet the WHERE clause looking at the subquery
should produce a "True" condition.
Example queries include:
{noformat}
USE functional;
SELECT id
FROM alltypestiny
WHERE -1 NOT IN (SELECT COUNT(id) FROM alltypestiny HAVING false);
SELECT id
FROM alltypestiny
WHERE NULL NOT IN (SELECT COUNT(id) FROM alltypestiny HAVING false);
SELECT id
FROM alltypestiny
WHERE (SELECT COUNT(id) FROM alltypestiny HAVING false) IS NULL;
{noformat}
These queries do not produce any rows in Impala. In Postgres, the queries
produce all 8 rows for the functional.alltypestiny id column.
Thinking maybe there were Impala and Postgres differences with {{NOT IN}}
behavior, I also tried this:
{noformat}
USE functional;
SELECT id
FROM alltypestiny
WHERE -1 NOT IN (SELECT 1 FROM alltypestiny WHERE bool_col IS NULL);
{noformat}
This subquery also produces an empty set just like the subquery in the
problematic queries at the top, but unlike those queries, this full query
returns the same results in Impala and Postgres (all 8 rows for the
functional.alltypestiny id column).
For anyone interested in this bug, you can migrate data into postgres in a dev
environment using
{noformat}
tests/comparison/data_generator.py --use-postgresql --migrate-table-names
alltypestiny --db-name functional migrate
{noformat}
This is in 2.12 at least, so it's not a 3.1 regression.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (IMPALA-5103) gerrit-verify-dryrun can spin off competing Jobs
[ https://issues.apache.org/jira/browse/IMPALA-5103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Brown resolved IMPALA-5103. --- Resolution: Cannot Reproduce > gerrit-verify-dryrun can spin off competing Jobs > > > Key: IMPALA-5103 > URL: https://issues.apache.org/jira/browse/IMPALA-5103 > Project: IMPALA > Issue Type: Bug > Components: Infrastructure >Affects Versions: Impala 2.9.0 >Reporter: Michael Brown >Priority: Major > > This review spun off competing Jenkins jobs. > https://gerrit.cloudera.org/#/c/6408/3 > I have a suspicion that the bug lies in passing the value > "https://gerrit.cloudera.org/#/c/6408/3; to the PATCH_URL (i.e., the patch > set is explicitly included) > When I tried again without the "/3", 1 job fired off. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-5103) gerrit-verify-dryrun can spin off competing Jobs
[ https://issues.apache.org/jira/browse/IMPALA-5103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Brown resolved IMPALA-5103. --- Resolution: Cannot Reproduce > gerrit-verify-dryrun can spin off competing Jobs > > > Key: IMPALA-5103 > URL: https://issues.apache.org/jira/browse/IMPALA-5103 > Project: IMPALA > Issue Type: Bug > Components: Infrastructure >Affects Versions: Impala 2.9.0 >Reporter: Michael Brown >Priority: Major > > This review spun off competing Jenkins jobs. > https://gerrit.cloudera.org/#/c/6408/3 > I have a suspicion that the bug lies in passing the value > "https://gerrit.cloudera.org/#/c/6408/3; to the PATCH_URL (i.e., the patch > set is explicitly included) > When I tried again without the "/3", 1 job fired off. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-7736) stress: AssertionError: The number of executing queries is negative
[
https://issues.apache.org/jira/browse/IMPALA-7736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16659627#comment-16659627
]
Michael Brown commented on IMPALA-7736:
---
So I did some combing to find where else this may have happened, it and it all
hits on 2.12-based, not 3.1-based builds, with the stress test executed on
master.
The assert is always this second one:
{noformat}
@property
def _num_queries_active(self):
"""The number of queries that are currently active (i.e. submitted to a
query runner
and haven't yet completed)."""
num_running = self._num_queries_submitted.value -
self._num_queries_finished.value
assert num_running >= 0, "The number of running queries is negative"
return num_running
@property
def _num_queries_executing(self):
"""The number of queries that are currently executing (i.e. entered the
RUNNING state
and haven't yet completed)."""
num_executing = (self._num_queries_started_running_or_cancelled.value -
self._num_queries_finished.value)
assert num_executing >= 0, "The number of executing queries is negative"
return num_executing
{noformat}
I..e, it's the {{num_executing}} assert.
> stress: AssertionError: The number of executing queries is negative
> ---
>
> Key: IMPALA-7736
> URL: https://issues.apache.org/jira/browse/IMPALA-7736
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
>Affects Versions: Impala 3.1.0
>Reporter: Michael Brown
>Priority: Critical
> Labels: stress
>
> I'd heard about this anecdotally but haven't seen it until now:
> {noformat}
> Done | Active | Executing | Mem Lmt Ex | AC Reject | AC Timeout | Time Out |
> Cancel | Err | Incorrect | Next Qry Mem Lmt | Tot Qry Mem Lmt | Tracked Mem |
> RSS Mem
> 263 | 6 | 3 | 20 | 0 | 0 |0 |
>17 | 0 | 0 | 9818 | 26802 | 15656 |
> 19329
> 263 | 6 | 3 | 20 | 0 | 0 |0 |
>17 | 0 | 0 | 9818 | 26802 | 15534 |
> 19341
> 263 | 6 | 3 | 20 | 0 | 0 |0 |
>17 | 0 | 0 | 9818 | 26802 | 15462 |
> 19190
> 265 | 4 | 1 | 20 | 0 | 0 |0 |
>17 | 0 | 0 | 9818 | 22938 | 15482 |
> 19565
> Traceback (most recent call last):
> File "tests/stress/concurrent_select.py", line 2320, in
> main()
> File "tests/stress/concurrent_select.py", line 2316, in main
> select_probability=args.select_probability)
> File "tests/stress/concurrent_select.py", line 529, in run_queries
> self._wait_for_test_to_finish(impala, should_print_status)
> File "tests/stress/concurrent_select.py", line 943, in
> _wait_for_test_to_finish
> self._print_status(print_header=(lines_printed == 0))
> File "tests/stress/concurrent_select.py", line 837, in _print_status
> self._num_queries_executing,
> File "tests/stress/concurrent_select.py", line 690, in
> _num_queries_executing
> assert num_executing >= 0, "The number of executing queries is negative"
> AssertionError: The number of executing queries is negative
> {noformat}
> I've run many thousand-query stress test runs against 3.1 without hitting
> this. Where this finally happened was against a 2.12-based build, if that
> matters.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-7736) stress: AssertionError: The number of executing queries is negative
[
https://issues.apache.org/jira/browse/IMPALA-7736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Brown updated IMPALA-7736:
--
Description:
I'd heard about this anecdotally but haven't seen it until now:
{noformat}
Done | Active | Executing | Mem Lmt Ex | AC Reject | AC Timeout | Time Out |
Cancel | Err | Incorrect | Next Qry Mem Lmt | Tot Qry Mem Lmt | Tracked Mem |
RSS Mem
263 | 6 | 3 | 20 | 0 | 0 |0 |
17 | 0 | 0 | 9818 | 26802 | 15656 |
19329
263 | 6 | 3 | 20 | 0 | 0 |0 |
17 | 0 | 0 | 9818 | 26802 | 15534 |
19341
263 | 6 | 3 | 20 | 0 | 0 |0 |
17 | 0 | 0 | 9818 | 26802 | 15462 |
19190
265 | 4 | 1 | 20 | 0 | 0 |0 |
17 | 0 | 0 | 9818 | 22938 | 15482 |
19565
Traceback (most recent call last):
File "tests/stress/concurrent_select.py", line 2320, in
main()
File "tests/stress/concurrent_select.py", line 2316, in main
select_probability=args.select_probability)
File "tests/stress/concurrent_select.py", line 529, in run_queries
self._wait_for_test_to_finish(impala, should_print_status)
File "tests/stress/concurrent_select.py", line 943, in
_wait_for_test_to_finish
self._print_status(print_header=(lines_printed == 0))
File "tests/stress/concurrent_select.py", line 837, in _print_status
self._num_queries_executing,
File "tests/stress/concurrent_select.py", line 690, in _num_queries_executing
assert num_executing >= 0, "The number of executing queries is negative"
AssertionError: The number of executing queries is negative
{noformat}
I've run many thousand-query stress test runs against 3.1 without hitting this.
Where this finally happened was against a 2.12-based build, if that matters.
was:
I'd heard about this anecdotally but haven't seen it until now:
{noformat}
Done | Active | Executing | Mem Lmt Ex | AC Reject | AC Timeout | Time Out |
Cancel | Err | Incorrect | Next Qry Mem Lmt | Tot Qry Mem Lmt | Tracked Mem |
RSS Mem
263 | 6 | 3 | 20 | 0 | 0 |0 |
17 | 0 | 0 | 9818 | 26802 | 15656 |
19329
263 | 6 | 3 | 20 | 0 | 0 |0 |
17 | 0 | 0 | 9818 | 26802 | 15534 |
19341
263 | 6 | 3 | 20 | 0 | 0 |0 |
17 | 0 | 0 | 9818 | 26802 | 15462 |
19190
265 | 4 | 1 | 20 | 0 | 0 |0 |
17 | 0 | 0 | 9818 | 22938 | 15482 |
19565
Traceback (most recent call last):
File "tests/stress/concurrent_select.py", line 2320, in
main()
File "tests/stress/concurrent_select.py", line 2316, in main
select_probability=args.select_probability)
File "tests/stress/concurrent_select.py", line 529, in run_queries
self._wait_for_test_to_finish(impala, should_print_status)
File "tests/stress/concurrent_select.py", line 943, in
_wait_for_test_to_finish
self._print_status(print_header=(lines_printed == 0))
File "tests/stress/concurrent_select.py", line 837, in _print_status
self._num_queries_executing,
File "tests/stress/concurrent_select.py", line 690, in _num_queries_executing
assert num_executing >= 0, "The number of executing queries is negative"
AssertionError: The number of executing queries is negative
{noformat}
I've run many-thousand stress test runs against 3.1 without hitting this. Where
this finally happened was against a 2.12-based build, if that matters.
> stress: AssertionError: The number of executing queries is negative
> ---
>
> Key: IMPALA-7736
> URL: https://issues.apache.org/jira/browse/IMPALA-7736
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
>Affects Versions: Impala 3.1.0
>Reporter: Michael Brown
>Priority: Critical
> Labels: stress
>
> I'd heard about this anecdotally but haven't seen it until now:
> {noformat}
> Done | Active | Executing | Mem Lmt Ex | AC Reject | AC Timeout | Time Out |
> Cancel | Err | Incorrect | Next Qry Mem Lmt | Tot Qry Mem Lmt | Tracked Mem |
> RSS Mem
> 263 | 6 | 3 | 20 | 0 | 0 |0 |
>17 | 0 | 0 | 9818 | 26802 | 15656 |
> 19329
> 263 | 6 | 3 | 20 | 0 | 0 |0 |
>17 | 0 | 0 |
[jira] [Created] (IMPALA-7736) stress: AssertionError: The number of executing queries is negative
Michael Brown created IMPALA-7736:
-
Summary: stress: AssertionError: The number of executing queries
is negative
Key: IMPALA-7736
URL: https://issues.apache.org/jira/browse/IMPALA-7736
Project: IMPALA
Issue Type: Bug
Components: Infrastructure
Affects Versions: Impala 3.1.0
Reporter: Michael Brown
I'd heard about this anecdotally but haven't seen it until now:
{noformat}
Done | Active | Executing | Mem Lmt Ex | AC Reject | AC Timeout | Time Out |
Cancel | Err | Incorrect | Next Qry Mem Lmt | Tot Qry Mem Lmt | Tracked Mem |
RSS Mem
263 | 6 | 3 | 20 | 0 | 0 |0 |
17 | 0 | 0 | 9818 | 26802 | 15656 |
19329
263 | 6 | 3 | 20 | 0 | 0 |0 |
17 | 0 | 0 | 9818 | 26802 | 15534 |
19341
263 | 6 | 3 | 20 | 0 | 0 |0 |
17 | 0 | 0 | 9818 | 26802 | 15462 |
19190
265 | 4 | 1 | 20 | 0 | 0 |0 |
17 | 0 | 0 | 9818 | 22938 | 15482 |
19565
Traceback (most recent call last):
File "tests/stress/concurrent_select.py", line 2320, in
main()
File "tests/stress/concurrent_select.py", line 2316, in main
select_probability=args.select_probability)
File "tests/stress/concurrent_select.py", line 529, in run_queries
self._wait_for_test_to_finish(impala, should_print_status)
File "tests/stress/concurrent_select.py", line 943, in
_wait_for_test_to_finish
self._print_status(print_header=(lines_printed == 0))
File "tests/stress/concurrent_select.py", line 837, in _print_status
self._num_queries_executing,
File "tests/stress/concurrent_select.py", line 690, in _num_queries_executing
assert num_executing >= 0, "The number of executing queries is negative"
AssertionError: The number of executing queries is negative
{noformat}
I've run many-thousand stress test runs against 3.1 without hitting this. Where
this finally happened was against a 2.12-based build, if that matters.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-7736) stress: AssertionError: The number of executing queries is negative
Michael Brown created IMPALA-7736:
-
Summary: stress: AssertionError: The number of executing queries
is negative
Key: IMPALA-7736
URL: https://issues.apache.org/jira/browse/IMPALA-7736
Project: IMPALA
Issue Type: Bug
Components: Infrastructure
Affects Versions: Impala 3.1.0
Reporter: Michael Brown
I'd heard about this anecdotally but haven't seen it until now:
{noformat}
Done | Active | Executing | Mem Lmt Ex | AC Reject | AC Timeout | Time Out |
Cancel | Err | Incorrect | Next Qry Mem Lmt | Tot Qry Mem Lmt | Tracked Mem |
RSS Mem
263 | 6 | 3 | 20 | 0 | 0 |0 |
17 | 0 | 0 | 9818 | 26802 | 15656 |
19329
263 | 6 | 3 | 20 | 0 | 0 |0 |
17 | 0 | 0 | 9818 | 26802 | 15534 |
19341
263 | 6 | 3 | 20 | 0 | 0 |0 |
17 | 0 | 0 | 9818 | 26802 | 15462 |
19190
265 | 4 | 1 | 20 | 0 | 0 |0 |
17 | 0 | 0 | 9818 | 22938 | 15482 |
19565
Traceback (most recent call last):
File "tests/stress/concurrent_select.py", line 2320, in
main()
File "tests/stress/concurrent_select.py", line 2316, in main
select_probability=args.select_probability)
File "tests/stress/concurrent_select.py", line 529, in run_queries
self._wait_for_test_to_finish(impala, should_print_status)
File "tests/stress/concurrent_select.py", line 943, in
_wait_for_test_to_finish
self._print_status(print_header=(lines_printed == 0))
File "tests/stress/concurrent_select.py", line 837, in _print_status
self._num_queries_executing,
File "tests/stress/concurrent_select.py", line 690, in _num_queries_executing
assert num_executing >= 0, "The number of executing queries is negative"
AssertionError: The number of executing queries is negative
{noformat}
I've run many-thousand stress test runs against 3.1 without hitting this. Where
this finally happened was against a 2.12-based build, if that matters.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (IMPALA-7727) failed compute stats child query status no longer propagates to parent query
[
https://issues.apache.org/jira/browse/IMPALA-7727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Brown updated IMPALA-7727:
--
Attachment: 3.1-compute-stats-profile.txt
3.1-child-profile.txt
2.12-compute-stats-profile.txt
2.12-child-profile.txt
> failed compute stats child query status no longer propagates to parent query
>
>
> Key: IMPALA-7727
> URL: https://issues.apache.org/jira/browse/IMPALA-7727
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
>Affects Versions: Impala 3.1.0
>Reporter: Michael Brown
>Assignee: bharath v
>Priority: Blocker
> Labels: regression, stress
> Attachments: 2.12-child-profile.txt, 2.12-compute-stats-profile.txt,
> 3.1-child-profile.txt, 3.1-compute-stats-profile.txt
>
>
> [~bharathv] since you have been dealing with stats, please take a look.
> Otherwise feel free to reassign. This bug prevents the stress test from
> running with compute stats statements. It triggers in non-stressful
> conditions, too.
> {noformat}
> $ impala-shell.sh -d tpch_parquet
> [localhost:21000] tpch_parquet> set mem_limit=24m;
> MEM_LIMIT set to 24m
> [localhost:21000] tpch_parquet> compute stats customer;
> Query: compute stats customer
> WARNINGS: Cancelled
> [localhost:21000] tpch_parquet>
> {noformat}
> The problem is that the child query didn't have enough memory to run, but
> this error didn't propagate up.
> {noformat}
> Query (id=384d37fb2826a962:f4b10357):
> DEBUG MODE WARNING: Query profile created while running a DEBUG build of
> Impala. Use RELEASE builds to measure query performance.
> Summary:
> Session ID: d343e1026d497bb0:7e87b342c73c108d
> Session Type: BEESWAX
> Start Time: 2018-10-18 15:16:34.036363000
> End Time: 2018-10-18 15:16:34.177711000
> Query Type: QUERY
> Query State: EXCEPTION
> Query Status: Rejected query from pool default-pool: minimum memory
> reservation is greater than memory available to the query for buffer
> reservations. Memory reservation needed given the current plan: 128.00 KB.
> Adjust either the mem_limit or the pool config (max-query-mem-limit,
> min-query-mem-limit) for the query to allow the query memory limit to be at
> least 32.12 MB. Note that changing the mem_limit may also change the plan.
> See the query profile for more information about the per-node memory
> requirements.
> Impala Version: impalad version 3.1.0-SNAPSHOT DEBUG (build
> 9f5c5e6df03824cba292fe5a619153462c11669c)
> User: mikeb
> Connected User: mikeb
> Delegated User:
> Network Address: :::127.0.0.1:46458
> Default Db: tpch_parquet
> Sql Statement: SELECT COUNT(*) FROM customer
> Coordinator: mikeb-ub162:22000
> Query Options (set by configuration): MEM_LIMIT=25165824,MT_DOP=4
> Query Options (set by configuration and planner):
> MEM_LIMIT=25165824,NUM_SCANNER_THREADS=1,MT_DOP=4
> Plan:
>
> Max Per-Host Resource Reservation: Memory=512.00KB Threads=5
> Per-Host Resource Estimates: Memory=146MB
> F01:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
> | Per-Host Resources: mem-estimate=10.00MB mem-reservation=0B
> thread-reservation=1
> PLAN-ROOT SINK
> | mem-estimate=0B mem-reservation=0B thread-reservation=0
> |
> 03:AGGREGATE [FINALIZE]
> | output: count:merge(*)
> | mem-estimate=10.00MB mem-reservation=0B spill-buffer=2.00MB
> thread-reservation=0
> | tuple-ids=1 row-size=8B cardinality=1
> | in pipelines: 03(GETNEXT), 01(OPEN)
> |
> 02:EXCHANGE [UNPARTITIONED]
> | mem-estimate=0B mem-reservation=0B thread-reservation=0
> | tuple-ids=1 row-size=8B cardinality=1
> | in pipelines: 01(GETNEXT)
> |
> F00:PLAN FRAGMENT [RANDOM] hosts=1 instances=4
> Per-Host Resources: mem-estimate=136.00MB mem-reservation=512.00KB
> thread-reservation=4
> 01:AGGREGATE
> | output: sum_init_zero(tpch_parquet.customer.parquet-stats: num_rows)
> | mem-estimate=10.00MB mem-reservation=0B spill-buffer=2.00MB
> thread-reservation=0
> | tuple-ids=1 row-size=8B cardinality=1
> | in pipelines: 01(GETNEXT), 00(OPEN)
> |
> 00:SCAN HDFS [tpch_parquet.customer, RANDOM]
>partitions=1/1 files=1 size=12.34MB
>stored statistics:
> table: rows=15 size=12.34MB
> columns: all
>extrapolated-rows=disabled max-scan-range-rows=15
>mem-estimate=24.00MB mem-reservation=128.00KB thread-reservation=0
>tuple-ids=0 row-size=8B cardinality=15
>in pipelines: 00(GETNEXT)
>
> Estimated Per-Host Mem: 153092096
> Per Host Min Memory Reservation: mikeb-ub162:22000(0)
> mikeb-ub162:22001(128.00 KB)
> Request Pool: default-pool
> Admission result: Rejected
> Query Compilation: 126.903ms
>
[jira] [Commented] (IMPALA-7727) compute stats child query errors no longer propagate to parent query
[
https://issues.apache.org/jira/browse/IMPALA-7727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657183#comment-16657183
]
Michael Brown commented on IMPALA-7727:
---
I played around with this a little bit more and the *Query Status* is what
isn't propagating in 3.1 vs. 2.12. I'll attach profiles to this Jira.
> compute stats child query errors no longer propagate to parent query
>
>
> Key: IMPALA-7727
> URL: https://issues.apache.org/jira/browse/IMPALA-7727
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
>Affects Versions: Impala 3.1.0
>Reporter: Michael Brown
>Assignee: bharath v
>Priority: Blocker
> Labels: regression, stress
>
> [~bharathv] since you have been dealing with stats, please take a look.
> Otherwise feel free to reassign. This bug prevents the stress test from
> running with compute stats statements. It triggers in non-stressful
> conditions, too.
> {noformat}
> $ impala-shell.sh -d tpch_parquet
> [localhost:21000] tpch_parquet> set mem_limit=24m;
> MEM_LIMIT set to 24m
> [localhost:21000] tpch_parquet> compute stats customer;
> Query: compute stats customer
> WARNINGS: Cancelled
> [localhost:21000] tpch_parquet>
> {noformat}
> The problem is that the child query didn't have enough memory to run, but
> this error didn't propagate up.
> {noformat}
> Query (id=384d37fb2826a962:f4b10357):
> DEBUG MODE WARNING: Query profile created while running a DEBUG build of
> Impala. Use RELEASE builds to measure query performance.
> Summary:
> Session ID: d343e1026d497bb0:7e87b342c73c108d
> Session Type: BEESWAX
> Start Time: 2018-10-18 15:16:34.036363000
> End Time: 2018-10-18 15:16:34.177711000
> Query Type: QUERY
> Query State: EXCEPTION
> Query Status: Rejected query from pool default-pool: minimum memory
> reservation is greater than memory available to the query for buffer
> reservations. Memory reservation needed given the current plan: 128.00 KB.
> Adjust either the mem_limit or the pool config (max-query-mem-limit,
> min-query-mem-limit) for the query to allow the query memory limit to be at
> least 32.12 MB. Note that changing the mem_limit may also change the plan.
> See the query profile for more information about the per-node memory
> requirements.
> Impala Version: impalad version 3.1.0-SNAPSHOT DEBUG (build
> 9f5c5e6df03824cba292fe5a619153462c11669c)
> User: mikeb
> Connected User: mikeb
> Delegated User:
> Network Address: :::127.0.0.1:46458
> Default Db: tpch_parquet
> Sql Statement: SELECT COUNT(*) FROM customer
> Coordinator: mikeb-ub162:22000
> Query Options (set by configuration): MEM_LIMIT=25165824,MT_DOP=4
> Query Options (set by configuration and planner):
> MEM_LIMIT=25165824,NUM_SCANNER_THREADS=1,MT_DOP=4
> Plan:
>
> Max Per-Host Resource Reservation: Memory=512.00KB Threads=5
> Per-Host Resource Estimates: Memory=146MB
> F01:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
> | Per-Host Resources: mem-estimate=10.00MB mem-reservation=0B
> thread-reservation=1
> PLAN-ROOT SINK
> | mem-estimate=0B mem-reservation=0B thread-reservation=0
> |
> 03:AGGREGATE [FINALIZE]
> | output: count:merge(*)
> | mem-estimate=10.00MB mem-reservation=0B spill-buffer=2.00MB
> thread-reservation=0
> | tuple-ids=1 row-size=8B cardinality=1
> | in pipelines: 03(GETNEXT), 01(OPEN)
> |
> 02:EXCHANGE [UNPARTITIONED]
> | mem-estimate=0B mem-reservation=0B thread-reservation=0
> | tuple-ids=1 row-size=8B cardinality=1
> | in pipelines: 01(GETNEXT)
> |
> F00:PLAN FRAGMENT [RANDOM] hosts=1 instances=4
> Per-Host Resources: mem-estimate=136.00MB mem-reservation=512.00KB
> thread-reservation=4
> 01:AGGREGATE
> | output: sum_init_zero(tpch_parquet.customer.parquet-stats: num_rows)
> | mem-estimate=10.00MB mem-reservation=0B spill-buffer=2.00MB
> thread-reservation=0
> | tuple-ids=1 row-size=8B cardinality=1
> | in pipelines: 01(GETNEXT), 00(OPEN)
> |
> 00:SCAN HDFS [tpch_parquet.customer, RANDOM]
>partitions=1/1 files=1 size=12.34MB
>stored statistics:
> table: rows=15 size=12.34MB
> columns: all
>extrapolated-rows=disabled max-scan-range-rows=15
>mem-estimate=24.00MB mem-reservation=128.00KB thread-reservation=0
>tuple-ids=0 row-size=8B cardinality=15
>in pipelines: 00(GETNEXT)
>
> Estimated Per-Host Mem: 153092096
> Per Host Min Memory Reservation: mikeb-ub162:22000(0)
> mikeb-ub162:22001(128.00 KB)
> Request Pool: default-pool
> Admission result: Rejected
> Query Compilation: 126.903ms
>- Metadata of all 1 tables cached: 5.484ms (5.484ms)
>- Analysis finished: 16.104ms (10.619ms)
>- Value transfer
[jira] [Updated] (IMPALA-7727) failed compute stats child query status no longer propagates to parent query
[
https://issues.apache.org/jira/browse/IMPALA-7727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Brown updated IMPALA-7727:
--
Summary: failed compute stats child query status no longer propagates to
parent query (was: compute stats child query errors no longer propagate to
parent query)
> failed compute stats child query status no longer propagates to parent query
>
>
> Key: IMPALA-7727
> URL: https://issues.apache.org/jira/browse/IMPALA-7727
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
>Affects Versions: Impala 3.1.0
>Reporter: Michael Brown
>Assignee: bharath v
>Priority: Blocker
> Labels: regression, stress
>
> [~bharathv] since you have been dealing with stats, please take a look.
> Otherwise feel free to reassign. This bug prevents the stress test from
> running with compute stats statements. It triggers in non-stressful
> conditions, too.
> {noformat}
> $ impala-shell.sh -d tpch_parquet
> [localhost:21000] tpch_parquet> set mem_limit=24m;
> MEM_LIMIT set to 24m
> [localhost:21000] tpch_parquet> compute stats customer;
> Query: compute stats customer
> WARNINGS: Cancelled
> [localhost:21000] tpch_parquet>
> {noformat}
> The problem is that the child query didn't have enough memory to run, but
> this error didn't propagate up.
> {noformat}
> Query (id=384d37fb2826a962:f4b10357):
> DEBUG MODE WARNING: Query profile created while running a DEBUG build of
> Impala. Use RELEASE builds to measure query performance.
> Summary:
> Session ID: d343e1026d497bb0:7e87b342c73c108d
> Session Type: BEESWAX
> Start Time: 2018-10-18 15:16:34.036363000
> End Time: 2018-10-18 15:16:34.177711000
> Query Type: QUERY
> Query State: EXCEPTION
> Query Status: Rejected query from pool default-pool: minimum memory
> reservation is greater than memory available to the query for buffer
> reservations. Memory reservation needed given the current plan: 128.00 KB.
> Adjust either the mem_limit or the pool config (max-query-mem-limit,
> min-query-mem-limit) for the query to allow the query memory limit to be at
> least 32.12 MB. Note that changing the mem_limit may also change the plan.
> See the query profile for more information about the per-node memory
> requirements.
> Impala Version: impalad version 3.1.0-SNAPSHOT DEBUG (build
> 9f5c5e6df03824cba292fe5a619153462c11669c)
> User: mikeb
> Connected User: mikeb
> Delegated User:
> Network Address: :::127.0.0.1:46458
> Default Db: tpch_parquet
> Sql Statement: SELECT COUNT(*) FROM customer
> Coordinator: mikeb-ub162:22000
> Query Options (set by configuration): MEM_LIMIT=25165824,MT_DOP=4
> Query Options (set by configuration and planner):
> MEM_LIMIT=25165824,NUM_SCANNER_THREADS=1,MT_DOP=4
> Plan:
>
> Max Per-Host Resource Reservation: Memory=512.00KB Threads=5
> Per-Host Resource Estimates: Memory=146MB
> F01:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
> | Per-Host Resources: mem-estimate=10.00MB mem-reservation=0B
> thread-reservation=1
> PLAN-ROOT SINK
> | mem-estimate=0B mem-reservation=0B thread-reservation=0
> |
> 03:AGGREGATE [FINALIZE]
> | output: count:merge(*)
> | mem-estimate=10.00MB mem-reservation=0B spill-buffer=2.00MB
> thread-reservation=0
> | tuple-ids=1 row-size=8B cardinality=1
> | in pipelines: 03(GETNEXT), 01(OPEN)
> |
> 02:EXCHANGE [UNPARTITIONED]
> | mem-estimate=0B mem-reservation=0B thread-reservation=0
> | tuple-ids=1 row-size=8B cardinality=1
> | in pipelines: 01(GETNEXT)
> |
> F00:PLAN FRAGMENT [RANDOM] hosts=1 instances=4
> Per-Host Resources: mem-estimate=136.00MB mem-reservation=512.00KB
> thread-reservation=4
> 01:AGGREGATE
> | output: sum_init_zero(tpch_parquet.customer.parquet-stats: num_rows)
> | mem-estimate=10.00MB mem-reservation=0B spill-buffer=2.00MB
> thread-reservation=0
> | tuple-ids=1 row-size=8B cardinality=1
> | in pipelines: 01(GETNEXT), 00(OPEN)
> |
> 00:SCAN HDFS [tpch_parquet.customer, RANDOM]
>partitions=1/1 files=1 size=12.34MB
>stored statistics:
> table: rows=15 size=12.34MB
> columns: all
>extrapolated-rows=disabled max-scan-range-rows=15
>mem-estimate=24.00MB mem-reservation=128.00KB thread-reservation=0
>tuple-ids=0 row-size=8B cardinality=15
>in pipelines: 00(GETNEXT)
>
> Estimated Per-Host Mem: 153092096
> Per Host Min Memory Reservation: mikeb-ub162:22000(0)
> mikeb-ub162:22001(128.00 KB)
> Request Pool: default-pool
> Admission result: Rejected
> Query Compilation: 126.903ms
>- Metadata of all 1 tables cached: 5.484ms (5.484ms)
>- Analysis finished: 16.104ms (10.619ms)
>- Value transfer
[jira] [Commented] (IMPALA-7727) compute stats child query errors no longer propagate to parent query
[
https://issues.apache.org/jira/browse/IMPALA-7727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16655993#comment-16655993
]
Michael Brown commented on IMPALA-7727:
---
We've been hit by bugs like this before, like IMPALA-4981. Suggestion is an
end-to-end test that does the following:
1. Run compute stats child queries directly (as real queries) with a low memory
limit and make sure the error string has a reasonable match.
2. Run compute stats with the same low memory limit and assert the same error
message is propagated.
> compute stats child query errors no longer propagate to parent query
>
>
> Key: IMPALA-7727
> URL: https://issues.apache.org/jira/browse/IMPALA-7727
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
>Affects Versions: Impala 3.1.0
>Reporter: Michael Brown
>Assignee: bharath v
>Priority: Blocker
> Labels: regression, stress
>
> [~bharathv] since you have been dealing with stats, please take a look.
> Otherwise feel free to reassign. This bug prevents the stress test from
> running with compute stats statements. It triggers in non-stressful
> conditions, too.
> {noformat}
> $ impala-shell.sh -d tpch_parquet
> [localhost:21000] tpch_parquet> set mem_limit=24m;
> MEM_LIMIT set to 24m
> [localhost:21000] tpch_parquet> compute stats customer;
> Query: compute stats customer
> WARNINGS: Cancelled
> [localhost:21000] tpch_parquet>
> {noformat}
> The problem is that the child query didn't have enough memory to run, but
> this error didn't propagate up.
> {noformat}
> Query (id=384d37fb2826a962:f4b10357):
> DEBUG MODE WARNING: Query profile created while running a DEBUG build of
> Impala. Use RELEASE builds to measure query performance.
> Summary:
> Session ID: d343e1026d497bb0:7e87b342c73c108d
> Session Type: BEESWAX
> Start Time: 2018-10-18 15:16:34.036363000
> End Time: 2018-10-18 15:16:34.177711000
> Query Type: QUERY
> Query State: EXCEPTION
> Query Status: Rejected query from pool default-pool: minimum memory
> reservation is greater than memory available to the query for buffer
> reservations. Memory reservation needed given the current plan: 128.00 KB.
> Adjust either the mem_limit or the pool config (max-query-mem-limit,
> min-query-mem-limit) for the query to allow the query memory limit to be at
> least 32.12 MB. Note that changing the mem_limit may also change the plan.
> See the query profile for more information about the per-node memory
> requirements.
> Impala Version: impalad version 3.1.0-SNAPSHOT DEBUG (build
> 9f5c5e6df03824cba292fe5a619153462c11669c)
> User: mikeb
> Connected User: mikeb
> Delegated User:
> Network Address: :::127.0.0.1:46458
> Default Db: tpch_parquet
> Sql Statement: SELECT COUNT(*) FROM customer
> Coordinator: mikeb-ub162:22000
> Query Options (set by configuration): MEM_LIMIT=25165824,MT_DOP=4
> Query Options (set by configuration and planner):
> MEM_LIMIT=25165824,NUM_SCANNER_THREADS=1,MT_DOP=4
> Plan:
>
> Max Per-Host Resource Reservation: Memory=512.00KB Threads=5
> Per-Host Resource Estimates: Memory=146MB
> F01:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
> | Per-Host Resources: mem-estimate=10.00MB mem-reservation=0B
> thread-reservation=1
> PLAN-ROOT SINK
> | mem-estimate=0B mem-reservation=0B thread-reservation=0
> |
> 03:AGGREGATE [FINALIZE]
> | output: count:merge(*)
> | mem-estimate=10.00MB mem-reservation=0B spill-buffer=2.00MB
> thread-reservation=0
> | tuple-ids=1 row-size=8B cardinality=1
> | in pipelines: 03(GETNEXT), 01(OPEN)
> |
> 02:EXCHANGE [UNPARTITIONED]
> | mem-estimate=0B mem-reservation=0B thread-reservation=0
> | tuple-ids=1 row-size=8B cardinality=1
> | in pipelines: 01(GETNEXT)
> |
> F00:PLAN FRAGMENT [RANDOM] hosts=1 instances=4
> Per-Host Resources: mem-estimate=136.00MB mem-reservation=512.00KB
> thread-reservation=4
> 01:AGGREGATE
> | output: sum_init_zero(tpch_parquet.customer.parquet-stats: num_rows)
> | mem-estimate=10.00MB mem-reservation=0B spill-buffer=2.00MB
> thread-reservation=0
> | tuple-ids=1 row-size=8B cardinality=1
> | in pipelines: 01(GETNEXT), 00(OPEN)
> |
> 00:SCAN HDFS [tpch_parquet.customer, RANDOM]
>partitions=1/1 files=1 size=12.34MB
>stored statistics:
> table: rows=15 size=12.34MB
> columns: all
>extrapolated-rows=disabled max-scan-range-rows=15
>mem-estimate=24.00MB mem-reservation=128.00KB thread-reservation=0
>tuple-ids=0 row-size=8B cardinality=15
>in pipelines: 00(GETNEXT)
>
> Estimated Per-Host Mem: 153092096
> Per Host Min Memory Reservation: mikeb-ub162:22000(0)
> mikeb-ub162:22001(128.00 KB)
> Request Pool:
[jira] [Created] (IMPALA-7727) compute stats child query errors no longer propagate to parent query
Michael Brown created IMPALA-7727:
-
Summary: compute stats child query errors no longer propagate to
parent query
Key: IMPALA-7727
URL: https://issues.apache.org/jira/browse/IMPALA-7727
Project: IMPALA
Issue Type: Bug
Components: Frontend
Affects Versions: Impala 3.1.0
Reporter: Michael Brown
Assignee: bharath v
[~bharathv] since you have been dealing with stats, please take a look.
Otherwise feel free to reassign. This bug prevents the stress test from running
with compute stats statements. It triggers in non-stressful conditions, too.
{noformat}
$ impala-shell.sh -d tpch_parquet
[localhost:21000] tpch_parquet> set mem_limit=24m;
MEM_LIMIT set to 24m
[localhost:21000] tpch_parquet> compute stats customer;
Query: compute stats customer
WARNINGS: Cancelled
[localhost:21000] tpch_parquet>
{noformat}
The problem is that the child query didn't have enough memory to run, but this
error didn't propagate up.
{noformat}
Query (id=384d37fb2826a962:f4b10357):
DEBUG MODE WARNING: Query profile created while running a DEBUG build of
Impala. Use RELEASE builds to measure query performance.
Summary:
Session ID: d343e1026d497bb0:7e87b342c73c108d
Session Type: BEESWAX
Start Time: 2018-10-18 15:16:34.036363000
End Time: 2018-10-18 15:16:34.177711000
Query Type: QUERY
Query State: EXCEPTION
Query Status: Rejected query from pool default-pool: minimum memory
reservation is greater than memory available to the query for buffer
reservations. Memory reservation needed given the current plan: 128.00 KB.
Adjust either the mem_limit or the pool config (max-query-mem-limit,
min-query-mem-limit) for the query to allow the query memory limit to be at
least 32.12 MB. Note that changing the mem_limit may also change the plan. See
the query profile for more information about the per-node memory requirements.
Impala Version: impalad version 3.1.0-SNAPSHOT DEBUG (build
9f5c5e6df03824cba292fe5a619153462c11669c)
User: mikeb
Connected User: mikeb
Delegated User:
Network Address: :::127.0.0.1:46458
Default Db: tpch_parquet
Sql Statement: SELECT COUNT(*) FROM customer
Coordinator: mikeb-ub162:22000
Query Options (set by configuration): MEM_LIMIT=25165824,MT_DOP=4
Query Options (set by configuration and planner):
MEM_LIMIT=25165824,NUM_SCANNER_THREADS=1,MT_DOP=4
Plan:
Max Per-Host Resource Reservation: Memory=512.00KB Threads=5
Per-Host Resource Estimates: Memory=146MB
F01:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
| Per-Host Resources: mem-estimate=10.00MB mem-reservation=0B
thread-reservation=1
PLAN-ROOT SINK
| mem-estimate=0B mem-reservation=0B thread-reservation=0
|
03:AGGREGATE [FINALIZE]
| output: count:merge(*)
| mem-estimate=10.00MB mem-reservation=0B spill-buffer=2.00MB
thread-reservation=0
| tuple-ids=1 row-size=8B cardinality=1
| in pipelines: 03(GETNEXT), 01(OPEN)
|
02:EXCHANGE [UNPARTITIONED]
| mem-estimate=0B mem-reservation=0B thread-reservation=0
| tuple-ids=1 row-size=8B cardinality=1
| in pipelines: 01(GETNEXT)
|
F00:PLAN FRAGMENT [RANDOM] hosts=1 instances=4
Per-Host Resources: mem-estimate=136.00MB mem-reservation=512.00KB
thread-reservation=4
01:AGGREGATE
| output: sum_init_zero(tpch_parquet.customer.parquet-stats: num_rows)
| mem-estimate=10.00MB mem-reservation=0B spill-buffer=2.00MB
thread-reservation=0
| tuple-ids=1 row-size=8B cardinality=1
| in pipelines: 01(GETNEXT), 00(OPEN)
|
00:SCAN HDFS [tpch_parquet.customer, RANDOM]
partitions=1/1 files=1 size=12.34MB
stored statistics:
table: rows=15 size=12.34MB
columns: all
extrapolated-rows=disabled max-scan-range-rows=15
mem-estimate=24.00MB mem-reservation=128.00KB thread-reservation=0
tuple-ids=0 row-size=8B cardinality=15
in pipelines: 00(GETNEXT)
Estimated Per-Host Mem: 153092096
Per Host Min Memory Reservation: mikeb-ub162:22000(0)
mikeb-ub162:22001(128.00 KB)
Request Pool: default-pool
Admission result: Rejected
Query Compilation: 126.903ms
- Metadata of all 1 tables cached: 5.484ms (5.484ms)
- Analysis finished: 16.104ms (10.619ms)
- Value transfer graph computed: 32.646ms (16.542ms)
- Single node plan created: 61.289ms (28.642ms)
- Runtime filters computed: 66.148ms (4.859ms)
- Distributed plan created: 66.428ms (280.057us)
- Parallel plans created: 67.866ms (1.437ms)
- Planning finished: 126.903ms (59.037ms)
Query Timeline: 140.000ms
- Query submitted: 0.000ns (0.000ns)
- Planning finished: 140.000ms (140.000ms)
- Submit for admission: 140.000ms (0.000ns)
- Completed admission: 140.000ms (0.000ns)
- Rows available: 140.000ms (0.000ns)
- Unregister query: 140.000ms (0.000ns)
-
[jira] [Created] (IMPALA-7727) compute stats child query errors no longer propagate to parent query
Michael Brown created IMPALA-7727:
-
Summary: compute stats child query errors no longer propagate to
parent query
Key: IMPALA-7727
URL: https://issues.apache.org/jira/browse/IMPALA-7727
Project: IMPALA
Issue Type: Bug
Components: Frontend
Affects Versions: Impala 3.1.0
Reporter: Michael Brown
Assignee: bharath v
[~bharathv] since you have been dealing with stats, please take a look.
Otherwise feel free to reassign. This bug prevents the stress test from running
with compute stats statements. It triggers in non-stressful conditions, too.
{noformat}
$ impala-shell.sh -d tpch_parquet
[localhost:21000] tpch_parquet> set mem_limit=24m;
MEM_LIMIT set to 24m
[localhost:21000] tpch_parquet> compute stats customer;
Query: compute stats customer
WARNINGS: Cancelled
[localhost:21000] tpch_parquet>
{noformat}
The problem is that the child query didn't have enough memory to run, but this
error didn't propagate up.
{noformat}
Query (id=384d37fb2826a962:f4b10357):
DEBUG MODE WARNING: Query profile created while running a DEBUG build of
Impala. Use RELEASE builds to measure query performance.
Summary:
Session ID: d343e1026d497bb0:7e87b342c73c108d
Session Type: BEESWAX
Start Time: 2018-10-18 15:16:34.036363000
End Time: 2018-10-18 15:16:34.177711000
Query Type: QUERY
Query State: EXCEPTION
Query Status: Rejected query from pool default-pool: minimum memory
reservation is greater than memory available to the query for buffer
reservations. Memory reservation needed given the current plan: 128.00 KB.
Adjust either the mem_limit or the pool config (max-query-mem-limit,
min-query-mem-limit) for the query to allow the query memory limit to be at
least 32.12 MB. Note that changing the mem_limit may also change the plan. See
the query profile for more information about the per-node memory requirements.
Impala Version: impalad version 3.1.0-SNAPSHOT DEBUG (build
9f5c5e6df03824cba292fe5a619153462c11669c)
User: mikeb
Connected User: mikeb
Delegated User:
Network Address: :::127.0.0.1:46458
Default Db: tpch_parquet
Sql Statement: SELECT COUNT(*) FROM customer
Coordinator: mikeb-ub162:22000
Query Options (set by configuration): MEM_LIMIT=25165824,MT_DOP=4
Query Options (set by configuration and planner):
MEM_LIMIT=25165824,NUM_SCANNER_THREADS=1,MT_DOP=4
Plan:
Max Per-Host Resource Reservation: Memory=512.00KB Threads=5
Per-Host Resource Estimates: Memory=146MB
F01:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
| Per-Host Resources: mem-estimate=10.00MB mem-reservation=0B
thread-reservation=1
PLAN-ROOT SINK
| mem-estimate=0B mem-reservation=0B thread-reservation=0
|
03:AGGREGATE [FINALIZE]
| output: count:merge(*)
| mem-estimate=10.00MB mem-reservation=0B spill-buffer=2.00MB
thread-reservation=0
| tuple-ids=1 row-size=8B cardinality=1
| in pipelines: 03(GETNEXT), 01(OPEN)
|
02:EXCHANGE [UNPARTITIONED]
| mem-estimate=0B mem-reservation=0B thread-reservation=0
| tuple-ids=1 row-size=8B cardinality=1
| in pipelines: 01(GETNEXT)
|
F00:PLAN FRAGMENT [RANDOM] hosts=1 instances=4
Per-Host Resources: mem-estimate=136.00MB mem-reservation=512.00KB
thread-reservation=4
01:AGGREGATE
| output: sum_init_zero(tpch_parquet.customer.parquet-stats: num_rows)
| mem-estimate=10.00MB mem-reservation=0B spill-buffer=2.00MB
thread-reservation=0
| tuple-ids=1 row-size=8B cardinality=1
| in pipelines: 01(GETNEXT), 00(OPEN)
|
00:SCAN HDFS [tpch_parquet.customer, RANDOM]
partitions=1/1 files=1 size=12.34MB
stored statistics:
table: rows=15 size=12.34MB
columns: all
extrapolated-rows=disabled max-scan-range-rows=15
mem-estimate=24.00MB mem-reservation=128.00KB thread-reservation=0
tuple-ids=0 row-size=8B cardinality=15
in pipelines: 00(GETNEXT)
Estimated Per-Host Mem: 153092096
Per Host Min Memory Reservation: mikeb-ub162:22000(0)
mikeb-ub162:22001(128.00 KB)
Request Pool: default-pool
Admission result: Rejected
Query Compilation: 126.903ms
- Metadata of all 1 tables cached: 5.484ms (5.484ms)
- Analysis finished: 16.104ms (10.619ms)
- Value transfer graph computed: 32.646ms (16.542ms)
- Single node plan created: 61.289ms (28.642ms)
- Runtime filters computed: 66.148ms (4.859ms)
- Distributed plan created: 66.428ms (280.057us)
- Parallel plans created: 67.866ms (1.437ms)
- Planning finished: 126.903ms (59.037ms)
Query Timeline: 140.000ms
- Query submitted: 0.000ns (0.000ns)
- Planning finished: 140.000ms (140.000ms)
- Submit for admission: 140.000ms (0.000ns)
- Completed admission: 140.000ms (0.000ns)
- Rows available: 140.000ms (0.000ns)
- Unregister query: 140.000ms (0.000ns)
-
[jira] [Resolved] (IMPALA-7693) stress test binary search fails to set query names
[
https://issues.apache.org/jira/browse/IMPALA-7693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Brown resolved IMPALA-7693.
---
Resolution: Fixed
Fix Version/s: Impala 3.1.0
https://gerrit.cloudera.org/#/c/11651/
> stress test binary search fails to set query names
> --
>
> Key: IMPALA-7693
> URL: https://issues.apache.org/jira/browse/IMPALA-7693
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
>Affects Versions: Impala 3.1.0
>Reporter: Michael Brown
>Assignee: Michael Brown
>Priority: Major
> Fix For: Impala 3.1.0
>
>
> Commit {{IMPALA-7460 part 1: require user to install Paramiko and Fabric}}
> refactored the stress test to make the existing unit testable pieces not
> require imports of external Python dependencies. The refactor failed to set
> the stress test {{Query().name}} attributes properly. Logging, runtime info
> JSON, and profile files all have "None" in them.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
