[jira] [Updated] (IMPALA-8466) Unrelated failures occurred after sending a patch set to Jenkins for dry-run tests
[
https://issues.apache.org/jira/browse/IMPALA-8466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fang-Yu Rao updated IMPALA-8466:
Description:
After sending my patch set to Jenkins for dry-run tests
(https://jenkins.impala.io/job/gerrit-verify-dryrun/4085/console), unrelated
failures occurred. Specifically, some tests under ubuntu-16.04-dockerised-tests
failed
(https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/90/console). In
the following the failed test cases are given.
{code:java}
19:33:57 [gw7] xfail
webserver/test_web_pages.py::TestWebPage::test_backend_instances
19:33:57 webserver/test_web_pages.py::TestWebPage::test_io_mgr_threads
19:43:36 [gw11] xfail
webserver/test_web_pages.py::TestWebPage::test_backend_instances_mt_dop
19:43:37 [gw8] xfail
webserver/test_web_pages.py::TestWebPage::test_backend_states
19:47:28 XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: {'batch_size': 1, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
avro/snap/block]
19:47:28 reason: Expected results differ across file formats
19:47:28 XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
avro/snap/block]
19:47:28 reason: Expected results differ across file formats
19:47:28 XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: {'batch_size': 1, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': True, 'abort_on_error':
1, 'exec_single_node_rows_threshold': 0} | table_format: rc/snap/block]
19:47:28 reason: Expected results differ across file formats
19:47:28 XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: {'batch_size': 1, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
orc/def/block]
19:47:28 reason: Expected results differ across file formats
19:47:28 XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': True, 'abort_on_error':
1, 'exec_single_node_rows_threshold': 0} | table_format: kudu/none]
19:47:28 reason: Expected results differ across file formats
19:47:28 XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': True, 'abort_on_error':
1, 'exec_single_node_rows_threshold': 0} | table_format: avro/snap/block]
19:47:28 reason: Expected results differ across file formats
19:47:28 XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: {'batch_size': 1, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': True, 'abort_on_error':
1, 'exec_single_node_rows_threshold': 0} | table_format: kudu/none]
19:47:28 reason: Expected results differ across file formats
19:47:28 XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: {'batch_size': 1, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
seq/snap/block]
19:47:28 reason: Expected results differ across file formats
19:47:28 XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: {'batch_size': 1, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': True, 'abort_on_error':
1, 'exec_single_node_rows_threshold': 0} | table_format: avro/snap/block]
19:47:28 reason: Expected results differ across file formats
19:47:28 XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: {'batch_size': 1, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
kudu/none]
19:47:28 reason: Expected results differ across file formats
19:47:28 XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: {'batch_size': 1, 'num_nodes': 0,
[jira] [Updated] (IMPALA-8466) Unrelated failures occurred after sending a patch set to Jenkins for dry-run tests
[
https://issues.apache.org/jira/browse/IMPALA-8466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fang-Yu Rao updated IMPALA-8466:
Description:
After sending my patch set to Jenkins for dry-run tests
([https://jenkins.impala.io/job/gerrit-verify-dryrun/4085/console]), unrelated
failures occurred. Specifically, some tests under ubuntu-16.04-dockerised-tests
failed
([https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/90/console]). In
the following the failed test cases are given.
19:33:57 [gw7] xfail
webserver/test_web_pages.py::TestWebPage::test_backend_instances
19:33:57 webserver/test_web_pages.py::TestWebPage::test_io_mgr_threads
19:43:36 [gw11] xfail
webserver/test_web_pages.py::TestWebPage::test_backend_instances_mt_dop
19:43:37 [gw8] xfail
webserver/test_web_pages.py::TestWebPage::test_backend_states
19:47:28 XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option:
{'batch_size': 1, 'num_nodes': 0, 'disable_codegen_rows_threshold': 0,
'disable_codegen': False, 'abort_on_error': 1,
'exec_single_node_rows_threshold': 0}
| table_format: avro/snap/block]
19:47:28 reason: Expected results differ across file formats
19:47:28 XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option:
{'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 0,
'disable_codegen': False, 'abort_on_error': 1,
'exec_single_node_rows_threshold': 0}
| table_format: avro/snap/block]
19:47:28 reason: Expected results differ across file formats
19:47:28 XFAIL data_errors/test_data_errors.py::TestHdfsScanNodeErrors:
{code:java}
// code placeholder
:({code}
)::test_hdfs_scan_node_errors[protocol: beeswax | exec_option:
{'batch_size': 1, 'num_nodes': 0, 'disable_codegen_rows_threshold': 0,
'disable_codegen': True, 'abort_on_error': 1,
'exec_single_node_rows_threshold': 0}
| table_format: rc/snap/block]
19:47:28 reason: Expected results differ across file formats
19:47:28 XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option:
{'batch_size': 1, 'num_nodes': 0, 'disable_codegen_rows_threshold': 0,
'disable_codegen': False, 'abort_on_error': 1,
'exec_single_node_rows_threshold': 0}
| table_format: orc/def/block]
19:47:28 reason: Expected results differ across file formats
19:47:28 XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option:
{'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 0,
'disable_codegen': True, 'abort_on_error': 1,
'exec_single_node_rows_threshold': 0}
| table_format: kudu/none]
19:47:28 reason: Expected results differ across file formats
19:47:28 XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option:
{'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 0,
'disable_codegen': True, 'abort_on_error': 1,
'exec_single_node_rows_threshold': 0}
| table_format: avro/snap/block]
19:47:28 reason: Expected results differ across file formats
19:47:28 XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option:
{'batch_size': 1, 'num_nodes': 0, 'disable_codegen_rows_threshold': 0,
'disable_codegen': True, 'abort_on_error': 1,
'exec_single_node_rows_threshold': 0}
| table_format: kudu/none]
19:47:28 reason: Expected results differ across file formats
19:47:28 XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option:
{'batch_size': 1, 'num_nodes': 0, 'disable_codegen_rows_threshold': 0,
'disable_codegen': False, 'abort_on_error': 1,
'exec_single_node_rows_threshold': 0}
| table_format: seq/snap/block]
19:47:28 reason: Expected results differ across file formats
19:47:28 XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option:
{'batch_size': 1, 'num_nodes': 0, 'disable_codegen_rows_threshold': 0,
'disable_codegen': True, 'abort_on_error': 1,
'exec_single_node_rows_threshold': 0}
| table_format: avro/snap/block]
19:47:28 reason: Expected results differ across file formats
19:47:28 XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option:
{'batch_size': 1, 'num_nodes': 0, 'disable_codegen_rows_threshold': 0,
'disable_codegen': False, 'abort_on_error': 1,
'exec_single_node_rows_threshold': 0}
| table_format: kudu/none]
19:47:28 reason: Expected results differ across file formats
19:47:28 XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option:
[jira] [Updated] (IMPALA-8466) Unrelated failures occurred after sending a patch set to Jenkins for dry-run tests
[
https://issues.apache.org/jira/browse/IMPALA-8466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fang-Yu Rao updated IMPALA-8466:
Description:
After sending my patch set to Jenkins for dry-run tests
([https://jenkins.impala.io/job/gerrit-verify-dryrun/4085/console]), unrelated
failures occurred. Specifically, some tests under ubuntu-16.04-dockerised-tests
failed
([https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/90/console]). In
the following the failed test cases are given.
*19:33:57* [gw7] xfail
webserver/test_web_pages.py::TestWebPage::test_backend_instances
*19:33:57* webserver/test_web_pages.py::TestWebPage::test_io_mgr_threads
*19:43:36* [gw11] xfail
webserver/test_web_pages.py::TestWebPage::test_backend_instances_mt_dop
*19:43:37* [gw8] xfail
webserver/test_web_pages.py::TestWebPage::test_backend_states
*19:47:28* XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: \{'batch_size': 1, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
avro/snap/block]
*19:47:28* reason: Expected results differ across file formats
was:
After sending my patch set to Jenkins for dry-run tests
([https://jenkins.impala.io/job/gerrit-verify-dryrun/4085/console]), unrelated
failures occurred. Specifically, some tests under ubuntu-16.04-dockerised-tests
failed
([https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/90/console]). In
the following the failed test cases are given.
*19:33:57* [gw7] xfail
webserver/test_web_pages.py::TestWebPage::test_backend_instances
*19:33:57* webserver/test_web_pages.py::TestWebPage::test_io_mgr_threads
*19:43:36* [gw11] xfail
webserver/test_web_pages.py::TestWebPage::test_backend_instances_mt_dop
*19:43:37* [gw8] xfail
webserver/test_web_pages.py::TestWebPage::test_backend_states
*19:47:28* XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: \{'batch_size': 1, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
avro/snap/block]
*19:47:28* reason: Expected results differ across file formats
*19:47:28* XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: \{'batch_size': 0, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
avro/snap/block]
*19:47:28* reason: Expected results differ across file formats
*19:47:28* XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: \{'batch_size': 1, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': True, 'abort_on_error':
1, 'exec_single_node_rows_threshold': 0} | table_format: rc/snap/block]
*19:47:28* reason: Expected results differ across file formats
*19:47:28* XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: \{'batch_size': 1, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
orc/def/block]
*19:47:28* reason: Expected results differ across file formats
*19:47:28* XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: \{'batch_size': 0, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': True, 'abort_on_error':
1, 'exec_single_node_rows_threshold': 0} | table_format: kudu/none]
*19:47:28* reason: Expected results differ across file formats
*19:47:28* XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: \{'batch_size': 0, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': True, 'abort_on_error':
1, 'exec_single_node_rows_threshold': 0} | table_format: avro/snap/block]
*19:47:28* reason: Expected results differ across file formats
*19:47:28* XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: \{'batch_size': 1, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': True, 'abort_on_error':
1, 'exec_single_node_rows_threshold': 0} | table_format: kudu/none]
*19:47:28* reason: Expected results differ across file formats
*19:47:28* XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: \{'batch_size': 1, 'num_nodes': 0,
[jira] [Updated] (IMPALA-8466) Unrelated failures occurred after sending a patch set to Jenkins for dry-run tests
[
https://issues.apache.org/jira/browse/IMPALA-8466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fang-Yu Rao updated IMPALA-8466:
Description:
After sending my patch set to Jenkins for dry-run tests
(https://jenkins.impala.io/job/gerrit-verify-dryrun/4085/console), unrelated
failures occurred. Specifically, some tests under ubuntu-16.04-dockerised-tests
failed
(https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/90/console). In
the following the failed test cases are given.
19:33:57 [gw7] xfail
webserver/test_web_pages.py::TestWebPage::test_backend_instances
19:33:57 webserver/test_web_pages.py::TestWebPage::test_io_mgr_threads
19:43:36 [gw11] xfail
webserver/test_web_pages.py::TestWebPage::test_backend_instances_mt_dop
19:43:37 [gw8] xfail
webserver/test_web_pages.py::TestWebPage::test_backend_states
19:47:28 XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: {'batch_size': 1, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
avro/snap/block]
19:47:28 reason: Expected results differ across file formats
19:47:28 XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
avro/snap/block]
19:47:28 reason: Expected results differ across file formats
19:47:28 XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: {'batch_size': 1, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': True, 'abort_on_error':
1, 'exec_single_node_rows_threshold': 0} | table_format: rc/snap/block]
19:47:28 reason: Expected results differ across file formats
19:47:28 XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: {'batch_size': 1, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
orc/def/block]
19:47:28 reason: Expected results differ across file formats
19:47:28 XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': True, 'abort_on_error':
1, 'exec_single_node_rows_threshold': 0} | table_format: kudu/none]
19:47:28 reason: Expected results differ across file formats
19:47:28 XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': True, 'abort_on_error':
1, 'exec_single_node_rows_threshold': 0} | table_format: avro/snap/block]
19:47:28 reason: Expected results differ across file formats
19:47:28 XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: {'batch_size': 1, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': True, 'abort_on_error':
1, 'exec_single_node_rows_threshold': 0} | table_format: kudu/none]
19:47:28 reason: Expected results differ across file formats
19:47:28 XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: {'batch_size': 1, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
seq/snap/block]
19:47:28 reason: Expected results differ across file formats
19:47:28 XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: {'batch_size': 1, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': True, 'abort_on_error':
1, 'exec_single_node_rows_threshold': 0} | table_format: avro/snap/block]
19:47:28 reason: Expected results differ across file formats
19:47:28 XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: {'batch_size': 1, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
kudu/none]
19:47:28 reason: Expected results differ across file formats
19:47:28 XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: {'batch_size': 1, 'num_nodes': 0,
'disable_codegen_rows_threshold':
[jira] [Updated] (IMPALA-8466) Unrelated failures occurred after sending a patch set to Jenkins for dry-run tests
[
https://issues.apache.org/jira/browse/IMPALA-8466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fang-Yu Rao updated IMPALA-8466:
Description:
After sending my patch set to Jenkins for dry-run tests
([https://jenkins.impala.io/job/gerrit-verify-dryrun/4085/console]), unrelated
failures occurred. Specifically, some tests under ubuntu-16.04-dockerised-tests
failed
([https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/90/console]). In
the following the failed test cases are given.
*19:33:57* [gw7] xfail
webserver/test_web_pages.py::TestWebPage::test_backend_instances
*19:33:57* webserver/test_web_pages.py::TestWebPage::test_io_mgr_threads
*19:43:36* [gw11] xfail
webserver/test_web_pages.py::TestWebPage::test_backend_instances_mt_dop
was:
After sending my patch set to Jenkins for dry-run tests
([https://jenkins.impala.io/job/gerrit-verify-dryrun/4085/console]), unrelated
failures occurred. Specifically, some tests under ubuntu-16.04-dockerised-tests
failed
([https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/90/console]). In
the following the failed test cases are given.
*19:33:57* [gw7] xfail
webserver/test_web_pages.py::TestWebPage::test_backend_instances *19:33:57*
webserver/test_web_pages.py::TestWebPage::test_io_mgr_threads
*19:43:36* [gw11] xfail
webserver/test_web_pages.py::TestWebPage::test_backend_instances_mt_dop
*19:43:37* [gw8] xfail
webserver/test_web_pages.py::TestWebPage::test_backend_states
*19:47:28* XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: \{'batch_size': 1, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
avro/snap/block]*19:47:28* reason: Expected results differ across file
formats*19:47:28* XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: \{'batch_size': 0, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
avro/snap/block]*19:47:28* reason: Expected results differ across file
formats*19:47:28* XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: \{'batch_size': 1, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': True, 'abort_on_error':
1, 'exec_single_node_rows_threshold': 0} | table_format:
rc/snap/block]*19:47:28* reason: Expected results differ across file
formats*19:47:28* XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: \{'batch_size': 1, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
orc/def/block]*19:47:28* reason: Expected results differ across file
formats*19:47:28* XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: \{'batch_size': 0, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': True, 'abort_on_error':
1, 'exec_single_node_rows_threshold': 0} | table_format: kudu/none]*19:47:28*
reason: Expected results differ across file formats*19:47:28* XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: \{'batch_size': 0, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': True, 'abort_on_error':
1, 'exec_single_node_rows_threshold': 0} | table_format:
avro/snap/block]*19:47:28* reason: Expected results differ across file
formats*19:47:28* XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: \{'batch_size': 1, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': True, 'abort_on_error':
1, 'exec_single_node_rows_threshold': 0} | table_format: kudu/none]*19:47:28*
reason: Expected results differ across file formats*19:47:28* XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: \{'batch_size': 1, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
seq/snap/block]*19:47:28* reason: Expected results differ across file
formats*19:47:28* XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: \{'batch_size': 1, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': True, 'abort_on_error':
1, 'exec_single_node_rows_threshold': 0} |
[jira] [Updated] (IMPALA-8466) Unrelated failures occurred after sending a patch set to Jenkins for dry-run tests
[
https://issues.apache.org/jira/browse/IMPALA-8466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fang-Yu Rao updated IMPALA-8466:
Description:
After sending my patch set to Jenkins for dry-run tests
([https://jenkins.impala.io/job/gerrit-verify-dryrun/4085/console]), unrelated
failures occurred. Specifically, some tests under ubuntu-16.04-dockerised-tests
failed
([https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/90/console]). In
the following the failed test cases are given.
*19:33:57* [gw7] xfail
webserver/test_web_pages.py::TestWebPage::test_backend_instances
*19:33:57* webserver/test_web_pages.py::TestWebPage::test_io_mgr_threads
*19:43:36* [gw11] xfail
webserver/test_web_pages.py::TestWebPage::test_backend_instances_mt_dop
*19:43:37* [gw8] xfail
webserver/test_web_pages.py::TestWebPage::test_backend_states
*19:47:28* XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: \{'batch_size': 1, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
avro/snap/block]
*19:47:28* reason: Expected results differ across file formats
*19:47:28* XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: \{'batch_size': 0, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
avro/snap/block]
*19:47:28* reason: Expected results differ across file formats
*19:47:28* XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: \{'batch_size': 1, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': True, 'abort_on_error':
1, 'exec_single_node_rows_threshold': 0} | table_format: rc/snap/block]
*19:47:28* reason: Expected results differ across file formats
*19:47:28* XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: \{'batch_size': 1, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
orc/def/block]
*19:47:28* reason: Expected results differ across file formats
*19:47:28* XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: \{'batch_size': 0, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': True, 'abort_on_error':
1, 'exec_single_node_rows_threshold': 0} | table_format: kudu/none]
*19:47:28* reason: Expected results differ across file formats
*19:47:28* XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: \{'batch_size': 0, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': True, 'abort_on_error':
1, 'exec_single_node_rows_threshold': 0} | table_format: avro/snap/block]
*19:47:28* reason: Expected results differ across file formats
*19:47:28* XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: \{'batch_size': 1, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': True, 'abort_on_error':
1, 'exec_single_node_rows_threshold': 0} | table_format: kudu/none]
*19:47:28* reason: Expected results differ across file formats
*19:47:28* XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: \{'batch_size': 1, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
seq/snap/block]
*19:47:28* reason: Expected results differ across file formats
*19:47:28* XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: \{'batch_size': 1, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': True, 'abort_on_error':
1, 'exec_single_node_rows_threshold': 0} | table_format: avro/snap/block]
*19:47:28* reason: Expected results differ across file formats
*19:47:28* XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: \{'batch_size': 1, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
kudu/none]
*19:47:28* reason: Expected results differ across file formats
*19:47:28* XFAIL
[jira] [Updated] (IMPALA-8466) Unrelated failures occurred after sending a patch set to Jenkins for dry-run tests
[
https://issues.apache.org/jira/browse/IMPALA-8466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fang-Yu Rao updated IMPALA-8466:
Description:
* After sending my patch set to Jenkins for dry-run tests
([https://jenkins.impala.io/job/gerrit-verify-dryrun/4085/console]), unrelated
failures occurred. Specifically, some tests under ubuntu-16.04-dockerised-tests
failed
([https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/90/console]). In
the following the failed test cases are given.
* *19:33:57* [gw7] xfail
webserver/test_web_pages.py::TestWebPage::test_backend_instances *19:33:57*
webserver/test_web_pages.py::TestWebPage::test_io_mgr_threads
* *19:43:36* [gw11] xfail
webserver/test_web_pages.py::TestWebPage::test_backend_instances_mt_dop
*19:43:37* [gw8] xfail
webserver/test_web_pages.py::TestWebPage::test_backend_states
* *19:47:28* XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: \{'batch_size': 1, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
avro/snap/block] *19:47:28* reason: Expected results differ across file
formats *19:47:28* XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: \{'batch_size': 0, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
avro/snap/block] *19:47:28* reason: Expected results differ across file
formats *19:47:28* XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: \{'batch_size': 1, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': True, 'abort_on_error':
1, 'exec_single_node_rows_threshold': 0} | table_format: rc/snap/block]
*19:47:28* reason: Expected results differ across file formats *19:47:28*
XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: \{'batch_size': 1, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
orc/def/block] *19:47:28* reason: Expected results differ across file formats
*19:47:28* XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: \{'batch_size': 0, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': True, 'abort_on_error':
1, 'exec_single_node_rows_threshold': 0} | table_format: kudu/none] *19:47:28*
reason: Expected results differ across file formats *19:47:28* XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: \{'batch_size': 0, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': True, 'abort_on_error':
1, 'exec_single_node_rows_threshold': 0} | table_format: avro/snap/block]
*19:47:28* reason: Expected results differ across file formats *19:47:28*
XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: \{'batch_size': 1, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': True, 'abort_on_error':
1, 'exec_single_node_rows_threshold': 0} | table_format: kudu/none] *19:47:28*
reason: Expected results differ across file formats *19:47:28* XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: \{'batch_size': 1, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
seq/snap/block] *19:47:28* reason: Expected results differ across file formats
*19:47:28* XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: \{'batch_size': 1, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': True, 'abort_on_error':
1, 'exec_single_node_rows_threshold': 0} | table_format: avro/snap/block]
*19:47:28* reason: Expected results differ across file formats *19:47:28*
XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: \{'batch_size': 1, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
kudu/none] *19:47:28* reason: Expected results differ across file formats
*19:47:28* XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
[jira] [Updated] (IMPALA-8466) Unrelated failures occurred after sending a patch set to Jenkins for dry-run tests
[
https://issues.apache.org/jira/browse/IMPALA-8466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fang-Yu Rao updated IMPALA-8466:
Description:
After sending my patch set to Jenkins for dry-run tests
([https://jenkins.impala.io/job/gerrit-verify-dryrun/4085/console]), unrelated
failures occurred. Specifically, some tests under ubuntu-16.04-dockerised-tests
failed
([https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/90/console]). In
the following the failed test cases are given.
*19:33:57* [gw7] xfail
webserver/test_web_pages.py::TestWebPage::test_backend_instances *19:33:57*
webserver/test_web_pages.py::TestWebPage::test_io_mgr_threads
*19:43:36* [gw11] xfail
webserver/test_web_pages.py::TestWebPage::test_backend_instances_mt_dop
*19:43:37* [gw8] xfail
webserver/test_web_pages.py::TestWebPage::test_backend_states
*19:47:28* XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: \{'batch_size': 1, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
avro/snap/block]*19:47:28* reason: Expected results differ across file
formats*19:47:28* XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: \{'batch_size': 0, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
avro/snap/block]*19:47:28* reason: Expected results differ across file
formats*19:47:28* XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: \{'batch_size': 1, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': True, 'abort_on_error':
1, 'exec_single_node_rows_threshold': 0} | table_format:
rc/snap/block]*19:47:28* reason: Expected results differ across file
formats*19:47:28* XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: \{'batch_size': 1, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
orc/def/block]*19:47:28* reason: Expected results differ across file
formats*19:47:28* XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: \{'batch_size': 0, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': True, 'abort_on_error':
1, 'exec_single_node_rows_threshold': 0} | table_format: kudu/none]*19:47:28*
reason: Expected results differ across file formats*19:47:28* XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: \{'batch_size': 0, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': True, 'abort_on_error':
1, 'exec_single_node_rows_threshold': 0} | table_format:
avro/snap/block]*19:47:28* reason: Expected results differ across file
formats*19:47:28* XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: \{'batch_size': 1, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': True, 'abort_on_error':
1, 'exec_single_node_rows_threshold': 0} | table_format: kudu/none]*19:47:28*
reason: Expected results differ across file formats*19:47:28* XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: \{'batch_size': 1, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
seq/snap/block]*19:47:28* reason: Expected results differ across file
formats*19:47:28* XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: \{'batch_size': 1, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': True, 'abort_on_error':
1, 'exec_single_node_rows_threshold': 0} | table_format:
avro/snap/block]*19:47:28* reason: Expected results differ across file
formats*19:47:28* XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: \{'batch_size': 1, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
kudu/none]*19:47:28* reason: Expected results differ across file
formats*19:47:28* XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option:
[jira] [Updated] (IMPALA-8466) Unrelated failures occurred after sending a patch set to Jenkins for dry-run tests
[
https://issues.apache.org/jira/browse/IMPALA-8466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fang-Yu Rao updated IMPALA-8466:
Description:
After sending my patch set to Jenkins for dry-run tests
([https://jenkins.impala.io/job/gerrit-verify-dryrun/4085/console]), unrelated
failures occurred. Specifically, some tests under ubuntu-16.04-dockerised-tests
failed
([https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/90/console]). In
the following the messages related to the failed test cases are given.
*19:33:57* [gw7] xfail
webserver/test_web_pages.py::TestWebPage::test_backend_instances *19:33:57*
webserver/test_web_pages.py::TestWebPage::test_io_mgr_threads
*19:43:36* [gw11] xfail
webserver/test_web_pages.py::TestWebPage::test_backend_instances_mt_dop
*19:43:37* [gw8] xfail
webserver/test_web_pages.py::TestWebPage::test_backend_states
*19:47:28* XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: \{'batch_size': 1, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
avro/snap/block]*19:47:28* reason: Expected results differ across file
formats*19:47:28* XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: \{'batch_size': 0, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
avro/snap/block]*19:47:28* reason: Expected results differ across file
formats*19:47:28* XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: \{'batch_size': 1, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': True, 'abort_on_error':
1, 'exec_single_node_rows_threshold': 0} | table_format:
rc/snap/block]*19:47:28* reason: Expected results differ across file
formats*19:47:28* XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: \{'batch_size': 1, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
orc/def/block]*19:47:28* reason: Expected results differ across file
formats*19:47:28* XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: \{'batch_size': 0, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': True, 'abort_on_error':
1, 'exec_single_node_rows_threshold': 0} | table_format: kudu/none]*19:47:28*
reason: Expected results differ across file formats*19:47:28* XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: \{'batch_size': 0, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': True, 'abort_on_error':
1, 'exec_single_node_rows_threshold': 0} | table_format:
avro/snap/block]*19:47:28* reason: Expected results differ across file
formats*19:47:28* XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: \{'batch_size': 1, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': True, 'abort_on_error':
1, 'exec_single_node_rows_threshold': 0} | table_format: kudu/none]*19:47:28*
reason: Expected results differ across file formats*19:47:28* XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: \{'batch_size': 1, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
seq/snap/block]*19:47:28* reason: Expected results differ across file
formats*19:47:28* XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: \{'batch_size': 1, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': True, 'abort_on_error':
1, 'exec_single_node_rows_threshold': 0} | table_format:
avro/snap/block]*19:47:28* reason: Expected results differ across file
formats*19:47:28* XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
beeswax | exec_option: \{'batch_size': 1, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
kudu/none]*19:47:28* reason: Expected results differ across file
formats*19:47:28* XFAIL
data_errors/test_data_errors.py::TestHdfsScanNodeErrors::()::test_hdfs_scan_node_errors[protocol:
[jira] [Commented] (IMPALA-8381) Remove branch from ParquetPlainEncoder::Decode()
[ https://issues.apache.org/jira/browse/IMPALA-8381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16827373#comment-16827373 ] ASF subversion and git services commented on IMPALA-8381: - Commit 6a703741d8fdc359833a0d593ca8b121cd5d890d in impala's branch refs/heads/master from Daniel Becker [ https://gitbox.apache.org/repos/asf?p=impala.git;h=6a70374 ] IMPALA-8381: Optimize ParquetPlainEncoder::DecodeBatch() for simple types Refactored the ParquetPlainEncoder::Decode() and ParquetPlainEncoder::DecodeBatch() methods to increase performance in batch decoding. The `Decode` and `DecodeBatch` methods retain their behaviour and outward interface, but the internal structure changes. We change how we split up the `Decode` template specialisations. The generic unspecialised template is used for numerical parquet types (INT32, INT64, INT96, FLOAT and DOUBLE) and various specialisations are used for BYTE_ARRAY and FIXED_LEN_BYTE_ARRAY. We add a new method template, DecodeNoCheck, which does the actual decoding without bounds checking. It is called by the generic Decode method template internally. For all parquet types except for BYTE_ARRAY, DecodeBatch performs the bounds check once for the whole batch at the same time and calls DecodeNoCheck, so we save the cost of bounds checking for every decoded value. For BYTE_ARRAY, this cannot be done and we have to perform the checks for every value. In the non-BYTE_ARRAY version of DecodeBatch, we explicitly unroll the loop in batches of 8 to increase performance. The overall performance increase is up to 2x for small strides (8 bytes, INT32) but decreases as the stride increases, and disappears from around 40 bytes. With bigger strides, there is no performance difference from the previous implementation. Testing: Added tests to parquet-plain-test.cc to test the `Decode` and the `DecodeBatch` methods both in single-value decoding and batch decoding. Change-Id: I57b7d2573bb6dfd038e581acb3bd8ea1565aa20d Reviewed-on: http://gerrit.cloudera.org:8080/12985 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins > Remove branch from ParquetPlainEncoder::Decode() > > > Key: IMPALA-8381 > URL: https://issues.apache.org/jira/browse/IMPALA-8381 > Project: IMPALA > Issue Type: Improvement > Components: Backend >Reporter: Csaba Ringhofer >Assignee: Daniel Becker >Priority: Minor > Labels: newbie, parquet, performance, ramp-up > > Removing the "if" at > https://github.com/apache/impala/blob/5670f96b828d57f9e36510bb9af02bcc31de775c/be/src/exec/parquet/parquet-common.h#L203 > can lead to 1.5x speed up in plain decoding (type=int32, stride=16). For > primitive types, the same check can be done for a whole batch, so the speedup > can be gained for large batches without loosing safety. The only Parquet type > where this check is needed per element is BYTE_ARRAY (typically used for > STRING columns), which already has a template specialization for > ParquetPlainEncoder::Decode(). -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-8444) Analysis perf regression after IMPALA-7616
[
https://issues.apache.org/jira/browse/IMPALA-8444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16827374#comment-16827374
]
ASF subversion and git services commented on IMPALA-8444:
-
Commit 349142635250f92412003a1e62829fc4b441dc75 in impala's branch
refs/heads/master from Fredy Wijaya
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=3491426 ]
IMPALA-8444: Fix performance regression when building privilege name
This patch fixes the performance regression when building privilege name
by rewriting PrincipalPrivilege.buildPrivilegeName() with a simple
string concatentation instead of using a list that gets converted into a
string.
Below is the result of running a benchmark using JMH comparing the old
and new implementations:
Result "org.apache.impala.BuildPrivilegeNameBenchmark.fast":
0.344 ±(99.9%) 0.004 us/op [Average]
(min, avg, max) = (0.336, 0.344, 0.355), stdev = 0.005
CI (99.9%): [0.339, 0.348] (assumes normal distribution)
Result "org.apache.impala.BuildPrivilegeNameBenchmark.slow":
0.831 ±(99.9%) 0.011 us/op [Average]
(min, avg, max) = (0.807, 0.831, 0.856), stdev = 0.015
CI (99.9%): [0.820, 0.842] (assumes normal distribution)
Benchmark Mode Cnt Score Error Units
BuildPrivilegeNameBenchmark.fast avgt 25 0.344 ± 0.004 us/op
BuildPrivilegeNameBenchmark.slow avgt 25 0.831 ± 0.011 us/op
This patch also updates SentryAuthorizationPolicy.listPrivileges() to
reuse the privilege names that have already been built instead of
building them again. While fixing this, I found a bug where Principal
stores the PrincipalPrivilege in a case insensitive way. This is true
for all privilege scopes, except URI. This patch fixes the issue by
making privilege name to be case sensitive instead.
This patch removes incorrect synchronization in
SentryAuthorizationPolicy.listPrivileges() that can cause the operation
to run in serial in a highly concurrent workload.
Testing:
- Ran all FE tests
- Ran all E2E authorization tests
- Added E2E test for privilege name case sensitivity bug
Change-Id: I942d9b55f07c8972f69e532567d9b7d80fceb6e5
Reviewed-on: http://gerrit.cloudera.org:8080/13095
Reviewed-by: Impala Public Jenkins
Tested-by: Impala Public Jenkins
> Analysis perf regression after IMPALA-7616
> --
>
> Key: IMPALA-8444
> URL: https://issues.apache.org/jira/browse/IMPALA-8444
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
>Affects Versions: Impala 3.2.0
>Reporter: bharath v
>Assignee: Fredy Wijaya
>Priority: Critical
> Fix For: Impala 3.3.0
>
>
> The patch for IMPALA-7616 caused a performance regression in analysis time
> when run in an environment with ~1k roles and 10.5k. privileges. The
> regression is evident when run as a role that has a large number of
> privileges.
> Following is the stack to look for when jstacking the coordnator.
> {noformat}
> "Thread-21" #49 prio=5 os_prio=0 tid=0x0cd6e000 nid=0x6a3d runnable
> [0x7fa28e4a]
>java.lang.Thread.State: RUNNABLE
> at java.lang.String.toLowerCase(String.java:2670)
> at
> org.apache.impala.catalog.PrincipalPrivilege.buildPrivilegeName(PrincipalPrivilege.java:82)
> at
> org.apache.impala.catalog.PrincipalPrivilege.getName(PrincipalPrivilege.java:143)
> at
> org.apache.impala.catalog.AuthorizationPolicy.listPrivileges(AuthorizationPolicy.java:423)
> - locked <0x7fa376987100> (a
> org.apache.impala.catalog.AuthorizationPolicy)
> at
> org.apache.impala.catalog.AuthorizationPolicy.listPrivileges(AuthorizationPolicy.java:443)
> - locked <0x7fa376987100> (a
> org.apache.impala.catalog.AuthorizationPolicy)
> at
> org.apache.sentry.provider.cache.SimpleCacheProviderBackend.getPrivileges(SimpleCacheProviderBackend.java:75)
> at
> org.apache.sentry.policy.db.SimpleDBPolicyEngine.getPrivileges(SimpleDBPolicyEngine.java:98)
> at
> org.apache.sentry.provider.common.ResourceAuthorizationProvider.getPrivileges(ResourceAuthorizationProvider.java:147)
> at
> org.apache.sentry.provider.common.ResourceAuthorizationProvider.doHasAccess(ResourceAuthorizationProvider.java:120)
> at
> org.apache.sentry.provider.common.ResourceAuthorizationProvider.hasAccess(ResourceAuthorizationProvider.java:107)
> at
> org.apache.impala.authorization.AuthorizationChecker.hasAccess(AuthorizationChecker.java:215)
> at
> org.apache.impala.authorization.AuthorizationChecker.checkAccess(AuthorizationChecker.java:128)
> at
> org.apache.impala.analysis.AnalysisContext.authorizePrivilegeRequest(AnalysisContext.java:592)
> at
> org.apache.impala.analysis.AnalysisContext.authorize(AnalysisContext.java:564)
> at
>
[jira] [Commented] (IMPALA-8369) Impala should be able to interoperate with Hive 3.1.0
[
https://issues.apache.org/jira/browse/IMPALA-8369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16827368#comment-16827368
]
Vihang Karajgaonkar commented on IMPALA-8369:
-
No, {{HiveServer2}} still remains HiveServer2 in Hive-3
> Impala should be able to interoperate with Hive 3.1.0
> -
>
> Key: IMPALA-8369
> URL: https://issues.apache.org/jira/browse/IMPALA-8369
> Project: IMPALA
> Issue Type: Improvement
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Major
> Labels: impala-acid
>
> Currently, Impala only works with Hive 2.1.1. Since Hive 3.1.0 has been
> released for a while it would be good to add support for Hive 3.1.0 (HMS
> 3.1.0). This patch will focus on ability to connect to HMS 3.1.0 and run
> existing tests. It will not focus on adding support for newer features like
> ACID in Hive 3.1.0 which can be taken up as separate JIRA.
> It would be good to make changes to Impala source code such that it can work
> with both Hive 2.1.0 and Hive 3.1.0 without the need to create a separate
> branch. However, this should be a aspirational goal. If we hit a blocker we
> should investigate alternative approaches.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-8466) Unrelated failures occurred after sending a patch set to Jenkins for dry-run tests
Fang-Yu Rao created IMPALA-8466: --- Summary: Unrelated failures occurred after sending a patch set to Jenkins for dry-run tests Key: IMPALA-8466 URL: https://issues.apache.org/jira/browse/IMPALA-8466 Project: IMPALA Issue Type: Bug Components: Catalog, Infrastructure Reporter: Fang-Yu Rao Assignee: Tim Armstrong After sending my patch set to Jenkins for dry-run tests, unrelated failures occurred. The messages could be found at [https://jenkins.impala.io/job/gerrit-verify-dryrun/4085/console]. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-8446) Create a unit test for Admission Controller
[ https://issues.apache.org/jira/browse/IMPALA-8446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman resolved IMPALA-8446. Resolution: Fixed Fix Version/s: Impala 2.13.0 > Create a unit test for Admission Controller > --- > > Key: IMPALA-8446 > URL: https://issues.apache.org/jira/browse/IMPALA-8446 > Project: IMPALA > Issue Type: Bug >Reporter: Andrew Sherman >Assignee: Andrew Sherman >Priority: Major > Fix For: Impala 2.13.0 > > > This will allow construction of white box tests that exercise Admission > Controller code. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-8444) Analysis perf regression after IMPALA-7616
[
https://issues.apache.org/jira/browse/IMPALA-8444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fredy Wijaya resolved IMPALA-8444.
--
Resolution: Fixed
Fix Version/s: Impala 3.3.0
> Analysis perf regression after IMPALA-7616
> --
>
> Key: IMPALA-8444
> URL: https://issues.apache.org/jira/browse/IMPALA-8444
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
>Affects Versions: Impala 3.2.0
>Reporter: bharath v
>Assignee: Fredy Wijaya
>Priority: Critical
> Fix For: Impala 3.3.0
>
>
> The patch for IMPALA-7616 caused a performance regression in analysis time
> when run in an environment with ~1k roles and 10.5k. privileges. The
> regression is evident when run as a role that has a large number of
> privileges.
> Following is the stack to look for when jstacking the coordnator.
> {noformat}
> "Thread-21" #49 prio=5 os_prio=0 tid=0x0cd6e000 nid=0x6a3d runnable
> [0x7fa28e4a]
>java.lang.Thread.State: RUNNABLE
> at java.lang.String.toLowerCase(String.java:2670)
> at
> org.apache.impala.catalog.PrincipalPrivilege.buildPrivilegeName(PrincipalPrivilege.java:82)
> at
> org.apache.impala.catalog.PrincipalPrivilege.getName(PrincipalPrivilege.java:143)
> at
> org.apache.impala.catalog.AuthorizationPolicy.listPrivileges(AuthorizationPolicy.java:423)
> - locked <0x7fa376987100> (a
> org.apache.impala.catalog.AuthorizationPolicy)
> at
> org.apache.impala.catalog.AuthorizationPolicy.listPrivileges(AuthorizationPolicy.java:443)
> - locked <0x7fa376987100> (a
> org.apache.impala.catalog.AuthorizationPolicy)
> at
> org.apache.sentry.provider.cache.SimpleCacheProviderBackend.getPrivileges(SimpleCacheProviderBackend.java:75)
> at
> org.apache.sentry.policy.db.SimpleDBPolicyEngine.getPrivileges(SimpleDBPolicyEngine.java:98)
> at
> org.apache.sentry.provider.common.ResourceAuthorizationProvider.getPrivileges(ResourceAuthorizationProvider.java:147)
> at
> org.apache.sentry.provider.common.ResourceAuthorizationProvider.doHasAccess(ResourceAuthorizationProvider.java:120)
> at
> org.apache.sentry.provider.common.ResourceAuthorizationProvider.hasAccess(ResourceAuthorizationProvider.java:107)
> at
> org.apache.impala.authorization.AuthorizationChecker.hasAccess(AuthorizationChecker.java:215)
> at
> org.apache.impala.authorization.AuthorizationChecker.checkAccess(AuthorizationChecker.java:128)
> at
> org.apache.impala.analysis.AnalysisContext.authorizePrivilegeRequest(AnalysisContext.java:592)
> at
> org.apache.impala.analysis.AnalysisContext.authorize(AnalysisContext.java:564)
> at
> org.apache.impala.analysis.AnalysisContext.analyzeAndAuthorize(AnalysisContext.java:415)
> at
> org.apache.impala.service.Frontend.doCreateExecRequest(Frontend.java:1240)
> at
> org.apache.impala.service.Frontend.getTExecRequest(Frontend.java:1210)
> at
> org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1182)
> at
> org.apache.impala.service.JniFrontend.createExecRequest(JniFrontend.java:158)
> {noformat}
> Issue worsens when running concurrent workloads, because the underlying
> Sentry {{listPrivileges()}} is synchronized and that serializes all the query
> analysis requests.
> {noformat}
> public synchronized Set listPrivileges(Set groups, <
> ActiveRoleSet roleSet) {
> Set privileges = Sets.newHashSet();
> if (roleSet != ActiveRoleSet.ALL) {
> throw new UnsupportedOperationException("Impala does not support role
> subsets.");
> }
> {noformat}
> Notes:
> - If the authorization metadata footprint is small, this issue can be ignored.
> - One workaround is to run the query using a role that has very small number
> of privileges (revoke privileges that are not necessary to run a given query).
> - Another workaround is to disable Sentry authorization.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Work started] (IMPALA-6590) Disable expr rewrites and codegen for VALUES() statements
[
https://issues.apache.org/jira/browse/IMPALA-6590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on IMPALA-6590 started by Abhishek Rawat.
--
> Disable expr rewrites and codegen for VALUES() statements
> -
>
> Key: IMPALA-6590
> URL: https://issues.apache.org/jira/browse/IMPALA-6590
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
>Affects Versions: Impala 2.8.0, Impala 2.9.0, Impala 2.10.0, Impala 2.11.0
>Reporter: Alexander Behm
>Assignee: Abhishek Rawat
>Priority: Major
> Labels: perf, planner, ramp-up, regression
>
> The analysis of statements with big VALUES clauses like INSERT INTO
> VALUES is slow due to expression rewrites like constant folding. The
> performance of such statements has regressed since the introduction of expr
> rewrites and constant folding in IMPALA-1788.
> We should skip expr rewrites for VALUES altogether since it mostly provides
> no benefit but can have a large overhead due to evaluation of expressions in
> the backend (constant folding). These expressions are ultimately evaluated
> and materialized in the backend anyway, so there's no point in folding them
> during analysis.
> Similarly, there is no point in doing codegen for these exprs in the backend
> union node.
> *Workaround*
> {code}
> SET ENABLE_EXPR_REWRITES=FALSE;
> SET DISABLE_CODEGEN=TRUE;
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-6088) Rack aware broadcast operator
[
https://issues.apache.org/jira/browse/IMPALA-6088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16827317#comment-16827317
]
Ruslan Dautkhanov commented on IMPALA-6088:
---
Would be also interesting to look if something like Torrent protocol helps here
too.
This Torrent protocol can take into account rack location, if available.
Spark uses this to avoid -query coordinator- Spark Driver to be a network
bottleneck
https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/broadcast/TorrentBroadcast.scala
{quote}
* The driver divides the serialized object into small chunks and
* stores those chunks in the BlockManager of the driver.
* On each executor, the executor first attempts to fetch the object from its
BlockManager. If
* it does not exist, it then uses remote fetches to fetch the small chunks
from the driver and/or
* other executors if available. Once it gets the chunks, it puts the chunks in
its own
* BlockManager, ready for other executors to fetch from.
* This prevents the driver from being the bottleneck in sending out multiple
copies of the
* broadcast data (one per executor).
{quote}
> Rack aware broadcast operator
> -
>
> Key: IMPALA-6088
> URL: https://issues.apache.org/jira/browse/IMPALA-6088
> Project: IMPALA
> Issue Type: Sub-task
> Components: Distributed Exec
>Reporter: Mostafa Mokhtar
>Priority: Major
>
> When conducting large scale experiments on a 6 rack cluster with aggregator
> core network topology overall cluster bandwidth utilization was limited.
> With aggregator core networks nodes and racks are not equidistant, which
> means a broadcast operation can be inefficient as the broadcasting node needs
> to send the same data N times to each node on a remote rack.
> Ideally Rowbatches should be sent once per remote rack then a node on each
> remote rack would broadcast within its rack.
> Table below represent rack to rack latency for the 90% of operations, ration
> between best and worst case is 7.3x
> | || va||vc||vd1|| vd3|| ve|
> |va| 4,238| 4,290| 9,692| 8,897| 8,208|
> |vc| 9,290| 4,396| 30,952| 13,529| 14,578|
> |vd1| 9,131| 29,066| 4,346| 17,265| 16,849|
> |vd3| 7,409| 15,517| 17,265| 4,370| 4,687|
> |ve| 4,914| 16,894| 16,430| 4,713| 4,472|
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Assigned] (IMPALA-8465) hs2.test_json_endpoints.TestJsonEndpoints fails on deployed clusters
[
https://issues.apache.org/jira/browse/IMPALA-8465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Armstrong reassigned IMPALA-8465:
-
Assignee: Tim Armstrong
> hs2.test_json_endpoints.TestJsonEndpoints fails on deployed clusters
>
>
> Key: IMPALA-8465
> URL: https://issues.apache.org/jira/browse/IMPALA-8465
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
>Affects Versions: Impala 3.3.0
>Reporter: David Knupp
>Assignee: Tim Armstrong
>Priority: Major
>
> Changes to tests.common.ImpalaCluster in [commit
> 2ca7f8e7|https://github.com/apache/impala/commit/2ca7f8e7c0781a1914275b3506cf8a7748c44c85#diff-6fea89ad0e6c440b0373bb136d7510b5]
> introduced a regression in this test.
> {noformat}
> hs2/test_json_endpoints.py:51: in test_waiting_in_flight_queries
> queries_json = self._get_json_queries(http_addr)
> hs2/test_json_endpoints.py:33: in _get_json_queries
> return cluster.impalads[0].service.get_debug_webpage_json("/queries")
> E IndexError: list index out of range
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-8465) hs2.test_json_endpoints.TestJsonEndpoints fails on deployed clusters
David Knupp created IMPALA-8465:
---
Summary: hs2.test_json_endpoints.TestJsonEndpoints fails on
deployed clusters
Key: IMPALA-8465
URL: https://issues.apache.org/jira/browse/IMPALA-8465
Project: IMPALA
Issue Type: Bug
Components: Infrastructure
Affects Versions: Impala 3.3.0
Reporter: David Knupp
Changes to tests.common.ImpalaCluster in [commit
2ca7f8e7|https://github.com/apache/impala/commit/2ca7f8e7c0781a1914275b3506cf8a7748c44c85#diff-6fea89ad0e6c440b0373bb136d7510b5]
introduced a regression in this test.
{noformat}
hs2/test_json_endpoints.py:51: in test_waiting_in_flight_queries
queries_json = self._get_json_queries(http_addr)
hs2/test_json_endpoints.py:33: in _get_json_queries
return cluster.impalads[0].service.get_debug_webpage_json("/queries")
E IndexError: list index out of range
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-8464) JdbcTest.testMetaDataGetColumnComments() fails with local catalog
[
https://issues.apache.org/jira/browse/IMPALA-8464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16827265#comment-16827265
]
Tim Armstrong commented on IMPALA-8464:
---
[~tlipcon] [~bharathv] any thoughts on what might be happening here?
> JdbcTest.testMetaDataGetColumnComments() fails with local catalog
> -
>
> Key: IMPALA-8464
> URL: https://issues.apache.org/jira/browse/IMPALA-8464
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
>Affects Versions: Impala 3.1.0, Impala 3.2.0
>Reporter: Tim Armstrong
>Assignee: Tim Armstrong
>Priority: Major
>
> {noformat}
> Error Message
> Incorrect table comment expected:<[]> but was:<[table comment]>
> Stacktrace
> org.junit.ComparisonFailure: Incorrect table comment expected:<[]> but
> was:<[table comment]>
> at org.junit.Assert.assertEquals(Assert.java:115)
> at
> org.apache.impala.service.JdbcTest.testMetaDataGetColumnComments(JdbcTest.java:484)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:272)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:236)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
> at
> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:386)
> at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:323)
> at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:143)
> Standard Output
> 19/04/26 18:35:36 INFO testutil.ImpalaJdbcClient: Using JDBC Driver Name:
> org.apache.hive.jdbc.HiveDriver
> 19/04/26 18:35:36 INFO testutil.ImpalaJdbcClient: Connecting to:
> jdbc:hive2://localhost:21050/default;auth=noSasl
> 19/04/26 18:35:36 INFO jdbc.Utils: Supplied authorities: localhost:21050
> 19/04/26 18:35:36 INFO jdbc.Utils: Resolved authority: localhost:21050
> 19/04/26 18:35:36 INFO testutil.ImpalaJdbcClient: Using JDBC Driver Name:
> org.apache.hive.jdbc.HiveDriver
> 19/04/26 18:35:36 INFO testutil.ImpalaJdbcClient: Connecting to:
> jdbc:hive2://localhost:21050/default;auth=noSasl
> 19/04/26 18:35:36 INFO jdbc.Utils: Supplied authorities: localhost:21050
> 19/04/26 18:35:36 INFO jdbc.Utils: Resolved authority: localhost:21050
> 19/04/26 18:35:36 INFO testutil.ImpalaJdbcClient: Using JDBC Driver Name:
> org.apache.hive.jdbc.HiveDriver
> 19/04/26 18:35:36 INFO testutil.ImpalaJdbcClient: Connecting to:
> jdbc:hive2://localhost:21050/default;auth=noSasl
> 19/04/26 18:35:36 INFO jdbc.Utils: Supplied authorities: localhost:21050
> 19/04/26 18:35:36 INFO jdbc.Utils: Resolved authority: localhost:21050
> 19/04/26 18:35:36 INFO testutil.ImpalaJdbcClient: Using JDBC Driver Name:
> org.apache.hive.jdbc.HiveDriver
> 19/04/26 18:35:36 INFO testutil.ImpalaJdbcClient: Connecting to:
> jdbc:hive2://localhost:21050/default;auth=noSasl
> 19/04/26 18:35:36 INFO jdbc.Utils: Supplied authorities:
[jira] [Commented] (IMPALA-8369) Impala should be able to interoperate with Hive 3.1.0
[ https://issues.apache.org/jira/browse/IMPALA-8369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16827259#comment-16827259 ] Alex Rodoni commented on IMPALA-8369: - [~vihangk1] [~tlipcon] When this is committed, should all occurrences of "HiveServer2" be changed to "HiveServer3" or just "HiveServer" in docs? > Impala should be able to interoperate with Hive 3.1.0 > - > > Key: IMPALA-8369 > URL: https://issues.apache.org/jira/browse/IMPALA-8369 > Project: IMPALA > Issue Type: Improvement >Reporter: Vihang Karajgaonkar >Assignee: Vihang Karajgaonkar >Priority: Major > Labels: impala-acid > > Currently, Impala only works with Hive 2.1.1. Since Hive 3.1.0 has been > released for a while it would be good to add support for Hive 3.1.0 (HMS > 3.1.0). This patch will focus on ability to connect to HMS 3.1.0 and run > existing tests. It will not focus on adding support for newer features like > ACID in Hive 3.1.0 which can be taken up as separate JIRA. > It would be good to make changes to Impala source code such that it can work > with both Hive 2.1.0 and Hive 3.1.0 without the need to create a separate > branch. However, this should be a aspirational goal. If we hit a blocker we > should investigate alternative approaches. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Work started] (IMPALA-8457) Fix python when Impala isn't fully built
[ https://issues.apache.org/jira/browse/IMPALA-8457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on IMPALA-8457 started by Thomas Tauber-Marshall. -- > Fix python when Impala isn't fully built > > > Key: IMPALA-8457 > URL: https://issues.apache.org/jira/browse/IMPALA-8457 > Project: IMPALA > Issue Type: Improvement > Components: Infrastructure >Affects Versions: Impala 3.3.0 >Reporter: Thomas Tauber-Marshall >Assignee: Thomas Tauber-Marshall >Priority: Critical > > A common pattern is to check out Impala, eg. in a Jenkins job, and then run > Impala's functional python tests. If the tests are being run against a real > cluster that has already been setup (as opposed to being run against the > mini-cluster), it would be nice to be able to run any of our python tests > without having to do a full build of Impala. > This usually works, but occasionally gets broken eg. due to people adding > imports of thrift structs to python files, since the thrift definitions > aren't available without a full build. We dealt with this once recently in > IMPALA-8199, and it has now cropped up again as a result of IMPALA-8158 > Going forward, I think a good solution would be to add a flag to buildall.sh, > say '-testonly', that ensures that everything needed to run the python tests > is done, eg. in this case something like 'make thrift-deps' should get run. > We could incorporate running this into our pre-commit job to ensure that it > doesn't get broken again in the future. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-8464) JdbcTest.testMetaDataGetColumnComments() fails with local catalog
Tim Armstrong created IMPALA-8464:
-
Summary: JdbcTest.testMetaDataGetColumnComments() fails with local
catalog
Key: IMPALA-8464
URL: https://issues.apache.org/jira/browse/IMPALA-8464
Project: IMPALA
Issue Type: Bug
Components: Catalog
Affects Versions: Impala 3.2.0, Impala 3.1.0
Reporter: Tim Armstrong
Assignee: Tim Armstrong
{noformat}
Error Message
Incorrect table comment expected:<[]> but was:<[table comment]>
Stacktrace
org.junit.ComparisonFailure: Incorrect table comment expected:<[]> but
was:<[table comment]>
at org.junit.Assert.assertEquals(Assert.java:115)
at
org.apache.impala.service.JdbcTest.testMetaDataGetColumnComments(JdbcTest.java:484)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at
org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:272)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:236)
at
org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
at
org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:386)
at
org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:323)
at
org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:143)
Standard Output
19/04/26 18:35:36 INFO testutil.ImpalaJdbcClient: Using JDBC Driver Name:
org.apache.hive.jdbc.HiveDriver
19/04/26 18:35:36 INFO testutil.ImpalaJdbcClient: Connecting to:
jdbc:hive2://localhost:21050/default;auth=noSasl
19/04/26 18:35:36 INFO jdbc.Utils: Supplied authorities: localhost:21050
19/04/26 18:35:36 INFO jdbc.Utils: Resolved authority: localhost:21050
19/04/26 18:35:36 INFO testutil.ImpalaJdbcClient: Using JDBC Driver Name:
org.apache.hive.jdbc.HiveDriver
19/04/26 18:35:36 INFO testutil.ImpalaJdbcClient: Connecting to:
jdbc:hive2://localhost:21050/default;auth=noSasl
19/04/26 18:35:36 INFO jdbc.Utils: Supplied authorities: localhost:21050
19/04/26 18:35:36 INFO jdbc.Utils: Resolved authority: localhost:21050
19/04/26 18:35:36 INFO testutil.ImpalaJdbcClient: Using JDBC Driver Name:
org.apache.hive.jdbc.HiveDriver
19/04/26 18:35:36 INFO testutil.ImpalaJdbcClient: Connecting to:
jdbc:hive2://localhost:21050/default;auth=noSasl
19/04/26 18:35:36 INFO jdbc.Utils: Supplied authorities: localhost:21050
19/04/26 18:35:36 INFO jdbc.Utils: Resolved authority: localhost:21050
19/04/26 18:35:36 INFO testutil.ImpalaJdbcClient: Using JDBC Driver Name:
org.apache.hive.jdbc.HiveDriver
19/04/26 18:35:36 INFO testutil.ImpalaJdbcClient: Connecting to:
jdbc:hive2://localhost:21050/default;auth=noSasl
19/04/26 18:35:36 INFO jdbc.Utils: Supplied authorities: localhost:21050
19/04/26 18:35:36 INFO jdbc.Utils: Resolved authority: localhost:21050
19/04/26 18:36:01 INFO service.FeSupport: Loading libfesupport.so
19/04/26 18:36:01 INFO service.FeSupport: Loaded libfesupport.so
19/04/26 18:36:01 INFO util.JvmPauseMonitor: Starting JVM pause monitor
{noformat}
Not sure what is going wrong here.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work started] (IMPALA-8463) skip.header.line.count ignored on local catalog
[
https://issues.apache.org/jira/browse/IMPALA-8463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on IMPALA-8463 started by Tim Armstrong.
-
> skip.header.line.count ignored on local catalog
> ---
>
> Key: IMPALA-8463
> URL: https://issues.apache.org/jira/browse/IMPALA-8463
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
>Affects Versions: Impala 3.1.0, Impala 3.2.0, Impala 3.3.0
>Reporter: Tim Armstrong
>Assignee: Tim Armstrong
>Priority: Critical
>
> test_text_scanner_with_header() fails when run against local catalog.
> {noformat}
> [tarmstrong-box2.ca.cloudera.com:21000] functional> select c1, c2 from
> table_with_header
> > ;
> Query: select c1, c2 from table_with_header
> Query submitted at: 2019-04-25 22:47:52 (Coordinator:
> http://2c766e5e2187:25000)
> Query progress can be monitored at:
> http://2c766e5e2187:25000/query_plan?query_id=68403d93ac914925:394ee6a1
> +--+--+
> | c1 | c2 |
> +--+--+
> | 5| 6|
> | 1| 2|
> | 3| 4|
> | NULL | NULL |
> +--+--+
> WARNINGS: Error converting column: 0 to INT
> Error converting column: 1 to DOUBLE
> Error parsing row: file:
> hdfs://172.19.0.1:20500/test-warehouse/table_with_header/table_with_header.csv,
> before offset: 22
> {noformat}
> This is suspicious in LocalFsTable.java
> {code}
> @Override
> public int parseSkipHeaderLineCount(StringBuilder error) {
> // TODO Auto-generated method stub
> return 0;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-8463) skip.header.line.count ignored on local catalog
[
https://issues.apache.org/jira/browse/IMPALA-8463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Armstrong updated IMPALA-8463:
--
Target Version: Impala 3.3.0
> skip.header.line.count ignored on local catalog
> ---
>
> Key: IMPALA-8463
> URL: https://issues.apache.org/jira/browse/IMPALA-8463
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
>Affects Versions: Impala 3.1.0, Impala 3.2.0, Impala 3.3.0
>Reporter: Tim Armstrong
>Assignee: Tim Armstrong
>Priority: Critical
>
> test_text_scanner_with_header() fails when run against local catalog.
> {noformat}
> [tarmstrong-box2.ca.cloudera.com:21000] functional> select c1, c2 from
> table_with_header
> > ;
> Query: select c1, c2 from table_with_header
> Query submitted at: 2019-04-25 22:47:52 (Coordinator:
> http://2c766e5e2187:25000)
> Query progress can be monitored at:
> http://2c766e5e2187:25000/query_plan?query_id=68403d93ac914925:394ee6a1
> +--+--+
> | c1 | c2 |
> +--+--+
> | 5| 6|
> | 1| 2|
> | 3| 4|
> | NULL | NULL |
> +--+--+
> WARNINGS: Error converting column: 0 to INT
> Error converting column: 1 to DOUBLE
> Error parsing row: file:
> hdfs://172.19.0.1:20500/test-warehouse/table_with_header/table_with_header.csv,
> before offset: 22
> {noformat}
> This is suspicious in LocalFsTable.java
> {code}
> @Override
> public int parseSkipHeaderLineCount(StringBuilder error) {
> // TODO Auto-generated method stub
> return 0;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-8406) Failed REFRESH can partially modify table without bumping version number
[ https://issues.apache.org/jira/browse/IMPALA-8406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16827133#comment-16827133 ] ASF subversion and git services commented on IMPALA-8406: - Commit 114712031f2293af5b3d9509776f88c23d3fa0fc in impala's branch refs/heads/master from Todd Lipcon [ https://gitbox.apache.org/repos/asf?p=impala.git;h=1147120 ] IMPALA-8454 (part 1): Refactor file descriptor loading code This refactors various file-descriptor loading code out of HdfsTable into new standalone classes. In order to support ACID tables, we'll need to make various changes to these bits of code, and having them extracted and cleaned up will make that easier. This consolidates all of the places in which we list partition directories into one method which does the appropriate thing regardless of situation. This has a small behavior change related to IMPALA-8406: previously, we had a bug where, while refreshing a table, if one or more partitions failed to refresh, the other partitions might still get refreshed despite an error being returned. Those other partitions wouldn't be available immediately until some other operation caused the table's catalog version number to increase. This was buggy behavior. Rather than tackle that problem in this "refactor" patch, this patch just slightly improves the behavior: we'll either atomically update or not update all partitions, but we might still add new partitions noticed by the REFRESH, and might still update other HMS metadata. This patch may end up slightly improving various other code paths that refresh file descriptor lists. We used to have slightly different ways of doing this in three different places, with different sets of optimizations. Now we do it all in one place, and we pull all the known tricks. Change-Id: I59edf493b9ba38be5f556b4795a7684d9c9e3a07 Reviewed-on: http://gerrit.cloudera.org:8080/12950 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins > Failed REFRESH can partially modify table without bumping version number > > > Key: IMPALA-8406 > URL: https://issues.apache.org/jira/browse/IMPALA-8406 > Project: IMPALA > Issue Type: Bug > Components: Catalog >Affects Versions: Impala 3.2.0 >Reporter: Todd Lipcon >Priority: Major > > Currently, various incremental operations in the catalogd modify Table > objects in place, including REFRESH, which modifies each partition. In this > case, if one partition fails to refresh (eg due to incorrect partitions or > some other file access problem), other partitions can still be modified, > either because they were modified first (in a non-parallel operation) or > modified in parallel (for REFRESH). > In this case, the REFRESH operation will throw an Exception back to the user, > but in fact it has modified the catalog entry. The version number, however, > is not bumped, which breaks some invariants of the catalog that an object > doesn't change without changing version numbers. > This also produces some unexpected behavior such as: > - SHOW FILES IN t; > - REFRESH t; -- gets a failure > - SHOW FILES in t; -- see the same result as originally > - ALTER TABLE t SET UNCACHED; -- bumps the version number due to unrelated > operation > - SHOW FILES IN t; -- the set of files has changed due to the earlier > partially-complete REFRESH -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-8454) Recursively list files within transactional tables
[ https://issues.apache.org/jira/browse/IMPALA-8454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16827132#comment-16827132 ] ASF subversion and git services commented on IMPALA-8454: - Commit 114712031f2293af5b3d9509776f88c23d3fa0fc in impala's branch refs/heads/master from Todd Lipcon [ https://gitbox.apache.org/repos/asf?p=impala.git;h=1147120 ] IMPALA-8454 (part 1): Refactor file descriptor loading code This refactors various file-descriptor loading code out of HdfsTable into new standalone classes. In order to support ACID tables, we'll need to make various changes to these bits of code, and having them extracted and cleaned up will make that easier. This consolidates all of the places in which we list partition directories into one method which does the appropriate thing regardless of situation. This has a small behavior change related to IMPALA-8406: previously, we had a bug where, while refreshing a table, if one or more partitions failed to refresh, the other partitions might still get refreshed despite an error being returned. Those other partitions wouldn't be available immediately until some other operation caused the table's catalog version number to increase. This was buggy behavior. Rather than tackle that problem in this "refactor" patch, this patch just slightly improves the behavior: we'll either atomically update or not update all partitions, but we might still add new partitions noticed by the REFRESH, and might still update other HMS metadata. This patch may end up slightly improving various other code paths that refresh file descriptor lists. We used to have slightly different ways of doing this in three different places, with different sets of optimizations. Now we do it all in one place, and we pull all the known tricks. Change-Id: I59edf493b9ba38be5f556b4795a7684d9c9e3a07 Reviewed-on: http://gerrit.cloudera.org:8080/12950 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins > Recursively list files within transactional tables > -- > > Key: IMPALA-8454 > URL: https://issues.apache.org/jira/browse/IMPALA-8454 > Project: IMPALA > Issue Type: Improvement > Components: Catalog >Reporter: Todd Lipcon >Assignee: Todd Lipcon >Priority: Major > > For transactional tables, the data files are not directly within the > partition directories, but instead are stored within subdirectories > corresponding to writeIds, compactions, etc. To support this, we need to be > able to recursively load file lists within partition directories. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-4018) Add support for SQL:2016 datetime templates/patterns/masks to CAST(... AS ... FORMAT )
[
https://issues.apache.org/jira/browse/IMPALA-4018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16827103#comment-16827103
]
Greg Rahn commented on IMPALA-4018:
---
[~gaborkaszab], [~maze] -
I'm strongly against introducing any behavior that makes the new
CAST(...FORMAT...) syntax not follow the SQL:2016 contract, even via a feature
flag. I am, however, ok with introducing a flag to bring the legacy date masks
and functions into ISO SQL compliance (and provide a UX that matches the
expectations of DBMS users). Here is the main reason why: SQL tools need to be
"taught" how to convert to and from these types and formats and there has to be
only correct way to do so. There is no way to teach a tool conditional logic
based on feature flags. We're working with vendors to add the new SQL:2016
syntax into their tools so there can be only one correct way and that way has
to adhere to the contract that other SQL systems will adopt and use as well. I
hope this makes it clear on why we need to follow and converge on ISO/ANSI SQL
behaviors.
> Add support for SQL:2016 datetime templates/patterns/masks to CAST(... AS ...
> FORMAT )
>
>
> Key: IMPALA-4018
> URL: https://issues.apache.org/jira/browse/IMPALA-4018
> Project: IMPALA
> Issue Type: New Feature
> Components: Frontend
>Affects Versions: Impala 2.2.4
>Reporter: Greg Rahn
>Assignee: Gabor Kaszab
>Priority: Critical
> Labels: ansi-sql, compatibility, sql-language
>
> *Summary*
> The format masks/templates for currently are implemented using the [Java
> SimpleDateFormat
> patterns|http://docs.oracle.com/javase/8/docs/api/java/text/SimpleDateFormat.html],
> and although this is what Hive has implemented, it is not what most standard
> SQL systems implement. For example see
> [Vertica|https://my.vertica.com/docs/7.2.x/HTML/Content/Authoring/SQLReferenceManual/Functions/Formatting/TemplatePatternsForDateTimeFormatting.htm],
>
> [Netezza|http://www.ibm.com/support/knowledgecenter/SSULQD_7.2.1/com.ibm.nz.dbu.doc/r_dbuser_ntz_sql_extns_templ_patterns_date_time_conv.html],
>
> [Oracle|https://docs.oracle.com/database/121/SQLRF/sql_elements004.htm#SQLRF00212],
> and
> [PostgreSQL|https://www.postgresql.org/docs/9.5/static/functions-formatting.html#FUNCTIONS-FORMATTING-DATETIME-TABLE].
>
> *Examples of incompatibilities*
> {noformat}
> -- PostgreSQL/Netezza/Vertica/Oracle
> select to_timestamp('May 15, 2015 12:00:00', 'mon dd, hh:mi:ss');
> -- Impala
> select to_timestamp('May 15, 2015 12:00:00', 'MMM dd, HH:mm:ss');
> -- PostgreSQL/Netezza/Vertica/Oracle
> select to_timestamp('2015-02-14 20:19:07','-mm-dd hh24:mi:ss');
> -- Impala
> select to_timestamp('2015-02-14 20:19:07','-MM-dd HH:mm:ss');
> -- Vertica/Oracle
> select to_timestamp('2015-02-14 20:19:07.123456','-mm-dd hh24:mi:ss.ff');
> -- Impala
> select to_timestamp('2015-02-14 20:19:07.123456','-MM-dd
> HH:mm:ss.SS');
> {noformat}
> *Considerations*
> Because this is a change in default behavior for to_timestamp(), if possible,
> having a feature flag to revert to the legacy Java SimpleDateFormat patterns
> should be strongly considered. This would allow users to chose the behavior
> they desire and scope it to a session if need be.
> SQL:2016 defines the following datetime templates
> {noformat}
> ::=
> { }...
> ::=
>
> |
> ::=
>
> |
> |
> |
> |
> |
> |
> |
> |
> |
> |
> |
> |
> |
> ::=
>
> |
> |
> |
> |
> |
> |
> |
> ::=
> | YYY | YY | Y
> ::=
> | RR
> ::=
> MM
> ::=
> DD
> ::=
> DDD
> ::=
> HH | HH12
> ::=
> HH24
> ::=
> MI
> ::=
> SS
> ::=
> S
> ::=
> FF1 | FF2 | FF3 | FF4 | FF5 | FF6 | FF7 | FF8 | FF9
> ::=
> A.M. | P.M.
> ::=
> TZH
> ::=
> TZM
> {noformat}
> SQL:2016 also introduced the FORMAT clause for CAST which is the standard way
> to do string <> datetime conversions
> {noformat}
> ::=
> CAST
>AS
> [ FORMAT ]
>
> ::=
>
> |
> ::=
>
> |
> ::=
>
> {noformat}
> For example:
> {noformat}
> CAST( AS [FORMAT ])
> CAST( AS [FORMAT ])
> cast(dt as string format 'DD-MM-')
> cast('01-05-2017' as date format 'DD-MM-')
> {noformat}
> *Update*
> Here is the proposal for the new datetime patterns and their semantics:
> https://docs.google.com/document/d/1V7k6-lrPGW7_uhqM-FhKl3QsxwCRy69v2KIxPsGjc1k/
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-4018) Add support for SQL:2016 datetime templates/patterns/masks to CAST(... AS ... FORMAT )
[
https://issues.apache.org/jira/browse/IMPALA-4018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gabor Kaszab updated IMPALA-4018:
-
Description:
*Summary*
The format masks/templates for currently are implemented using the [Java
SimpleDateFormat
patterns|http://docs.oracle.com/javase/8/docs/api/java/text/SimpleDateFormat.html],
and although this is what Hive has implemented, it is not what most standard
SQL systems implement. For example see
[Vertica|https://my.vertica.com/docs/7.2.x/HTML/Content/Authoring/SQLReferenceManual/Functions/Formatting/TemplatePatternsForDateTimeFormatting.htm],
[Netezza|http://www.ibm.com/support/knowledgecenter/SSULQD_7.2.1/com.ibm.nz.dbu.doc/r_dbuser_ntz_sql_extns_templ_patterns_date_time_conv.html],
[Oracle|https://docs.oracle.com/database/121/SQLRF/sql_elements004.htm#SQLRF00212],
and
[PostgreSQL|https://www.postgresql.org/docs/9.5/static/functions-formatting.html#FUNCTIONS-FORMATTING-DATETIME-TABLE].
*Examples of incompatibilities*
{noformat}
-- PostgreSQL/Netezza/Vertica/Oracle
select to_timestamp('May 15, 2015 12:00:00', 'mon dd, hh:mi:ss');
-- Impala
select to_timestamp('May 15, 2015 12:00:00', 'MMM dd, HH:mm:ss');
-- PostgreSQL/Netezza/Vertica/Oracle
select to_timestamp('2015-02-14 20:19:07','-mm-dd hh24:mi:ss');
-- Impala
select to_timestamp('2015-02-14 20:19:07','-MM-dd HH:mm:ss');
-- Vertica/Oracle
select to_timestamp('2015-02-14 20:19:07.123456','-mm-dd hh24:mi:ss.ff');
-- Impala
select to_timestamp('2015-02-14 20:19:07.123456','-MM-dd HH:mm:ss.SS');
{noformat}
*Considerations*
Because this is a change in default behavior for to_timestamp(), if possible,
having a feature flag to revert to the legacy Java SimpleDateFormat patterns
should be strongly considered. This would allow users to chose the behavior
they desire and scope it to a session if need be.
SQL:2016 defines the following datetime templates
{noformat}
::=
{ }...
::=
|
::=
|
|
|
|
|
|
|
|
|
|
|
|
|
::=
|
|
|
|
|
|
|
::=
| YYY | YY | Y
::=
| RR
::=
MM
::=
DD
::=
DDD
::=
HH | HH12
::=
HH24
::=
MI
::=
SS
::=
S
::=
FF1 | FF2 | FF3 | FF4 | FF5 | FF6 | FF7 | FF8 | FF9
::=
A.M. | P.M.
::=
TZH
::=
TZM
{noformat}
SQL:2016 also introduced the FORMAT clause for CAST which is the standard way
to do string <> datetime conversions
{noformat}
::=
CAST
AS
[ FORMAT ]
::=
|
::=
|
::=
{noformat}
For example:
{noformat}
CAST( AS [FORMAT ])
CAST( AS [FORMAT ])
cast(dt as string format 'DD-MM-')
cast('01-05-2017' as date format 'DD-MM-')
{noformat}
*Update*
Here is the proposal for the new datetime patterns and their semantics:
https://docs.google.com/document/d/1V7k6-lrPGW7_uhqM-FhKl3QsxwCRy69v2KIxPsGjc1k/
was:
*Summary*
The format masks/templates for currently are implemented using the [Java
SimpleDateFormat
patterns|http://docs.oracle.com/javase/8/docs/api/java/text/SimpleDateFormat.html],
and although this is what Hive has implemented, it is not what most standard
SQL systems implement. For example see
[Vertica|https://my.vertica.com/docs/7.2.x/HTML/Content/Authoring/SQLReferenceManual/Functions/Formatting/TemplatePatternsForDateTimeFormatting.htm],
[Netezza|http://www.ibm.com/support/knowledgecenter/SSULQD_7.2.1/com.ibm.nz.dbu.doc/r_dbuser_ntz_sql_extns_templ_patterns_date_time_conv.html],
[Oracle|https://docs.oracle.com/database/121/SQLRF/sql_elements004.htm#SQLRF00212],
and
[PostgreSQL|https://www.postgresql.org/docs/9.5/static/functions-formatting.html#FUNCTIONS-FORMATTING-DATETIME-TABLE].
*Examples of incompatibilities*
{noformat}
-- PostgreSQL/Netezza/Vertica/Oracle
select to_timestamp('May 15, 2015 12:00:00', 'mon dd, hh:mi:ss');
-- Impala
select to_timestamp('May 15, 2015 12:00:00', 'MMM dd, HH:mm:ss');
-- PostgreSQL/Netezza/Vertica/Oracle
select to_timestamp('2015-02-14 20:19:07','-mm-dd hh24:mi:ss');
-- Impala
select to_timestamp('2015-02-14 20:19:07','-MM-dd HH:mm:ss');
-- Vertica/Oracle
select to_timestamp('2015-02-14 20:19:07.123456','-mm-dd hh24:mi:ss.ff');
-- Impala
select to_timestamp('2015-02-14 20:19:07.123456','-MM-dd HH:mm:ss.SS');
{noformat}
*Considerations*
Because this is a change in default behavior for to_timestamp(), if possible,
having a feature flag to revert to the legacy Java SimpleDateFormat patterns
should be strongly considered. This would allow users to chose the behavior
they desire and scope it to a session if need be.
SQL:2016 defines the following datetime templates
{noformat}
::=
{ }...
::=
|
::=
|
|
|
|
|
|
|
|
|
|
|
|
|
::=
|
|
|
|
|
|
|
::=
| YYY | YY | Y
::=
| RR
::=
MM
::=
DD
::=
DDD
::=
HH |
[jira] [Commented] (IMPALA-4018) Add support for SQL:2016 datetime templates/patterns/masks to CAST(... AS ... FORMAT )
[
https://issues.apache.org/jira/browse/IMPALA-4018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16826993#comment-16826993
]
Gabor Kaszab commented on IMPALA-4018:
--
Hey [~grahn],
Meanwhile, I have reached out to the Spark community and [~maze] did an initial
run through on the proposal doc. His initial opinion was that it feels
unnatural to have the existing functionality toggle between the old and new
patterns while CAST(..FORMAT..) uses the new patterns only. We agreed that this
introduces a source of confusion on the user's side and would be much cleaner
to implement the toggling mechanism for CAST(..FORMAT..) as well. This way all
the functions within Impala would use the same pattern matching and the flag
would change all of them and the users won't experience that e.g.
to_timestamp() uses the old patterns while CAST(..FORMAT..) uses the new ones:
Depending on the value of the flag all of them would use either the old or the
new functionality.
What do you think? Can we revive the conversation we had around this?
Cheers,
Gabor
> Add support for SQL:2016 datetime templates/patterns/masks to CAST(... AS ...
> FORMAT )
>
>
> Key: IMPALA-4018
> URL: https://issues.apache.org/jira/browse/IMPALA-4018
> Project: IMPALA
> Issue Type: New Feature
> Components: Frontend
>Affects Versions: Impala 2.2.4
>Reporter: Greg Rahn
>Assignee: Gabor Kaszab
>Priority: Critical
> Labels: ansi-sql, compatibility, sql-language
>
> *Summary*
> The format masks/templates for currently are implemented using the [Java
> SimpleDateFormat
> patterns|http://docs.oracle.com/javase/8/docs/api/java/text/SimpleDateFormat.html],
> and although this is what Hive has implemented, it is not what most standard
> SQL systems implement. For example see
> [Vertica|https://my.vertica.com/docs/7.2.x/HTML/Content/Authoring/SQLReferenceManual/Functions/Formatting/TemplatePatternsForDateTimeFormatting.htm],
>
> [Netezza|http://www.ibm.com/support/knowledgecenter/SSULQD_7.2.1/com.ibm.nz.dbu.doc/r_dbuser_ntz_sql_extns_templ_patterns_date_time_conv.html],
>
> [Oracle|https://docs.oracle.com/database/121/SQLRF/sql_elements004.htm#SQLRF00212],
> and
> [PostgreSQL|https://www.postgresql.org/docs/9.5/static/functions-formatting.html#FUNCTIONS-FORMATTING-DATETIME-TABLE].
>
> *Examples of incompatibilities*
> {noformat}
> -- PostgreSQL/Netezza/Vertica/Oracle
> select to_timestamp('May 15, 2015 12:00:00', 'mon dd, hh:mi:ss');
> -- Impala
> select to_timestamp('May 15, 2015 12:00:00', 'MMM dd, HH:mm:ss');
> -- PostgreSQL/Netezza/Vertica/Oracle
> select to_timestamp('2015-02-14 20:19:07','-mm-dd hh24:mi:ss');
> -- Impala
> select to_timestamp('2015-02-14 20:19:07','-MM-dd HH:mm:ss');
> -- Vertica/Oracle
> select to_timestamp('2015-02-14 20:19:07.123456','-mm-dd hh24:mi:ss.ff');
> -- Impala
> select to_timestamp('2015-02-14 20:19:07.123456','-MM-dd
> HH:mm:ss.SS');
> {noformat}
> *Considerations*
> Because this is a change in default behavior for to_timestamp(), if possible,
> having a feature flag to revert to the legacy Java SimpleDateFormat patterns
> should be strongly considered. This would allow users to chose the behavior
> they desire and scope it to a session if need be.
> SQL:2016 defines the following datetime templates
> {noformat}
> ::=
> { }...
> ::=
>
> |
> ::=
>
> |
> |
> |
> |
> |
> |
> |
> |
> |
> |
> |
> |
> |
> ::=
>
> |
> |
> |
> |
> |
> |
> |
> ::=
> | YYY | YY | Y
> ::=
> | RR
> ::=
> MM
> ::=
> DD
> ::=
> DDD
> ::=
> HH | HH12
> ::=
> HH24
> ::=
> MI
> ::=
> SS
> ::=
> S
> ::=
> FF1 | FF2 | FF3 | FF4 | FF5 | FF6 | FF7 | FF8 | FF9
> ::=
> A.M. | P.M.
> ::=
> TZH
> ::=
> TZM
> {noformat}
> SQL:2016 also introduced the FORMAT clause for CAST which is the standard way
> to do string <> datetime conversions
> {noformat}
> ::=
> CAST
>AS
> [ FORMAT ]
>
> ::=
>
> |
> ::=
>
> |
> ::=
>
> {noformat}
> For example:
> {noformat}
> CAST( AS [FORMAT ])
> CAST( AS [FORMAT ])
> cast(dt as string format 'DD-MM-')
> cast('01-05-2017' as date format 'DD-MM-')
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-8381) Remove branch from ParquetPlainEncoder::Decode()
[ https://issues.apache.org/jira/browse/IMPALA-8381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16826924#comment-16826924 ] Daniel Becker commented on IMPALA-8381: --- We made some benchmarks comparing the old and the new implementations. The code for the benchmarks is not in the patch for this Jira as the benchmarks were originally written for the delta encoding (implementation in progress: [https://gerrit.cloudera.org/#/c/12621/]). The benchmarks read plain encoded values to a "scratch batch" with the given stride. `OutType` means the type of the values in the scratch batch. We use the Impala benchmark framework. For the values, bigger is better. Baseline (old implementation): ||ParquetType||OutType||Stride||p10||p50||p90|| |INT32|int32_t|8|136|140|141| |INT32|int32_t|12|136|139|139| |INT32|int32_t|16|137|139|140| |INT32|int32_t|20|137|139|140| |INT32|int32_t|30|125|128|129| |INT32|int32_t|40|92.3|93.9|94.7| |INT32|int32_t|50|72|72.7|73.1| |INT32|int32_t|80|54|54.4|55| |INT32|int32_t|100|51.7|52.2|52.7| |INT32|int32_t|120|49.4|49.9|50.4| |INT32|int32_t|150|46.3|46.6|47| |INT32|int32_t|180|46.9|47.6|48.4| |INT32|int32_t|200|46.9|47.9|48.6| |INT32|int32_t|400|51.7|52.9|54.4| ||ParquetType||OutType||Stride||p10||p50||p90|| |INT64|int64_t|8|137|140|140| |INT64|int64_t|12|135|140|141| |INT64|int64_t|16|137|139|141| |INT64|int64_t|20|132|138|139| |INT64|int64_t|30|115|117|117| |INT64|int64_t|40|88.2|89.5|90.1| |INT64|int64_t|50|69.5|70|70.5| |INT64|int64_t|80|52.7|53|53.4| |INT64|int64_t|100|48|48.5|48.9| |INT64|int64_t|120|48.1|48.6|49.1| |INT64|int64_t|150|43.1|43.6|43.8| |INT64|int64_t|180|42.2|42.4|43| |INT64|int64_t|200|43.9|44.4|44.8| |INT64|int64_t|400|42.5|42.8|43.2| New implementation: ||ParquetType||OutType||Stride||p10||p50||p90|| |INT32|int32_t|8|281|284|286| |INT32|int32_t|12|257|261|263| |INT32|int32_t|16|237|240|242| |INT32|int32_t|20|209|212|215| |INT32|int32_t|30|136|138|139| |INT32|int32_t|40|95.7|96.5|97.2| |INT32|int32_t|50|73.2|73.9|74.5| |INT32|int32_t|80|54|54.5|55.2| |INT32|int32_t|100|51.9|52.4|52.8| |INT32|int32_t|120|49.6|50|50.4| |INT32|int32_t|150|45.9|46.6|47| |INT32|int32_t|180|47.3|48.6|49.5| |INT32|int32_t|200|47.4|48.6|49.7| |INT32|int32_t|400|50.7|52.9|54.4| ||ParquetType||OutType||Stride||p10||p50||p90|| |INT64|int64_t|8|264|268|271| |INT64|int64_t|12|221|224|226| |INT64|int64_t|16|216|217|219| |INT64|int64_t|20|184|186|188| |INT64|int64_t|30|120|122|123| |INT64|int64_t|40|90.2|91.3|91.9| |INT64|int64_t|50|69.3|69.9|70.4| |INT64|int64_t|80|52.7|53.2|53.6| |INT64|int64_t|100|48.2|48.8|49.1| |INT64|int64_t|120|48.2|48.7|49| |INT64|int64_t|150|43.2|43.6|44| |INT64|int64_t|180|42.2|42.7|43| |INT64|int64_t|200|44|44.5|44.7| |INT64|int64_t|400|43.9|44.5|45.1| > Remove branch from ParquetPlainEncoder::Decode() > > > Key: IMPALA-8381 > URL: https://issues.apache.org/jira/browse/IMPALA-8381 > Project: IMPALA > Issue Type: Improvement > Components: Backend >Reporter: Csaba Ringhofer >Assignee: Daniel Becker >Priority: Minor > Labels: newbie, parquet, performance, ramp-up > > Removing the "if" at > https://github.com/apache/impala/blob/5670f96b828d57f9e36510bb9af02bcc31de775c/be/src/exec/parquet/parquet-common.h#L203 > can lead to 1.5x speed up in plain decoding (type=int32, stride=16). For > primitive types, the same check can be done for a whole batch, so the speedup > can be gained for large batches without loosing safety. The only Parquet type > where this check is needed per element is BYTE_ARRAY (typically used for > STRING columns), which already has a template specialization for > ParquetPlainEncoder::Decode(). -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-8446) Create a unit test for Admission Controller
[ https://issues.apache.org/jira/browse/IMPALA-8446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16826686#comment-16826686 ] ASF subversion and git services commented on IMPALA-8446: - Commit 2dda47ec27e8f15810907d77c6e41c275355cea6 in impala's branch refs/heads/master from Andrew Sherman [ https://gitbox.apache.org/repos/asf?p=impala.git;h=2dda47e ] IMPALA-8446: Create a unit test for Admission Controller. This test allows construction of white box tests that exercise Admission Controller code. The initial test cases are (1) a test which does the following: + creates a RequestPoolService which reads some Admission Controller configuration files + creates an Admission Controller object + creates a QuerySchedule for a query + tests if the query can be admitted by the Admission Controller + simulates activity in the cluster which consumes memory + tests that the earlier query cannot now be admitted by the Admission Controller (2) a test which verifies that the configuration files are read correctly. TESTING: Ran end-to-end tests cleanly and checked that the new tests were executed. Change-Id: I8a840590b868f2df1a06f3f397b7b0fc2b29462c Reviewed-on: http://gerrit.cloudera.org:8080/13078 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins > Create a unit test for Admission Controller > --- > > Key: IMPALA-8446 > URL: https://issues.apache.org/jira/browse/IMPALA-8446 > Project: IMPALA > Issue Type: Bug >Reporter: Andrew Sherman >Assignee: Andrew Sherman >Priority: Major > > This will allow construction of white box tests that exercise Admission > Controller code. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-8119) Improve experience for configuring JVM heap size in docker container.
[ https://issues.apache.org/jira/browse/IMPALA-8119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Armstrong resolved IMPALA-8119. --- Resolution: Fixed Fix Version/s: Impala 3.3.0 > Improve experience for configuring JVM heap size in docker container. > - > > Key: IMPALA-8119 > URL: https://issues.apache.org/jira/browse/IMPALA-8119 > Project: IMPALA > Issue Type: Sub-task > Components: Infrastructure >Affects Versions: Impala 3.2.0 >Reporter: Tim Armstrong >Assignee: Tim Armstrong >Priority: Major > Labels: docker > Fix For: Impala 3.3.0 > > > The heap size can be configured via JAVA_TOOL_OPTIONS. Configuring the JVM > heap size is likely a common task when deploying Impala containers to > production because the catalogd and coordinators need to be sized to fit the > working set of metadata - it would be worthwhile to have a more explicit and > clearly-documented way to set the heap size. > We could also try to be fancy and choose a heap size based on the container > memory limit and the role of the container. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-8119) Improve experience for configuring JVM heap size in docker container.
[ https://issues.apache.org/jira/browse/IMPALA-8119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16826685#comment-16826685 ] ASF subversion and git services commented on IMPALA-8119: - Commit 28d5a8299c27fd06779e108943a9aa128c5af15f in impala's branch refs/heads/master from Tim Armstrong [ https://gitbox.apache.org/repos/asf?p=impala.git;h=28d5a82 ] IMPALA-8119: document how to set heap size in docker JAVA_TOOL_OPTIONS is a standard mechanism to pass arguments to a JVM. Let's just document this as the canonical way to pass in the heap size. Change-Id: Ie6ddba3c42a698b52d7c4e2ff6a9c73068e198b2 Reviewed-on: http://gerrit.cloudera.org:8080/13119 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins > Improve experience for configuring JVM heap size in docker container. > - > > Key: IMPALA-8119 > URL: https://issues.apache.org/jira/browse/IMPALA-8119 > Project: IMPALA > Issue Type: Sub-task > Components: Infrastructure >Affects Versions: Impala 3.2.0 >Reporter: Tim Armstrong >Assignee: Tim Armstrong >Priority: Major > Labels: docker > > The heap size can be configured via JAVA_TOOL_OPTIONS. Configuring the JVM > heap size is likely a common task when deploying Impala containers to > production because the catalogd and coordinators need to be sized to fit the > working set of metadata - it would be worthwhile to have a more explicit and > clearly-documented way to set the heap size. > We could also try to be fancy and choose a heap size based on the container > memory limit and the role of the container. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
