[jira] [Created] (IMPALA-11962) Add Support for Rhel 9
Manish Maheshwari created IMPALA-11962: -- Summary: Add Support for Rhel 9 Key: IMPALA-11962 URL: https://issues.apache.org/jira/browse/IMPALA-11962 Project: IMPALA Issue Type: New Feature Reporter: Manish Maheshwari -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-11962) Add Support for Rhel 9
[ https://issues.apache.org/jira/browse/IMPALA-11962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17695577#comment-17695577 ] Manish Maheshwari commented on IMPALA-11962: Support to run Impala on RHEL 9 and also check impala-shell and impyla can use the OS Python from RHEL 9 to execute > Add Support for Rhel 9 > -- > > Key: IMPALA-11962 > URL: https://issues.apache.org/jira/browse/IMPALA-11962 > Project: IMPALA > Issue Type: New Feature >Reporter: Manish Maheshwari >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-11963) Make Iceberg catalog property support more generic
Gabor Kaszab created IMPALA-11963: - Summary: Make Iceberg catalog property support more generic Key: IMPALA-11963 URL: https://issues.apache.org/jira/browse/IMPALA-11963 Project: IMPALA Issue Type: Improvement Components: Frontend Reporter: Gabor Kaszab Currently the Iceberg catalog properties are added to a list and it requires an Impala code change to support additional properties. [https://github.com/apache/impala/blob/47c71bbb32d34d4583856af227206934b6f15136/fe/src/main/java/org/apache/impala/util/IcebergUtil.java#L1054] However, it would be nice to just simply add the property to the config xml and then Impala would pick it up automatically. Search for the ones with "iceberg." prefix maybe? -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-11963) Make Iceberg catalog property support more generic
[ https://issues.apache.org/jira/browse/IMPALA-11963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17695673#comment-17695673 ] Gabor Kaszab commented on IMPALA-11963: --- cc [~rizaon] [~boroknagyz] > Make Iceberg catalog property support more generic > -- > > Key: IMPALA-11963 > URL: https://issues.apache.org/jira/browse/IMPALA-11963 > Project: IMPALA > Issue Type: Improvement > Components: Frontend >Reporter: Gabor Kaszab >Priority: Major > Labels: impala-iceberg > > Currently the Iceberg catalog properties are added to a list and it requires > an Impala code change to support additional properties. > [https://github.com/apache/impala/blob/47c71bbb32d34d4583856af227206934b6f15136/fe/src/main/java/org/apache/impala/util/IcebergUtil.java#L1054] > However, it would be nice to just simply add the property to the config xml > and then Impala would pick it up automatically. Search for the ones with > "iceberg." prefix maybe? -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-11963) Make Iceberg catalog property support more generic
[ https://issues.apache.org/jira/browse/IMPALA-11963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Kaszab updated IMPALA-11963: -- Description: Currently the Iceberg catalog properties are added to a list and it requires an Impala code change to support additional properties. [https://github.com/apache/impala/blob/47c71bbb32d34d4583856af227206934b6f15136/fe/src/main/java/org/apache/impala/util/IcebergUtil.java#L1054] However, it would be nice to just simply add the property to the core-site.xml config and then Impala would pick it up automatically. Search for the ones with "iceberg." prefix maybe? was: Currently the Iceberg catalog properties are added to a list and it requires an Impala code change to support additional properties. [https://github.com/apache/impala/blob/47c71bbb32d34d4583856af227206934b6f15136/fe/src/main/java/org/apache/impala/util/IcebergUtil.java#L1054] However, it would be nice to just simply add the property to the config xml and then Impala would pick it up automatically. Search for the ones with "iceberg." prefix maybe? > Make Iceberg catalog property support more generic > -- > > Key: IMPALA-11963 > URL: https://issues.apache.org/jira/browse/IMPALA-11963 > Project: IMPALA > Issue Type: Improvement > Components: Frontend >Reporter: Gabor Kaszab >Priority: Major > Labels: impala-iceberg > > Currently the Iceberg catalog properties are added to a list and it requires > an Impala code change to support additional properties. > [https://github.com/apache/impala/blob/47c71bbb32d34d4583856af227206934b6f15136/fe/src/main/java/org/apache/impala/util/IcebergUtil.java#L1054] > However, it would be nice to just simply add the property to the > core-site.xml config and then Impala would pick it up automatically. Search > for the ones with "iceberg." prefix maybe? -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-11963) Make Iceberg catalog property support more generic
[ https://issues.apache.org/jira/browse/IMPALA-11963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17695753#comment-17695753 ] Riza Suminto commented on IMPALA-11963: --- [~gaborkaszab] [~boroknagyz] Agree. I have question though. Will Impala support multiple HiveCatalog/HadoopCatalog in the future? Right now, Impala reads single catalog properties and apply to to all instances of HiveCatalog and HadoopCatalog. What if individual catalog wants to have their own properties? > Make Iceberg catalog property support more generic > -- > > Key: IMPALA-11963 > URL: https://issues.apache.org/jira/browse/IMPALA-11963 > Project: IMPALA > Issue Type: Improvement > Components: Frontend >Reporter: Gabor Kaszab >Priority: Major > Labels: impala-iceberg > > Currently the Iceberg catalog properties are added to a list and it requires > an Impala code change to support additional properties. > [https://github.com/apache/impala/blob/47c71bbb32d34d4583856af227206934b6f15136/fe/src/main/java/org/apache/impala/util/IcebergUtil.java#L1054] > However, it would be nice to just simply add the property to the > core-site.xml config and then Impala would pick it up automatically. Search > for the ones with "iceberg." prefix maybe? -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-11963) Make Iceberg catalog property support more generic
[ https://issues.apache.org/jira/browse/IMPALA-11963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17695785#comment-17695785 ] Zoltán Borók-Nagy commented on IMPALA-11963: [~rizaon] is this the bug you repoerted here? IMPALA-11636 Yeah, I think with Iceberg's Catalogs class we should be able to configure multiple instances of different catalogs with different configurations. > Make Iceberg catalog property support more generic > -- > > Key: IMPALA-11963 > URL: https://issues.apache.org/jira/browse/IMPALA-11963 > Project: IMPALA > Issue Type: Improvement > Components: Frontend >Reporter: Gabor Kaszab >Priority: Major > Labels: impala-iceberg > > Currently the Iceberg catalog properties are added to a list and it requires > an Impala code change to support additional properties. > [https://github.com/apache/impala/blob/47c71bbb32d34d4583856af227206934b6f15136/fe/src/main/java/org/apache/impala/util/IcebergUtil.java#L1054] > However, it would be nice to just simply add the property to the > core-site.xml config and then Impala would pick it up automatically. Search > for the ones with "iceberg." prefix maybe? -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-11964) Make sure Impala returns error for Iceberg tables with equality deletes
Zoltán Borók-Nagy created IMPALA-11964: -- Summary: Make sure Impala returns error for Iceberg tables with equality deletes Key: IMPALA-11964 URL: https://issues.apache.org/jira/browse/IMPALA-11964 Project: IMPALA Issue Type: Bug Components: Frontend Reporter: Zoltán Borók-Nagy Impala only supports position deletes currently. It should raise an error when equality deletes are encountered: [https://github.com/apache/impala/blob/eaf71bca0a3538056765dbbce13c0f503045b38b/fe/src/main/java/org/apache/impala/planner/IcebergScanPlanner.java#L344] But if we are not invoking planFiles() it's possible that Impala starts reading equality delete files as position delete files which should also raise an error. But it would be still better to catch the error during planning. To make sure we should add tests for these scenarios. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-11963) Make Iceberg catalog property support more generic
[ https://issues.apache.org/jira/browse/IMPALA-11963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17695808#comment-17695808 ] Riza Suminto commented on IMPALA-11963: --- Yes! I forgot I already file that before. So for HiveCatalog, Impala should double check to hive-site.xml. What about HadoopCatalog? can we have multiple HadoopCatalog? It does look possible here, but maybe all are short-lived since they are not singleton like IcebergHiveCatalog: https://github.com/apache/impala/blob/eaf71bca0a3538056765dbbce13c0f503045b38b/fe/src/main/java/org/apache/impala/util/IcebergUtil.java#L124-L125 > Make Iceberg catalog property support more generic > -- > > Key: IMPALA-11963 > URL: https://issues.apache.org/jira/browse/IMPALA-11963 > Project: IMPALA > Issue Type: Improvement > Components: Frontend >Reporter: Gabor Kaszab >Priority: Major > Labels: impala-iceberg > > Currently the Iceberg catalog properties are added to a list and it requires > an Impala code change to support additional properties. > [https://github.com/apache/impala/blob/47c71bbb32d34d4583856af227206934b6f15136/fe/src/main/java/org/apache/impala/util/IcebergUtil.java#L1054] > However, it would be nice to just simply add the property to the > core-site.xml config and then Impala would pick it up automatically. Search > for the ones with "iceberg." prefix maybe? -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-11965) TestCodegenCache fails in exhaustive runs
Zoltán Borók-Nagy created IMPALA-11965:
--
Summary: TestCodegenCache fails in exhaustive runs
Key: IMPALA-11965
URL: https://issues.apache.org/jira/browse/IMPALA-11965
Project: IMPALA
Issue Type: Bug
Components: Backend
Reporter: Zoltán Borók-Nagy
TestCodegenCache fails with the following errors:
{noformat}
AssertionError: Could not find metric: impala.codegen-cache.entries-evicted
AssertionError: Could not find metric: impala.codegen-cache.entries-in-use
{noformat}
Example stack trace:
{noformat}
custom_cluster/test_codegen_cache.py:89: in test_codegen_cache_date_string_col
"select * from functional.alltypes where date_string_col != ''")
custom_cluster/test_codegen_cache.py:141: in _test_codegen_cache
self._check_metric_expect_init()
custom_cluster/test_codegen_cache.py:131: in _check_metric_expect_init
assert self.get_metric('impala.codegen-cache.entries-evicted') == 0
common/impala_test_suite.py:509: in get_metric
assert False, "Could not find metric: %s" % name
E AssertionError: Could not find metric:
impala.codegen-cache.entries-evicted{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Assigned] (IMPALA-11965) TestCodegenCache fails in exhaustive runs
[
https://issues.apache.org/jira/browse/IMPALA-11965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoltán Borók-Nagy reassigned IMPALA-11965:
--
Assignee: Yida Wu
> TestCodegenCache fails in exhaustive runs
> -
>
> Key: IMPALA-11965
> URL: https://issues.apache.org/jira/browse/IMPALA-11965
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Reporter: Zoltán Borók-Nagy
>Assignee: Yida Wu
>Priority: Major
> Labels: broken-build
>
> TestCodegenCache fails with the following errors:
>
> {noformat}
> AssertionError: Could not find metric: impala.codegen-cache.entries-evicted
> AssertionError: Could not find metric: impala.codegen-cache.entries-in-use
> {noformat}
>
> Example stack trace:
> {noformat}
> custom_cluster/test_codegen_cache.py:89: in test_codegen_cache_date_string_col
> "select * from functional.alltypes where date_string_col != ''")
> custom_cluster/test_codegen_cache.py:141: in _test_codegen_cache
> self._check_metric_expect_init()
> custom_cluster/test_codegen_cache.py:131: in _check_metric_expect_init
> assert self.get_metric('impala.codegen-cache.entries-evicted') == 0
> common/impala_test_suite.py:509: in get_metric
> assert False, "Could not find metric: %s" % name
> E AssertionError: Could not find metric:
> impala.codegen-cache.entries-evicted{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-11965) TestCodegenCache fails in exhaustive runs
[
https://issues.apache.org/jira/browse/IMPALA-11965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17695818#comment-17695818
]
Zoltán Borók-Nagy commented on IMPALA-11965:
Hi [~baggio000], I'm assigning this ticket to you as you are the most familiar
with these tests. Feel free to re-assign.
> TestCodegenCache fails in exhaustive runs
> -
>
> Key: IMPALA-11965
> URL: https://issues.apache.org/jira/browse/IMPALA-11965
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Reporter: Zoltán Borók-Nagy
>Assignee: Yida Wu
>Priority: Major
> Labels: broken-build
>
> TestCodegenCache fails with the following errors:
>
> {noformat}
> AssertionError: Could not find metric: impala.codegen-cache.entries-evicted
> AssertionError: Could not find metric: impala.codegen-cache.entries-in-use
> {noformat}
>
> Example stack trace:
> {noformat}
> custom_cluster/test_codegen_cache.py:89: in test_codegen_cache_date_string_col
> "select * from functional.alltypes where date_string_col != ''")
> custom_cluster/test_codegen_cache.py:141: in _test_codegen_cache
> self._check_metric_expect_init()
> custom_cluster/test_codegen_cache.py:131: in _check_metric_expect_init
> assert self.get_metric('impala.codegen-cache.entries-evicted') == 0
> common/impala_test_suite.py:509: in get_metric
> assert False, "Could not find metric: %s" % name
> E AssertionError: Could not find metric:
> impala.codegen-cache.entries-evicted{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Comment Edited] (IMPALA-11816) Flaky TestEventProcessing tests
[
https://issues.apache.org/jira/browse/IMPALA-11816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17695835#comment-17695835
]
Michael Smith edited comment on IMPALA-11816 at 3/2/23 5:23 PM:
Hit an instance of this in
https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/18818/testReport/junit/metadata.test_event_processing/TestEventProcessing/test_event_based_replication/.
was (Author: JIRAUSER288956):
Hit an incident of this in
https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/18818/testReport/junit/metadata.test_event_processing/TestEventProcessing/test_event_based_replication/.
> Flaky TestEventProcessing tests
> ---
>
> Key: IMPALA-11816
> URL: https://issues.apache.org/jira/browse/IMPALA-11816
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
>Affects Versions: Impala 4.3.0
>Reporter: Tamas Mate
>Priority: Major
>
> h3. Failing tests are:
> metadata.test_event_processing.TestEventProcessing.test_transactional_insert_events
> metadata.test_event_processing.TestEventProcessing.test_event_based_replication
> h3. Error Message
> {code:none}
> metadata/test_event_processing.py:176:
>in test_event_based_replication self.__run_event_based_replication_tests()
> metadata/test_event_processing.py:223:
>in __run_event_based_replication_tests
> EventProcessorUtils.wait_for_event_processing(self)
> util/event_processor_utils.py:61:
>in wait_for_event_processing within \{1} seconds".format(current_event_id,
> timeout))
> E Exception: Event processor did not sync till last known event id 38523
> within 10 seconds
> {code}
> h3. Stacktrace
> {code:none}
> metadata/test_event_processing.py:176:
>in test_event_based_replication self.__run_event_based_replication_tests()
> metadata/test_event_processing.py:223:
>in __run_event_based_replication_tests
> EventProcessorUtils.wait_for_event_processing(self)
> util/event_processor_utils.py:61:
>in wait_for_event_processing within \{1} seconds".format(current_event_id,
> timeout)) E
> Exception: Event processor did not sync till last known event id 38523 within
> 10 seconds
> {code}
> h3. Standard Error
> {code:none}
> SET
> client_identifier=metadata/test_event_processing.py::TestEventProcessing::()::test_event_based_replication;
> -- connecting to: localhost:21000 --
> 2022-12-27 13:08:41,389 INFO MainThread: Could not connect to ('::1', 21000,
> 0, 0)
> Traceback (most recent call last):
>File
> "/data/jenkins/workspace/impala-cdwh-2022.0.11.1-core-asan/Impala-Toolchain/toolchain-packages-gcc7.5.0/thrift-0.11.0-p5/python/lib/python2.7/site-packages/thrift/transport/TSocket.py",
> line 104, in open handle.connect(sockaddr)
>File
> "/data/jenkins/workspace/impala-cdwh-2022.0.11.1-core-asan/Impala-Toolchain/toolchain-packages-gcc7.5.0/python-2.7.16/lib/python2.7/socket.py",
> line 228, in meth
> return getattr(self._sock,name)(*args) error:
> [Errno 111] Connection refused
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-11816) Flaky TestEventProcessing tests
[
https://issues.apache.org/jira/browse/IMPALA-11816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17695835#comment-17695835
]
Michael Smith commented on IMPALA-11816:
Hit an incident of this in
https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/18818/testReport/junit/metadata.test_event_processing/TestEventProcessing/test_event_based_replication/.
> Flaky TestEventProcessing tests
> ---
>
> Key: IMPALA-11816
> URL: https://issues.apache.org/jira/browse/IMPALA-11816
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
>Affects Versions: Impala 4.3.0
>Reporter: Tamas Mate
>Priority: Major
>
> h3. Failing tests are:
> metadata.test_event_processing.TestEventProcessing.test_transactional_insert_events
> metadata.test_event_processing.TestEventProcessing.test_event_based_replication
> h3. Error Message
> {code:none}
> metadata/test_event_processing.py:176:
>in test_event_based_replication self.__run_event_based_replication_tests()
> metadata/test_event_processing.py:223:
>in __run_event_based_replication_tests
> EventProcessorUtils.wait_for_event_processing(self)
> util/event_processor_utils.py:61:
>in wait_for_event_processing within \{1} seconds".format(current_event_id,
> timeout))
> E Exception: Event processor did not sync till last known event id 38523
> within 10 seconds
> {code}
> h3. Stacktrace
> {code:none}
> metadata/test_event_processing.py:176:
>in test_event_based_replication self.__run_event_based_replication_tests()
> metadata/test_event_processing.py:223:
>in __run_event_based_replication_tests
> EventProcessorUtils.wait_for_event_processing(self)
> util/event_processor_utils.py:61:
>in wait_for_event_processing within \{1} seconds".format(current_event_id,
> timeout)) E
> Exception: Event processor did not sync till last known event id 38523 within
> 10 seconds
> {code}
> h3. Standard Error
> {code:none}
> SET
> client_identifier=metadata/test_event_processing.py::TestEventProcessing::()::test_event_based_replication;
> -- connecting to: localhost:21000 --
> 2022-12-27 13:08:41,389 INFO MainThread: Could not connect to ('::1', 21000,
> 0, 0)
> Traceback (most recent call last):
>File
> "/data/jenkins/workspace/impala-cdwh-2022.0.11.1-core-asan/Impala-Toolchain/toolchain-packages-gcc7.5.0/thrift-0.11.0-p5/python/lib/python2.7/site-packages/thrift/transport/TSocket.py",
> line 104, in open handle.connect(sockaddr)
>File
> "/data/jenkins/workspace/impala-cdwh-2022.0.11.1-core-asan/Impala-Toolchain/toolchain-packages-gcc7.5.0/python-2.7.16/lib/python2.7/socket.py",
> line 228, in meth
> return getattr(self._sock,name)(*args) error:
> [Errno 111] Connection refused
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-11920) Spill to HDFS/Ozone can't address by service name
[
https://issues.apache.org/jira/browse/IMPALA-11920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17695843#comment-17695843
]
ASF subversion and git services commented on IMPALA-11920:
--
Commit 1321b5ce54b4c1d70715ffde9c898612ac9f3ed8 in impala's branch
refs/heads/master from Michael Smith
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=1321b5ce5 ]
IMPALA-11920: [DOCS] Cleanup and update spill examples
Updates documentation to include examples with service identifier. Also
fixes inconsistent use of ASCII quotes for example text, highlighting
code and variable names, and normalizes descriptions between
S3/HDFS/Ozone. Removes "priority" from remote descriptions as it is
optional and does nothing.
Change-Id: I624a607bda33ab47100e1540ff1d66c8d19a7329
Reviewed-on: http://gerrit.cloudera.org:8080/19504
Reviewed-by: Michael Smith
Tested-by: Michael Smith
> Spill to HDFS/Ozone can't address by service name
> -
>
> Key: IMPALA-11920
> URL: https://issues.apache.org/jira/browse/IMPALA-11920
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 4.2.0
>Reporter: Michael Smith
>Assignee: Michael Smith
>Priority: Major
>
> HDFS and Ozone HA deployments often use a service name, such as
> {{ofs://ozone1/...}}. The current {{scratch_dirs}} token parsing for requires
> a {{hostname:port}}, which doesn't support HA addressing. We should make the
> parsing smarter.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-11920) Spill to HDFS/Ozone can't address by service name
[
https://issues.apache.org/jira/browse/IMPALA-11920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17695842#comment-17695842
]
ASF subversion and git services commented on IMPALA-11920:
--
Commit 99d676f8fb71304838c8fde70d3dd220f8f1f52a in impala's branch
refs/heads/master from Michael Smith
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=99d676f8f ]
IMPALA-11920: Support spill to HDFS address by service ID
Allows addressing HDFS (and Ozone) filesystems in `scratch_dirs` by a
service identifier that doesn't include a port number. Examples
- "hdfs://hdfs1/:10G" uses the root directory of HDFS with a 10G limit
- "ofs://ozone1/tmp::" uses /tmp in Ozone with default limit/priority
Updates `scratch_dirs` parsing to allow whitespace after each specifier,
as in "hdfs://hdfs1/ , /tmp". This is unambiguous and avoids failures
for simple mistakes.
Testing:
- new backend test cases run with HDFS and Ozone
- manually tested that Impala starts with
--impalad_args=--scratch_dirs=ofs://localhost/tmp,/tmp
creates impala-scratch in both locations
Change-Id: Ie069cba211df85fe90d174900b20a26fcc1f7167
Reviewed-on: http://gerrit.cloudera.org:8080/19496
Reviewed-by: Michael Smith
Tested-by: Michael Smith
> Spill to HDFS/Ozone can't address by service name
> -
>
> Key: IMPALA-11920
> URL: https://issues.apache.org/jira/browse/IMPALA-11920
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 4.2.0
>Reporter: Michael Smith
>Assignee: Michael Smith
>Priority: Major
>
> HDFS and Ozone HA deployments often use a service name, such as
> {{ofs://ozone1/...}}. The current {{scratch_dirs}} token parsing for requires
> a {{hostname:port}}, which doesn't support HA addressing. We should make the
> parsing smarter.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-11565) Support IF NOT EXISTS in alter table add columns for kudu table
[ https://issues.apache.org/jira/browse/IMPALA-11565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17695840#comment-17695840 ] ASF subversion and git services commented on IMPALA-11565: -- Commit 8b375a66a29cfea33a4e418b1fa3b9b5139cf907 in impala's branch refs/heads/master from xiabaike [ https://gitbox.apache.org/repos/asf?p=impala.git;h=8b375a66a ] IMPALA-11565: Support IF NOT EXISTS in alter table add columns for kudu/iceberg table Impala already supports IF NOT EXISTS in alter table add columns for general hive table in IMPALA-7832, but not for kudu/iceberg table. This patch try to add such semantics for kudu/iceberg table. Testing: - Updated E2E DDL tests - Added fe tests Change-Id: I82590e5372e881f2e81d4ed3dd0d32a2d3ddb517 Reviewed-on: http://gerrit.cloudera.org:8080/18953 Tested-by: Impala Public Jenkins Reviewed-by: Wenzhe Zhou > Support IF NOT EXISTS in alter table add columns for kudu table > --- > > Key: IMPALA-11565 > URL: https://issues.apache.org/jira/browse/IMPALA-11565 > Project: IMPALA > Issue Type: New Feature >Reporter: Baike Xia >Assignee: Baike Xia >Priority: Major > > Impala already supports IF NOT EXISTS in alter table add columns for general > hive table in [IMPALA-7832|http://issues.apache.org/jira/browse/IMPALA-7832], > but not for kudu table. This patch try to add such semantics for kudu table. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-7832) Support IF NOT EXISTS in alter table add columns
[ https://issues.apache.org/jira/browse/IMPALA-7832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17695841#comment-17695841 ] ASF subversion and git services commented on IMPALA-7832: - Commit 8b375a66a29cfea33a4e418b1fa3b9b5139cf907 in impala's branch refs/heads/master from xiabaike [ https://gitbox.apache.org/repos/asf?p=impala.git;h=8b375a66a ] IMPALA-11565: Support IF NOT EXISTS in alter table add columns for kudu/iceberg table Impala already supports IF NOT EXISTS in alter table add columns for general hive table in IMPALA-7832, but not for kudu/iceberg table. This patch try to add such semantics for kudu/iceberg table. Testing: - Updated E2E DDL tests - Added fe tests Change-Id: I82590e5372e881f2e81d4ed3dd0d32a2d3ddb517 Reviewed-on: http://gerrit.cloudera.org:8080/18953 Tested-by: Impala Public Jenkins Reviewed-by: Wenzhe Zhou > Support IF NOT EXISTS in alter table add columns > > > Key: IMPALA-7832 > URL: https://issues.apache.org/jira/browse/IMPALA-7832 > Project: IMPALA > Issue Type: New Feature > Components: Frontend >Affects Versions: Impala 3.1.0 >Reporter: Thomas Tauber-Marshall >Assignee: Fredy Wijaya >Priority: Minor > Labels: ramp-up > Fix For: Impala 3.2.0 > > > alter table add [if not exists] columns ( [, > ...]) > would add the column only if a column of the same name does not already exist > Probably worth checking out what other databases do in different situations, > eg. if the column already exists but with a different type, if "replace" is > used instead of "add", etc. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-11920) Spill to HDFS/Ozone can't address by service name
[
https://issues.apache.org/jira/browse/IMPALA-11920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Smith resolved IMPALA-11920.
Fix Version/s: Impala 4.3.0
Resolution: Fixed
> Spill to HDFS/Ozone can't address by service name
> -
>
> Key: IMPALA-11920
> URL: https://issues.apache.org/jira/browse/IMPALA-11920
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 4.2.0
>Reporter: Michael Smith
>Assignee: Michael Smith
>Priority: Major
> Fix For: Impala 4.3.0
>
>
> HDFS and Ozone HA deployments often use a service name, such as
> {{ofs://ozone1/...}}. The current {{scratch_dirs}} token parsing for requires
> a {{hostname:port}}, which doesn't support HA addressing. We should make the
> parsing smarter.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-11814) TransmitData RPC become slow when multiple large queries are executed simultaneously
[
https://issues.apache.org/jira/browse/IMPALA-11814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17695847#comment-17695847
]
Michael Smith commented on IMPALA-11814:
The most likely candidates here are: CPU starvation, network starvation, or
lock contention.
You can look at something like pidstat (i.e.
https://access.redhat.com/solutions/69271) to look at CPU context switching and
top for high CPU activity.
Something like iftop could be a starting point for looking at network
saturation.
If neither are maxed out, there are a number of tools for analyzing lock
contention
- https://docs.kernel.org/locking/lockstat.html - usually requires recompiling
the kernel to add CONFIG_LOCK_STAT
- https://github.com/isc-projects/mutrace
-
https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/systemtap_beginners_guide/futexcontentionsect
- perf sampling and look at repeated locks
> TransmitData RPC become slow when multiple large queries are executed
> simultaneously
>
>
> Key: IMPALA-11814
> URL: https://issues.apache.org/jira/browse/IMPALA-11814
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 4.0.0
>Reporter: YifanZhang
>Priority: Major
> Labels: rpc
>
> We have seen a lot of slow TransmitData RPC in our clusters when multiple
> large queries are running. Logs in executors:
> {code:java}
> 1227 16:44:24.859488 1134 rpcz_store.cc:269] Call
> impala.DataStreamService.TransmitData from 11.174.152.179:53824 (request call
> id 10458) took 368799ms. Request Metrics: {}
> I1227 16:44:24.859828 1134 rpcz_store.cc:273] Trace:
> 1227 16:38:16.060450 (+ 0us) impala-service-pool.cc:179] Inserting onto
> call queue
> 1227 16:38:16.060480 (+ 30us) impala-service-pool.cc:278] Handling call
> 1227 16:38:16.060498 (+ 18us) krpc-data-stream-recvr.cc:325] Enqueuing
> deferred RPC
> 1227 16:44:24.859372 (+368798874us) krpc-data-stream-recvr.cc:504] Processing
> deferred RPC
> 1227 16:44:24.859382 (+ 10us) krpc-data-stream-recvr.cc:397] Deserializing
> batch
> 1227 16:44:24.859442 (+ 60us) krpc-data-stream-recvr.cc:424] Enqueuing
> deserialized batch
> 1227 16:44:24.859451 (+ 9us) inbound_call.cc:162] Queueing success
> response
> Metrics: {} {code}
> The RPC spends a lot of time waiting in the queue. Some other executors also
> take a long time to transmit data:
> {code:java}
> I1227 16:44:23.379583 528 krpc-data-stream-sender.cc:396] Slow TransmitData
> RPC to 11.136.184.52:50257
> (fragment_instance_id=f9477dcd54c9d817:1bc346de01e7): took 5m37s.
> Receiver time: 5m37s Network time: 1.343ms
> I1227 16:44:23.396365 631 krpc-data-stream-sender.cc:396] Slow TransmitData
> RPC to 11.174.158.152:58535
> (fragment_instance_id=f9477dcd54c9d817:1bc346de0231): took 5m57s.
> Receiver time: 5m57s Network time: 2.970ms
> I1227 16:44:23.396482 4654 krpc-data-stream-sender.cc:430]
> f9477dcd54c9d817:1bc346de01b6] Long delay waiting for RPC to
> 11.174.158.152:58535
> (fragment_instance_id=f9477dcd54c9d817:1bc346de0231): took 5m57s
> I1227 16:44:23.444633 572 krpc-data-stream-sender.cc:396] Slow TransmitData
> RPC to 11.174.153.240:60303
> (fragment_instance_id=f9477dcd54c9d817:1bc346de02c3): took 5m53s.
> Receiver time: 5m53s Network time: 17.604ms{code}
> The max handler latency time is also very long:
> |*Method: _TransmitData_*|
> |Handler Latency|Count: 2321637, min / max: 3.000us / 5m10s, 25th %-ile:
> 22.000us, 50th %-ile: 38.000us, 75th %-ile: 55.000us, 90th %-ile: 72.000us,
> 95th %-ile: 86.000us, 99.9th %-ile: 4.512ms|
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-4052) CREATE TABLE LIKE for Kudu tables
[ https://issues.apache.org/jira/browse/IMPALA-4052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Smith resolved IMPALA-4052. --- Fix Version/s: Impala 4.3.0 Resolution: Fixed > CREATE TABLE LIKE for Kudu tables > - > > Key: IMPALA-4052 > URL: https://issues.apache.org/jira/browse/IMPALA-4052 > Project: IMPALA > Issue Type: Improvement > Components: Catalog >Affects Versions: Impala 2.7.0 >Reporter: Dimitris Tsirogiannis >Assignee: gaoxiaoqing >Priority: Major > Labels: kudu > Fix For: Impala 4.3.0 > > > The semantics of CREATE TABLE LIKE when Kudu tables are involved, either as a > source or as a target table, are not well specified or properly implemented; > in some cases a misleading ImpalaRuntimeException is thrown. > Actions: > # Decide whether CREATE TABLE LIKE will be supported for Kudu tables. > # Implement whatever approach is decided > # Properly document both in Impala and Kudu docs the supported operations -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Reopened] (IMPALA-4052) CREATE TABLE LIKE for Kudu tables
[ https://issues.apache.org/jira/browse/IMPALA-4052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Smith reopened IMPALA-4052: --- > CREATE TABLE LIKE for Kudu tables > - > > Key: IMPALA-4052 > URL: https://issues.apache.org/jira/browse/IMPALA-4052 > Project: IMPALA > Issue Type: Improvement > Components: Catalog >Affects Versions: Impala 2.7.0 >Reporter: Dimitris Tsirogiannis >Assignee: gaoxiaoqing >Priority: Major > Labels: kudu > Fix For: Impala 4.3.0 > > > The semantics of CREATE TABLE LIKE when Kudu tables are involved, either as a > source or as a target table, are not well specified or properly implemented; > in some cases a misleading ImpalaRuntimeException is thrown. > Actions: > # Decide whether CREATE TABLE LIKE will be supported for Kudu tables. > # Implement whatever approach is decided > # Properly document both in Impala and Kudu docs the supported operations -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-4052) CREATE TABLE LIKE for Kudu tables
[ https://issues.apache.org/jira/browse/IMPALA-4052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17695853#comment-17695853 ] Michael Smith commented on IMPALA-4052: --- Oh, I guess we don't have docs for this yet. > CREATE TABLE LIKE for Kudu tables > - > > Key: IMPALA-4052 > URL: https://issues.apache.org/jira/browse/IMPALA-4052 > Project: IMPALA > Issue Type: Improvement > Components: Catalog >Affects Versions: Impala 2.7.0 >Reporter: Dimitris Tsirogiannis >Assignee: gaoxiaoqing >Priority: Major > Labels: kudu > Fix For: Impala 4.3.0 > > > The semantics of CREATE TABLE LIKE when Kudu tables are involved, either as a > source or as a target table, are not well specified or properly implemented; > in some cases a misleading ImpalaRuntimeException is thrown. > Actions: > # Decide whether CREATE TABLE LIKE will be supported for Kudu tables. > # Implement whatever approach is decided > # Properly document both in Impala and Kudu docs the supported operations -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-11633) Create table as select (CTAS) tests occasionally timeout for S3, Ozone
[
https://issues.apache.org/jira/browse/IMPALA-11633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Smith resolved IMPALA-11633.
Fix Version/s: Impala 4.3.0
Resolution: Fixed
> Create table as select (CTAS) tests occasionally timeout for S3, Ozone
> --
>
> Key: IMPALA-11633
> URL: https://issues.apache.org/jira/browse/IMPALA-11633
> Project: IMPALA
> Issue Type: Bug
>Reporter: Kurt Deschler
>Assignee: Michael Smith
>Priority: Major
> Fix For: Impala 4.3.0
>
>
> Several instances of this have been observed:
> * metadata.test_ddl.TestAsyncDDL.test_get_operation_status_for_async_ddl,
> where {{create table
> test_get_operation_status_for_async_ddl_f3067b3f.alltypes_clone as select *
> from functional_parquet.alltypes;}} takes more than the allowed 3 seconds
> * metadata/test_ddl.TestAsyncDDL.test_ctas times out in {{wait_for_state}}
> (20 seconds)
> * metadata.test_load.TestAsyncLoadData.test_async_load times out in
> {{wait_for_state}} (10 seconds)
> Need to check whether there are specific problems, or S3 and Ozone are
> sufficiently slower that the doubled timing for HDFS is insufficient to allow
> for environmental load.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-11633) Create table as select (CTAS) tests occasionally timeout for S3, Ozone
[
https://issues.apache.org/jira/browse/IMPALA-11633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17695916#comment-17695916
]
ASF subversion and git services commented on IMPALA-11633:
--
Commit 6ebf35cd5d9d8762147b73bd99b53fd2ffc426d3 in impala's branch
refs/heads/master from Michael Smith
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=6ebf35cd5 ]
IMPALA-11633: Use longer ctas timeout for S3/Ozone
File reads can be slower with S3 and Ozone in minicluster because
they're not using shortcircuiting to directly access the file on disk.
Allow more time for CTAS operations to complete when using a non-HDFS
filesystem.
Change-Id: I7a49cd44fc0fdc238313b92341467818a8f37fd1
Reviewed-on: http://gerrit.cloudera.org:8080/19497
Reviewed-by: Andrew Sherman
Tested-by: Impala Public Jenkins
> Create table as select (CTAS) tests occasionally timeout for S3, Ozone
> --
>
> Key: IMPALA-11633
> URL: https://issues.apache.org/jira/browse/IMPALA-11633
> Project: IMPALA
> Issue Type: Bug
>Reporter: Kurt Deschler
>Assignee: Michael Smith
>Priority: Major
> Fix For: Impala 4.3.0
>
>
> Several instances of this have been observed:
> * metadata.test_ddl.TestAsyncDDL.test_get_operation_status_for_async_ddl,
> where {{create table
> test_get_operation_status_for_async_ddl_f3067b3f.alltypes_clone as select *
> from functional_parquet.alltypes;}} takes more than the allowed 3 seconds
> * metadata/test_ddl.TestAsyncDDL.test_ctas times out in {{wait_for_state}}
> (20 seconds)
> * metadata.test_load.TestAsyncLoadData.test_async_load times out in
> {{wait_for_state}} (10 seconds)
> Need to check whether there are specific problems, or S3 and Ozone are
> sufficiently slower that the doubled timing for HDFS is insufficient to allow
> for environmental load.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-11966) Enable cache_ozone_file_handles by default
David Rorke created IMPALA-11966: Summary: Enable cache_ozone_file_handles by default Key: IMPALA-11966 URL: https://issues.apache.org/jira/browse/IMPALA-11966 Project: IMPALA Issue Type: Sub-task Components: be Reporter: David Rorke Assignee: Michael Smith Ozone file handle caching support was added in IMPALA-10214 but disabled by default initially. Subsequent performance testing with TPC-DS at 10TB scale has shown that enabling Ozone file handle caching improves TPC-DS geomean query time by 10%. The cache_ozone_file_handles parameter should be enabled by default. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Work started] (IMPALA-11966) Enable cache_ozone_file_handles by default
[ https://issues.apache.org/jira/browse/IMPALA-11966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on IMPALA-11966 started by Michael Smith. -- > Enable cache_ozone_file_handles by default > -- > > Key: IMPALA-11966 > URL: https://issues.apache.org/jira/browse/IMPALA-11966 > Project: IMPALA > Issue Type: Sub-task > Components: be >Reporter: David Rorke >Assignee: Michael Smith >Priority: Major > > Ozone file handle caching support was added in IMPALA-10214 but disabled by > default initially. Subsequent performance testing with TPC-DS at 10TB scale > has shown that enabling Ozone file handle caching improves TPC-DS geomean > query time by 10%. The cache_ozone_file_handles parameter should be enabled > by default. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-11966) Enable cache_ozone_file_handles by default
[ https://issues.apache.org/jira/browse/IMPALA-11966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17695930#comment-17695930 ] Michael Smith commented on IMPALA-11966: Ozone file handle caching was disabled by default partially due to HDDS-7135. That has been fixed in Ozone 1.3.0. > Enable cache_ozone_file_handles by default > -- > > Key: IMPALA-11966 > URL: https://issues.apache.org/jira/browse/IMPALA-11966 > Project: IMPALA > Issue Type: Sub-task > Components: be >Reporter: David Rorke >Assignee: Michael Smith >Priority: Major > > Ozone file handle caching support was added in IMPALA-10214 but disabled by > default initially. Subsequent performance testing with TPC-DS at 10TB scale > has shown that enabling Ozone file handle caching improves TPC-DS geomean > query time by 10%. The cache_ozone_file_handles parameter should be enabled > by default. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-11966) Enable cache_ozone_file_handles by default
[ https://issues.apache.org/jira/browse/IMPALA-11966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Smith updated IMPALA-11966: --- Parent: (was: IMPALA-9400) Issue Type: Improvement (was: Sub-task) > Enable cache_ozone_file_handles by default > -- > > Key: IMPALA-11966 > URL: https://issues.apache.org/jira/browse/IMPALA-11966 > Project: IMPALA > Issue Type: Improvement > Components: be >Reporter: David Rorke >Assignee: Michael Smith >Priority: Major > > Ozone file handle caching support was added in IMPALA-10214 but disabled by > default initially. Subsequent performance testing with TPC-DS at 10TB scale > has shown that enabling Ozone file handle caching improves TPC-DS geomean > query time by 10%. The cache_ozone_file_handles parameter should be enabled > by default. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-11966) Enable cache_ozone_file_handles by default
[ https://issues.apache.org/jira/browse/IMPALA-11966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Smith updated IMPALA-11966: --- Labels: ozone (was: ) > Enable cache_ozone_file_handles by default > -- > > Key: IMPALA-11966 > URL: https://issues.apache.org/jira/browse/IMPALA-11966 > Project: IMPALA > Issue Type: Improvement > Components: be >Reporter: David Rorke >Assignee: Michael Smith >Priority: Major > Labels: ozone > > Ozone file handle caching support was added in IMPALA-10214 but disabled by > default initially. Subsequent performance testing with TPC-DS at 10TB scale > has shown that enabling Ozone file handle caching improves TPC-DS geomean > query time by 10%. The cache_ozone_file_handles parameter should be enabled > by default. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Reopened] (IMPALA-11754) Fixes for Impala+Ozone and erasure coding
[ https://issues.apache.org/jira/browse/IMPALA-11754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Smith reopened IMPALA-11754: > Fixes for Impala+Ozone and erasure coding > - > > Key: IMPALA-11754 > URL: https://issues.apache.org/jira/browse/IMPALA-11754 > Project: IMPALA > Issue Type: Epic >Reporter: Michael Smith >Assignee: Michael Smith >Priority: Major > Labels: ozone > Fix For: Impala 4.3.0 > > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-11966) Enable cache_ozone_file_handles by default
[ https://issues.apache.org/jira/browse/IMPALA-11966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Smith updated IMPALA-11966: --- Epic Link: IMPALA-11754 > Enable cache_ozone_file_handles by default > -- > > Key: IMPALA-11966 > URL: https://issues.apache.org/jira/browse/IMPALA-11966 > Project: IMPALA > Issue Type: Improvement > Components: be >Reporter: David Rorke >Assignee: Michael Smith >Priority: Major > Labels: ozone > > Ozone file handle caching support was added in IMPALA-10214 but disabled by > default initially. Subsequent performance testing with TPC-DS at 10TB scale > has shown that enabling Ozone file handle caching improves TPC-DS geomean > query time by 10%. The cache_ozone_file_handles parameter should be enabled > by default. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-11960) Incorrect expression rewrites of date and timestamp conjuncts
[
https://issues.apache.org/jira/browse/IMPALA-11960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Quanlong Huang updated IMPALA-11960:
Summary: Incorrect expression rewrites of date and timestamp conjuncts
(was: Incorrect expression rewrites of data and timestamp conjuncts)
> Incorrect expression rewrites of date and timestamp conjuncts
> -
>
> Key: IMPALA-11960
> URL: https://issues.apache.org/jira/browse/IMPALA-11960
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
>Reporter: Csaba Ringhofer
>Priority: Major
> Labels: correctness
>
> To reproduce:
> {code}
> create table t3 (b timestamp, ti date);
> insert into t3 values ("2023-01-01 09:00:00", "2023-01-01");
> enable_expr_rewrites=true;
> select * from t3 where ti = cast(b as date) and b <'2023-01-01 15:00:00';
> -- returned 0 row
> enable_expr_rewrites=false;
> select * from t3 where ti = cast(b as date) and b <'2023-01-01 15:00:00';
> -- returned 1 row
> {code}
> The cause seems to be that a conjunct like ti 15:00:00' as timestamp) as date) is created during
> https://github.com/apache/impala/blob/23c265d12804c91c08a08a0be92c155424ea3d99/fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java#L1860
> Casting the right side to date leads to truncating the time part and
> rejecting values during the given day.
> The issue was probably introduced by IMPALA-10064:
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-11967) Document that Compute Incremental Stats support specifying the column list
Quanlong Huang created IMPALA-11967:
---
Summary: Document that Compute Incremental Stats support
specifying the column list
Key: IMPALA-11967
URL: https://issues.apache.org/jira/browse/IMPALA-11967
Project: IMPALA
Issue Type: Documentation
Components: Docs
Reporter: Quanlong Huang
Assignee: shajini thayasingh
IMPALA-10435 adds the support for specifying the column list to collect stats
in COMPUTE INCREMENTAL STATS. We should update the syntax about this in
[https://impala.apache.org/docs/build/html/topics/impala_compute_stats.html]
{code:java}
COMPUTE INCREMENTAL STATS [db_name.]table_name [PARTITION (partition_spec)] [ (
column_list ) ]{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Work started] (IMPALA-11965) TestCodegenCache fails in exhaustive runs
[
https://issues.apache.org/jira/browse/IMPALA-11965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on IMPALA-11965 started by Yida Wu.
> TestCodegenCache fails in exhaustive runs
> -
>
> Key: IMPALA-11965
> URL: https://issues.apache.org/jira/browse/IMPALA-11965
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Reporter: Zoltán Borók-Nagy
>Assignee: Yida Wu
>Priority: Major
> Labels: broken-build
>
> TestCodegenCache fails with the following errors:
>
> {noformat}
> AssertionError: Could not find metric: impala.codegen-cache.entries-evicted
> AssertionError: Could not find metric: impala.codegen-cache.entries-in-use
> {noformat}
>
> Example stack trace:
> {noformat}
> custom_cluster/test_codegen_cache.py:89: in test_codegen_cache_date_string_col
> "select * from functional.alltypes where date_string_col != ''")
> custom_cluster/test_codegen_cache.py:141: in _test_codegen_cache
> self._check_metric_expect_init()
> custom_cluster/test_codegen_cache.py:131: in _check_metric_expect_init
> assert self.get_metric('impala.codegen-cache.entries-evicted') == 0
> common/impala_test_suite.py:509: in get_metric
> assert False, "Could not find metric: %s" % name
> E AssertionError: Could not find metric:
> impala.codegen-cache.entries-evicted{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
