[jira] [Resolved] (IMPALA-13214) test_shell_commandline..test_removed_query_option failed with assertion failure
[
https://issues.apache.org/jira/browse/IMPALA-13214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Smith resolved IMPALA-13214.
Fix Version/s: Impala 4.5.0
Resolution: Fixed
> test_shell_commandline..test_removed_query_option failed with assertion
> failure
> ---
>
> Key: IMPALA-13214
> URL: https://issues.apache.org/jira/browse/IMPALA-13214
> Project: IMPALA
> Issue Type: Bug
> Components: Test
>Affects Versions: Impala 4.5.0
>Reporter: Laszlo Gaal
>Assignee: Michael Smith
>Priority: Blocker
> Fix For: Impala 4.5.0
>
>
> Happened during a recent s3-arm-data-cache build.
> Python backtrace:
> {code}
> /data/jenkins/workspace/impala-asf-master-core-s3-arm-data-cache/repos/Impala/tests/shell/test_shell_commandline.py:305:
> in test_removed_query_option
> expect_success=True)
> shell/util.py:135: in run_impala_shell_cmd
> stderr_file=stderr_file)
> shell/util.py:155: in run_impala_shell_cmd_no_expect
> stdout_file=stdout_file, stderr_file=stderr_file)
> shell/util.py:271: in __init__
> "Impala shell exited with return code {0}".format(process_status)
> E AssertionError: Impala shell exited with return code 0
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-13214) test_shell_commandline..test_removed_query_option failed with assertion failure
[
https://issues.apache.org/jira/browse/IMPALA-13214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Smith updated IMPALA-13214:
---
Component/s: Test
(was: Clients)
> test_shell_commandline..test_removed_query_option failed with assertion
> failure
> ---
>
> Key: IMPALA-13214
> URL: https://issues.apache.org/jira/browse/IMPALA-13214
> Project: IMPALA
> Issue Type: Bug
> Components: Test
>Affects Versions: Impala 4.5.0
>Reporter: Laszlo Gaal
>Assignee: Michael Smith
>Priority: Blocker
>
> Happened during a recent s3-arm-data-cache build.
> Python backtrace:
> {code}
> /data/jenkins/workspace/impala-asf-master-core-s3-arm-data-cache/repos/Impala/tests/shell/test_shell_commandline.py:305:
> in test_removed_query_option
> expect_success=True)
> shell/util.py:135: in run_impala_shell_cmd
> stderr_file=stderr_file)
> shell/util.py:155: in run_impala_shell_cmd_no_expect
> stdout_file=stdout_file, stderr_file=stderr_file)
> shell/util.py:271: in __init__
> "Impala shell exited with return code {0}".format(process_status)
> E AssertionError: Impala shell exited with return code 0
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Work started] (IMPALA-13252) Filter update log message prints TUniqueId in non-standard format
[
https://issues.apache.org/jira/browse/IMPALA-13252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on IMPALA-13252 started by Michael Smith.
--
> Filter update log message prints TUniqueId in non-standard format
> -
>
> Key: IMPALA-13252
> URL: https://issues.apache.org/jira/browse/IMPALA-13252
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 4.4.0
>Reporter: Michael Smith
>Assignee: Michael Smith
>Priority: Major
>
> Some error messages, such as
> {code}
> Filter update received for non-executing query with id:
> TUniqueId(hi=-8482965541048796556, lo=3501357296473079808)
> {code}
> print query id as the raw Thrift type rather than our colon-delimited format,
> e.g. "8a4673c8fbe83a74:309751e9". This makes it difficult to trace
> queries through logs.
> Normalize on the colon-delimited format.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Assigned] (IMPALA-13252) Filter update log message prints TUniqueId in non-standard format
[
https://issues.apache.org/jira/browse/IMPALA-13252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Smith reassigned IMPALA-13252:
--
Assignee: Michael Smith
> Filter update log message prints TUniqueId in non-standard format
> -
>
> Key: IMPALA-13252
> URL: https://issues.apache.org/jira/browse/IMPALA-13252
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 4.4.0
>Reporter: Michael Smith
>Assignee: Michael Smith
>Priority: Major
>
> Some error messages, such as
> {code}
> Filter update received for non-executing query with id:
> TUniqueId(hi=-8482965541048796556, lo=3501357296473079808)
> {code}
> print query id as the raw Thrift type rather than our colon-delimited format,
> e.g. "8a4673c8fbe83a74:309751e9". This makes it difficult to trace
> queries through logs.
> Normalize on the colon-delimited format.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-13252) Filter update log message prints TUniqueId in non-standard format
Michael Smith created IMPALA-13252:
--
Summary: Filter update log message prints TUniqueId in
non-standard format
Key: IMPALA-13252
URL: https://issues.apache.org/jira/browse/IMPALA-13252
Project: IMPALA
Issue Type: Bug
Components: Backend
Affects Versions: Impala 4.4.0
Reporter: Michael Smith
Some error messages, such as
{code}
Filter update received for non-executing query with id:
TUniqueId(hi=-8482965541048796556, lo=3501357296473079808)
{code}
print query id as the raw Thrift type rather than our colon-delimited format,
e.g. "8a4673c8fbe83a74:309751e9". This makes it difficult to trace
queries through logs.
Normalize on the colon-delimited format.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-13243) Update Dropwizard Metrics to supported version
[ https://issues.apache.org/jira/browse/IMPALA-13243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Smith resolved IMPALA-13243. Fix Version/s: Impala 4.5.0 Resolution: Fixed > Update Dropwizard Metrics to supported version > -- > > Key: IMPALA-13243 > URL: https://issues.apache.org/jira/browse/IMPALA-13243 > Project: IMPALA > Issue Type: Task > Components: Frontend >Affects Versions: Impala 4.4.0 >Reporter: Michael Smith >Assignee: Michael Smith >Priority: Major > Fix For: Impala 4.5.0 > > > [Dropwizard Metrics|https://metrics.dropwizard.io] 4.1.x was [EOL in > 2023|https://github.com/dropwizard/metrics/discussions/3029]. Impala's still > on 3.x. Update to 4.2.x to keep up with bug and security fixes. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Assigned] (IMPALA-13250) Document ENABLED_RUNTIME_FILTER_TYPES query option
[ https://issues.apache.org/jira/browse/IMPALA-13250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Smith reassigned IMPALA-13250: -- Assignee: Fang-Yu Rao (was: Michael Smith) > Document ENABLED_RUNTIME_FILTER_TYPES query option > -- > > Key: IMPALA-13250 > URL: https://issues.apache.org/jira/browse/IMPALA-13250 > Project: IMPALA > Issue Type: Documentation >Affects Versions: Impala 4.0.0 >Reporter: Michael Smith >Assignee: Fang-Yu Rao >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-13250) Document ENABLED_RUNTIME_FILTER_TYPES query option
[ https://issues.apache.org/jira/browse/IMPALA-13250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Smith updated IMPALA-13250: --- Affects Version/s: Impala 4.0.0 > Document ENABLED_RUNTIME_FILTER_TYPES query option > -- > > Key: IMPALA-13250 > URL: https://issues.apache.org/jira/browse/IMPALA-13250 > Project: IMPALA > Issue Type: Documentation >Affects Versions: Impala 4.0.0 >Reporter: Michael Smith >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Assigned] (IMPALA-13250) Document ENABLED_RUNTIME_FILTER_TYPES query option
[ https://issues.apache.org/jira/browse/IMPALA-13250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Smith reassigned IMPALA-13250: -- Assignee: Michael Smith > Document ENABLED_RUNTIME_FILTER_TYPES query option > -- > > Key: IMPALA-13250 > URL: https://issues.apache.org/jira/browse/IMPALA-13250 > Project: IMPALA > Issue Type: Documentation >Affects Versions: Impala 4.0.0 >Reporter: Michael Smith >Assignee: Michael Smith >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-13250) Document ENABLED_RUNTIME_FILTER_TYPES query option
Michael Smith created IMPALA-13250: -- Summary: Document ENABLED_RUNTIME_FILTER_TYPES query option Key: IMPALA-13250 URL: https://issues.apache.org/jira/browse/IMPALA-13250 Project: IMPALA Issue Type: Documentation Reporter: Michael Smith -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Work stopped] (IMPALA-13199) CLUSTER_TESTS on JDK11 failing with not being able to restart HMS
[
https://issues.apache.org/jira/browse/IMPALA-13199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on IMPALA-13199 stopped by Michael Smith.
--
> CLUSTER_TESTS on JDK11 failing with not being able to restart HMS
> -
>
> Key: IMPALA-13199
> URL: https://issues.apache.org/jira/browse/IMPALA-13199
> Project: IMPALA
> Issue Type: Bug
>Affects Versions: Impala 4.5.0
>Reporter: Laszlo Gaal
>Assignee: Michael Smith
>Priority: Critical
>
> The weekly test run on JDK 11 has been failing at least since August. Most
> failures happen during custom cluster tests, where multiple attempts to
> restart HSM fail with the following symptoms:
> Python backtrace:
> {code}
> common/custom_cluster_test_suite.py:174: in setup_method
> self._start_hive_service(method.__dict__[HIVE_CONF_DIR])
> common/custom_cluster_test_suite.py:281: in _start_hive_service
> raise RuntimeError("Unable to start Hive")
> E RuntimeError: Unable to start Hive
> {code}
> Console log:
> {code}
> 08:28:30.546 --- Captured stderr teardown
> ---
> 08:28:30.546 -- closing connection to: localhost:21000
> 08:28:30.546 -- closing connection to: localhost:21050
> 08:28:30.546 -- closing connection to: localhost:28000
> 08:28:30.546 Picked up JAVA_TOOL_OPTIONS:
> --add-opens=java.base/java.io=ALL-UNNAMED
> --add-opens=java.base/java.lang.invoke=ALL-UNNAMED
> --add-opens=java.base/java.lang.module=ALL-UNNAMED
> --add-opens=java.base/java.lang.ref=ALL-UNNAMED
> --add-opens=java.base/java.lang.reflect=ALL-UNNAMED
> --add-opens=java.base/java.lang=ALL-UNNAMED
> --add-opens=java.base/java.net=ALL-UNNAMED
> --add-opens=java.base/java.nio.charset=ALL-UNNAMED
> --add-opens=java.base/java.nio.file.attribute=ALL-UNNAMED
> --add-opens=java.base/java.nio=ALL-UNNAMED
> --add-opens=java.base/java.security=ALL-UNNAMED
> --add-opens=java.base/java.util.concurrent.locks=ALL-UNNAMED
> --add-opens=java.base/java.util.concurrent=ALL-UNNAMED
> --add-opens=java.base/java.util.jar=ALL-UNNAMED
> --add-opens=java.base/java.util.regex=ALL-UNNAMED
> --add-opens=java.base/java.util.zip=ALL-UNNAMED
> --add-opens=java.base/java.util=ALL-UNNAMED
> --add-opens=java.base/jdk.internal.loader=ALL-UNNAMED
> --add-opens=java.base/jdk.internal.math=ALL-UNNAMED
> --add-opens=java.base/jdk.internal.module=ALL-UNNAMED
> --add-opens=java.base/jdk.internal.perf=ALL-UNNAMED
> --add-opens=java.base/jdk.internal.platform=ALL-UNNAMED
> --add-opens=java.base/jdk.internal.platform.cgroupv1=ALL-UNNAMED
> --add-opens=java.base/jdk.internal.ref=ALL-UNNAMED
> --add-opens=java.base/jdk.internal.reflect=ALL-UNNAMED
> --add-opens=java.base/jdk.internal.util.jar=ALL-UNNAMED
> --add-opens=java.base/sun.net.www.protocol.jar=ALL-UNNAMED
> --add-opens=java.base/sun.nio.ch=ALL-UNNAMED
> --add-opens=java.base/sun.nio.fs=ALL-UNNAMED
> --add-opens=jdk.dynalink/jdk.dynalink.beans=ALL-UNNAMED
> --add-opens=jdk.dynalink/jdk.dynalink.linker.support=ALL-UNNAMED
> --add-opens=jdk.dynalink/jdk.dynalink.linker=ALL-UNNAMED
> --add-opens=jdk.dynalink/jdk.dynalink.support=ALL-UNNAMED
> --add-opens=jdk.dynalink/jdk.dynalink=ALL-UNNAMED
> --add-opens=jdk.management.jfr/jdk.management.jfr=ALL-UNNAMED
> --add-opens=jdk.management/com.sun.management.internal=ALL-UNNAMED
> -javaagent:/home/ubuntu/Impala/fe/target/dependency/jamm-0.4.0.jar
> 08:28:30.546 WARNING: Unknown module: jdk.dynalink specified to --add-opens
> 08:28:30.546 WARNING: Unknown module: jdk.dynalink specified to --add-opens
> 08:28:30.546 WARNING: Unknown module: jdk.dynalink specified to --add-opens
> 08:28:30.546 WARNING: Unknown module: jdk.dynalink specified to --add-opens
> 08:28:30.546 WARNING: Unknown module: jdk.dynalink specified to --add-opens
> 08:28:30.546 Exception in thread "main" java.lang.NoClassDefFoundError:
> sun/misc/Unsafe
> 08:28:30.546 at org.github.jamm.VM.loadUnsafe(VM.java:213)
> 08:28:30.546 at org.github.jamm.VM.(VM.java:26)
> 08:28:30.546 at
> org.github.jamm.accessors.FieldAccessor.newInstance(FieldAccessor.java:36)
> 08:28:30.546 at org.github.jamm.MemoryMeter.(MemoryMeter.java:184)
> 08:28:30.546 at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 08:28:30.546 at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> 08:28:30.546 at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 08:28:30.546 at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> 08:28:30.546 at
> java.instrument/sun.instrument.InstrumentationImpl.loadClassAndStartAgent(InstrumentationImpl.java:513)
> 08:28:30.546 at
>
[jira] [Updated] (IMPALA-13199) CLUSTER_TESTS on JDK11 failing with not being able to restart HMS
[
https://issues.apache.org/jira/browse/IMPALA-13199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Smith updated IMPALA-13199:
---
Priority: Major (was: Critical)
> CLUSTER_TESTS on JDK11 failing with not being able to restart HMS
> -
>
> Key: IMPALA-13199
> URL: https://issues.apache.org/jira/browse/IMPALA-13199
> Project: IMPALA
> Issue Type: Bug
>Affects Versions: Impala 4.5.0
>Reporter: Laszlo Gaal
>Assignee: Michael Smith
>Priority: Major
>
> The weekly test run on JDK 11 has been failing at least since August. Most
> failures happen during custom cluster tests, where multiple attempts to
> restart HSM fail with the following symptoms:
> Python backtrace:
> {code}
> common/custom_cluster_test_suite.py:174: in setup_method
> self._start_hive_service(method.__dict__[HIVE_CONF_DIR])
> common/custom_cluster_test_suite.py:281: in _start_hive_service
> raise RuntimeError("Unable to start Hive")
> E RuntimeError: Unable to start Hive
> {code}
> Console log:
> {code}
> 08:28:30.546 --- Captured stderr teardown
> ---
> 08:28:30.546 -- closing connection to: localhost:21000
> 08:28:30.546 -- closing connection to: localhost:21050
> 08:28:30.546 -- closing connection to: localhost:28000
> 08:28:30.546 Picked up JAVA_TOOL_OPTIONS:
> --add-opens=java.base/java.io=ALL-UNNAMED
> --add-opens=java.base/java.lang.invoke=ALL-UNNAMED
> --add-opens=java.base/java.lang.module=ALL-UNNAMED
> --add-opens=java.base/java.lang.ref=ALL-UNNAMED
> --add-opens=java.base/java.lang.reflect=ALL-UNNAMED
> --add-opens=java.base/java.lang=ALL-UNNAMED
> --add-opens=java.base/java.net=ALL-UNNAMED
> --add-opens=java.base/java.nio.charset=ALL-UNNAMED
> --add-opens=java.base/java.nio.file.attribute=ALL-UNNAMED
> --add-opens=java.base/java.nio=ALL-UNNAMED
> --add-opens=java.base/java.security=ALL-UNNAMED
> --add-opens=java.base/java.util.concurrent.locks=ALL-UNNAMED
> --add-opens=java.base/java.util.concurrent=ALL-UNNAMED
> --add-opens=java.base/java.util.jar=ALL-UNNAMED
> --add-opens=java.base/java.util.regex=ALL-UNNAMED
> --add-opens=java.base/java.util.zip=ALL-UNNAMED
> --add-opens=java.base/java.util=ALL-UNNAMED
> --add-opens=java.base/jdk.internal.loader=ALL-UNNAMED
> --add-opens=java.base/jdk.internal.math=ALL-UNNAMED
> --add-opens=java.base/jdk.internal.module=ALL-UNNAMED
> --add-opens=java.base/jdk.internal.perf=ALL-UNNAMED
> --add-opens=java.base/jdk.internal.platform=ALL-UNNAMED
> --add-opens=java.base/jdk.internal.platform.cgroupv1=ALL-UNNAMED
> --add-opens=java.base/jdk.internal.ref=ALL-UNNAMED
> --add-opens=java.base/jdk.internal.reflect=ALL-UNNAMED
> --add-opens=java.base/jdk.internal.util.jar=ALL-UNNAMED
> --add-opens=java.base/sun.net.www.protocol.jar=ALL-UNNAMED
> --add-opens=java.base/sun.nio.ch=ALL-UNNAMED
> --add-opens=java.base/sun.nio.fs=ALL-UNNAMED
> --add-opens=jdk.dynalink/jdk.dynalink.beans=ALL-UNNAMED
> --add-opens=jdk.dynalink/jdk.dynalink.linker.support=ALL-UNNAMED
> --add-opens=jdk.dynalink/jdk.dynalink.linker=ALL-UNNAMED
> --add-opens=jdk.dynalink/jdk.dynalink.support=ALL-UNNAMED
> --add-opens=jdk.dynalink/jdk.dynalink=ALL-UNNAMED
> --add-opens=jdk.management.jfr/jdk.management.jfr=ALL-UNNAMED
> --add-opens=jdk.management/com.sun.management.internal=ALL-UNNAMED
> -javaagent:/home/ubuntu/Impala/fe/target/dependency/jamm-0.4.0.jar
> 08:28:30.546 WARNING: Unknown module: jdk.dynalink specified to --add-opens
> 08:28:30.546 WARNING: Unknown module: jdk.dynalink specified to --add-opens
> 08:28:30.546 WARNING: Unknown module: jdk.dynalink specified to --add-opens
> 08:28:30.546 WARNING: Unknown module: jdk.dynalink specified to --add-opens
> 08:28:30.546 WARNING: Unknown module: jdk.dynalink specified to --add-opens
> 08:28:30.546 Exception in thread "main" java.lang.NoClassDefFoundError:
> sun/misc/Unsafe
> 08:28:30.546 at org.github.jamm.VM.loadUnsafe(VM.java:213)
> 08:28:30.546 at org.github.jamm.VM.(VM.java:26)
> 08:28:30.546 at
> org.github.jamm.accessors.FieldAccessor.newInstance(FieldAccessor.java:36)
> 08:28:30.546 at org.github.jamm.MemoryMeter.(MemoryMeter.java:184)
> 08:28:30.546 at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 08:28:30.546 at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> 08:28:30.546 at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 08:28:30.546 at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> 08:28:30.546 at
> java.instrument/sun.instrument.InstrumentationImpl.loadClassAndStartAgent(InstrumentationImpl.java:513)
> 08:28:30.546 at
>
[jira] [Commented] (IMPALA-13199) CLUSTER_TESTS on JDK11 failing with not being able to restart HMS
[
https://issues.apache.org/jira/browse/IMPALA-13199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17867122#comment-17867122
]
Michael Smith commented on IMPALA-13199:
So far I'm unable to reproduce this outside the Jenkins environment. Not sure
what I'm overlooking. For now I've added an Ubuntu 22.04 job and switched to
running that regularly.
> CLUSTER_TESTS on JDK11 failing with not being able to restart HMS
> -
>
> Key: IMPALA-13199
> URL: https://issues.apache.org/jira/browse/IMPALA-13199
> Project: IMPALA
> Issue Type: Bug
>Affects Versions: Impala 4.5.0
>Reporter: Laszlo Gaal
>Assignee: Michael Smith
>Priority: Critical
>
> The weekly test run on JDK 11 has been failing at least since August. Most
> failures happen during custom cluster tests, where multiple attempts to
> restart HSM fail with the following symptoms:
> Python backtrace:
> {code}
> common/custom_cluster_test_suite.py:174: in setup_method
> self._start_hive_service(method.__dict__[HIVE_CONF_DIR])
> common/custom_cluster_test_suite.py:281: in _start_hive_service
> raise RuntimeError("Unable to start Hive")
> E RuntimeError: Unable to start Hive
> {code}
> Console log:
> {code}
> 08:28:30.546 --- Captured stderr teardown
> ---
> 08:28:30.546 -- closing connection to: localhost:21000
> 08:28:30.546 -- closing connection to: localhost:21050
> 08:28:30.546 -- closing connection to: localhost:28000
> 08:28:30.546 Picked up JAVA_TOOL_OPTIONS:
> --add-opens=java.base/java.io=ALL-UNNAMED
> --add-opens=java.base/java.lang.invoke=ALL-UNNAMED
> --add-opens=java.base/java.lang.module=ALL-UNNAMED
> --add-opens=java.base/java.lang.ref=ALL-UNNAMED
> --add-opens=java.base/java.lang.reflect=ALL-UNNAMED
> --add-opens=java.base/java.lang=ALL-UNNAMED
> --add-opens=java.base/java.net=ALL-UNNAMED
> --add-opens=java.base/java.nio.charset=ALL-UNNAMED
> --add-opens=java.base/java.nio.file.attribute=ALL-UNNAMED
> --add-opens=java.base/java.nio=ALL-UNNAMED
> --add-opens=java.base/java.security=ALL-UNNAMED
> --add-opens=java.base/java.util.concurrent.locks=ALL-UNNAMED
> --add-opens=java.base/java.util.concurrent=ALL-UNNAMED
> --add-opens=java.base/java.util.jar=ALL-UNNAMED
> --add-opens=java.base/java.util.regex=ALL-UNNAMED
> --add-opens=java.base/java.util.zip=ALL-UNNAMED
> --add-opens=java.base/java.util=ALL-UNNAMED
> --add-opens=java.base/jdk.internal.loader=ALL-UNNAMED
> --add-opens=java.base/jdk.internal.math=ALL-UNNAMED
> --add-opens=java.base/jdk.internal.module=ALL-UNNAMED
> --add-opens=java.base/jdk.internal.perf=ALL-UNNAMED
> --add-opens=java.base/jdk.internal.platform=ALL-UNNAMED
> --add-opens=java.base/jdk.internal.platform.cgroupv1=ALL-UNNAMED
> --add-opens=java.base/jdk.internal.ref=ALL-UNNAMED
> --add-opens=java.base/jdk.internal.reflect=ALL-UNNAMED
> --add-opens=java.base/jdk.internal.util.jar=ALL-UNNAMED
> --add-opens=java.base/sun.net.www.protocol.jar=ALL-UNNAMED
> --add-opens=java.base/sun.nio.ch=ALL-UNNAMED
> --add-opens=java.base/sun.nio.fs=ALL-UNNAMED
> --add-opens=jdk.dynalink/jdk.dynalink.beans=ALL-UNNAMED
> --add-opens=jdk.dynalink/jdk.dynalink.linker.support=ALL-UNNAMED
> --add-opens=jdk.dynalink/jdk.dynalink.linker=ALL-UNNAMED
> --add-opens=jdk.dynalink/jdk.dynalink.support=ALL-UNNAMED
> --add-opens=jdk.dynalink/jdk.dynalink=ALL-UNNAMED
> --add-opens=jdk.management.jfr/jdk.management.jfr=ALL-UNNAMED
> --add-opens=jdk.management/com.sun.management.internal=ALL-UNNAMED
> -javaagent:/home/ubuntu/Impala/fe/target/dependency/jamm-0.4.0.jar
> 08:28:30.546 WARNING: Unknown module: jdk.dynalink specified to --add-opens
> 08:28:30.546 WARNING: Unknown module: jdk.dynalink specified to --add-opens
> 08:28:30.546 WARNING: Unknown module: jdk.dynalink specified to --add-opens
> 08:28:30.546 WARNING: Unknown module: jdk.dynalink specified to --add-opens
> 08:28:30.546 WARNING: Unknown module: jdk.dynalink specified to --add-opens
> 08:28:30.546 Exception in thread "main" java.lang.NoClassDefFoundError:
> sun/misc/Unsafe
> 08:28:30.546 at org.github.jamm.VM.loadUnsafe(VM.java:213)
> 08:28:30.546 at org.github.jamm.VM.(VM.java:26)
> 08:28:30.546 at
> org.github.jamm.accessors.FieldAccessor.newInstance(FieldAccessor.java:36)
> 08:28:30.546 at org.github.jamm.MemoryMeter.(MemoryMeter.java:184)
> 08:28:30.546 at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 08:28:30.546 at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> 08:28:30.546 at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 08:28:30.546 at java.base/java.lang.reflect.Method.invoke(Method.java:566)
>
[jira] [Created] (IMPALA-13243) Update Dropwizard Metrics to supported version
Michael Smith created IMPALA-13243: -- Summary: Update Dropwizard Metrics to supported version Key: IMPALA-13243 URL: https://issues.apache.org/jira/browse/IMPALA-13243 Project: IMPALA Issue Type: Task Components: Frontend Affects Versions: Impala 4.4.0 Reporter: Michael Smith Assignee: Michael Smith [Dropwizard Metrics|https://metrics.dropwizard.io] 4.1.x was [EOL in 2023|https://github.com/dropwizard/metrics/discussions/3029]. Impala's still on 3.x. Update to 4.2.x to keep up with bug and security fixes. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Work started] (IMPALA-13243) Update Dropwizard Metrics to supported version
[ https://issues.apache.org/jira/browse/IMPALA-13243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on IMPALA-13243 started by Michael Smith. -- > Update Dropwizard Metrics to supported version > -- > > Key: IMPALA-13243 > URL: https://issues.apache.org/jira/browse/IMPALA-13243 > Project: IMPALA > Issue Type: Task > Components: Frontend >Affects Versions: Impala 4.4.0 >Reporter: Michael Smith >Assignee: Michael Smith >Priority: Major > > [Dropwizard Metrics|https://metrics.dropwizard.io] 4.1.x was [EOL in > 2023|https://github.com/dropwizard/metrics/discussions/3029]. Impala's still > on 3.x. Update to 4.2.x to keep up with bug and security fixes. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-11026) [Build] Maven Dependencies in S3 Bucket fail to Download
[
https://issues.apache.org/jira/browse/IMPALA-11026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17867115#comment-17867115
]
Michael Smith commented on IMPALA-11026:
This is probably no longer relevant, but I'd look at any Maven settings.xml you
have. All dependencies are available on https://mvnrepository.com/, and that's
what our https://jenkins.impala.io/ builds use.
> [Build] Maven Dependencies in S3 Bucket fail to Download
>
>
> Key: IMPALA-11026
> URL: https://issues.apache.org/jira/browse/IMPALA-11026
> Project: IMPALA
> Issue Type: Improvement
>Affects Versions: Impala 4.0.0
> Environment: Impala 4.0.0 from Git & Archive, CentOS 7.9.2009
>Reporter: Christian Boenning
>Priority: Minor
>
> I‘ve been trying to build Apache Impala 4.0.0 (and master) over the course of
> the last few days from Source. Unfortunately I fail succeeding when it comes
> to the build some Java Components. Most of those Components from the „native
> toolchain“ bucket succeed to be downloaded and to be built, but „everything
> Java“ fails.
> Maven Artifacts from repository.cloudera.com download just fine, but the
> [https://native-toolchain.s3.amazonaws.com/build/cdp_components/11920537/maven]
> remains to be unavailable for me thus making (at least) the „Apache Impala
> Query Engine Frontend“ Maven Bits & Pieces to fail.
>
> Full Error:
> {noformat}
> [ERROR] Failed to execute goal on project impala-frontend: Could not resolve
> dependencies for project
> org.apache.impala:impala-frontend:jar:4.0.0-SNAPSHOT: Failed to collect
> dependencies for [net.minidev:json-smart:jar:2.3 (compile),
> org.apache.impala:impala-executor-deps:pom:4.0.0-SNAPSHOT (compile),
> org.apache.impala:query-event-hook-api:jar:4.0.0-SNAPSHOT (compile),
> org.apache.impala:impala-data-source-api:jar:4.0.0-SNAPSHOT (compile),
> org.apache.hadoop:hadoop-auth:jar:3.1.1.7.2.9.0-146 (compile),
> org.apache.hudi:hudi-hadoop-mr:jar:0.5.0-incubating (compile),
> org.apache.ranger:ranger-plugins-common:jar:2.1.0.7.2.9.0-146 (compile),
> org.apache.ranger:ranger-plugins-audit:jar:2.1.0.7.2.9.0-146 (compile),
> javax.mail:mail:jar:1.4 (compile), javax.ws.rs:javax.ws.rs-api:jar:2.1.1
> (compile), org.apache.impala:yarn-extras:jar:4.0.0-SNAPSHOT (compile),
> org.apache.parquet:parquet-hadoop-bundle:jar:1.10.99.7.2.9.0-146 (compile),
> org.apache.avro:avro:jar:1.8.2.7.2.9.0-146 (compile),
> org.apache.orc:orc-core:jar:1.6.2 (compile),
> org.apache.hbase:hbase-protocol:jar:2.2.6.7.2.9.0-146 (compile),
> commons-lang:commons-lang:jar:2.6 (compile),
> net.sourceforge.czt.dev:java-cup:jar:0.11-a-czt02-cdh (compile),
> org.apache.thrift:libthrift:jar:0.11.0 (compile),
> org.apache.httpcomponents:httpcore:jar:4.4.9 (compile),
> org.apache.kudu:kudu-client:jar:b5e7362e69 (compile),
> org.postgresql:postgresql:jar:42.2.14 (compile),
> org.antlr:antlr-runtime:jar:3.3 (compile), commons-cli:commons-cli:jar:1.2
> (compile), org.slf4j:slf4j-api:jar:1.7.30 (compile),
> org.slf4j:slf4j-log4j12:jar:1.7.30 (compile),
> com.google.guava:guava:jar:28.1-jre (compile),
> org.apache.derby:derby:jar:10.14.2.0 (compile),
> com.google.errorprone:error_prone_annotations:jar:2.3.1 (compile),
> junit:junit:jar:4.12 (test), org.ehcache:sizeof:jar:0.3.0 (compile),
> com.googlecode.json-simple:json-simple:jar:1.1.1 (compile),
> org.glassfish:javax.json:jar:1.0.2 (compile),
> com.github.davidmoten:flatbuffers-java:jar:1.6.0.1 (compile),
> io.dropwizard.metrics:metrics-core:jar:3.2.2 (compile),
> com.fasterxml.jackson.core:jackson-databind:jar:2.10.5.1 (compile),
> org.mockito:mockito-core:jar:2.23.4 (test),
> org.apache.directory.server:apacheds-test-framework:jar:2.0.0.AM25 (test),
> org.hamcrest:hamcrest-all:jar:1.3 (test),
> org.apache.iceberg:iceberg-api:jar:0.9.1.7.2.9.0-146 (compile),
> org.apache.iceberg:iceberg-hive-runtime:jar:0.9.1.7.2.9.0-146 (compile),
> org.apache.hive:hive-jdbc:jar:3.1.3000.7.2.9.0-146 (test),
> org.apache.hive:hive-classification:jar:3.1.3000.7.2.9.0-146 (compile),
> org.apache.hive:hive-standalone-metastore:jar:3.1.3000.7.2.9.0-146 (compile),
> org.apache.hadoop:hadoop-mapreduce-client-core:jar:3.1.1.7.2.9.0-146
> (compile), org.datanucleus:javax.jdo:jar:3.2.0-m3 (test),
> org.pac4j:pac4j-saml-opensamlv3:jar:4.0.3 (compile),
> org.bouncycastle:bcprov-jdk15on:jar:1.64 (compile),
> org.apache.santuario:xmlsec:jar:2.2.1 (compile),
> org.springframework:spring-core:jar:4.3.29.RELEASE (compile)]: Failed to read
> artifact descriptor for org.jamon:jamon-runtime:jar:2.3.1: Could not transfer
> artifact org.jamon:jamon-runtime:pom:2.3.1 from/to impala.cdp.repo
> (https://native-toolchain.s3.amazonaws.com/build/cdp_components/11920537/maven):
> Access denied to:
>
[jira] [Closed] (IMPALA-11026) [Build] Maven Dependencies in S3 Bucket fail to Download
[
https://issues.apache.org/jira/browse/IMPALA-11026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Smith closed IMPALA-11026.
--
Resolution: Information Provided
> [Build] Maven Dependencies in S3 Bucket fail to Download
>
>
> Key: IMPALA-11026
> URL: https://issues.apache.org/jira/browse/IMPALA-11026
> Project: IMPALA
> Issue Type: Improvement
>Affects Versions: Impala 4.0.0
> Environment: Impala 4.0.0 from Git & Archive, CentOS 7.9.2009
>Reporter: Christian Boenning
>Priority: Minor
>
> I‘ve been trying to build Apache Impala 4.0.0 (and master) over the course of
> the last few days from Source. Unfortunately I fail succeeding when it comes
> to the build some Java Components. Most of those Components from the „native
> toolchain“ bucket succeed to be downloaded and to be built, but „everything
> Java“ fails.
> Maven Artifacts from repository.cloudera.com download just fine, but the
> [https://native-toolchain.s3.amazonaws.com/build/cdp_components/11920537/maven]
> remains to be unavailable for me thus making (at least) the „Apache Impala
> Query Engine Frontend“ Maven Bits & Pieces to fail.
>
> Full Error:
> {noformat}
> [ERROR] Failed to execute goal on project impala-frontend: Could not resolve
> dependencies for project
> org.apache.impala:impala-frontend:jar:4.0.0-SNAPSHOT: Failed to collect
> dependencies for [net.minidev:json-smart:jar:2.3 (compile),
> org.apache.impala:impala-executor-deps:pom:4.0.0-SNAPSHOT (compile),
> org.apache.impala:query-event-hook-api:jar:4.0.0-SNAPSHOT (compile),
> org.apache.impala:impala-data-source-api:jar:4.0.0-SNAPSHOT (compile),
> org.apache.hadoop:hadoop-auth:jar:3.1.1.7.2.9.0-146 (compile),
> org.apache.hudi:hudi-hadoop-mr:jar:0.5.0-incubating (compile),
> org.apache.ranger:ranger-plugins-common:jar:2.1.0.7.2.9.0-146 (compile),
> org.apache.ranger:ranger-plugins-audit:jar:2.1.0.7.2.9.0-146 (compile),
> javax.mail:mail:jar:1.4 (compile), javax.ws.rs:javax.ws.rs-api:jar:2.1.1
> (compile), org.apache.impala:yarn-extras:jar:4.0.0-SNAPSHOT (compile),
> org.apache.parquet:parquet-hadoop-bundle:jar:1.10.99.7.2.9.0-146 (compile),
> org.apache.avro:avro:jar:1.8.2.7.2.9.0-146 (compile),
> org.apache.orc:orc-core:jar:1.6.2 (compile),
> org.apache.hbase:hbase-protocol:jar:2.2.6.7.2.9.0-146 (compile),
> commons-lang:commons-lang:jar:2.6 (compile),
> net.sourceforge.czt.dev:java-cup:jar:0.11-a-czt02-cdh (compile),
> org.apache.thrift:libthrift:jar:0.11.0 (compile),
> org.apache.httpcomponents:httpcore:jar:4.4.9 (compile),
> org.apache.kudu:kudu-client:jar:b5e7362e69 (compile),
> org.postgresql:postgresql:jar:42.2.14 (compile),
> org.antlr:antlr-runtime:jar:3.3 (compile), commons-cli:commons-cli:jar:1.2
> (compile), org.slf4j:slf4j-api:jar:1.7.30 (compile),
> org.slf4j:slf4j-log4j12:jar:1.7.30 (compile),
> com.google.guava:guava:jar:28.1-jre (compile),
> org.apache.derby:derby:jar:10.14.2.0 (compile),
> com.google.errorprone:error_prone_annotations:jar:2.3.1 (compile),
> junit:junit:jar:4.12 (test), org.ehcache:sizeof:jar:0.3.0 (compile),
> com.googlecode.json-simple:json-simple:jar:1.1.1 (compile),
> org.glassfish:javax.json:jar:1.0.2 (compile),
> com.github.davidmoten:flatbuffers-java:jar:1.6.0.1 (compile),
> io.dropwizard.metrics:metrics-core:jar:3.2.2 (compile),
> com.fasterxml.jackson.core:jackson-databind:jar:2.10.5.1 (compile),
> org.mockito:mockito-core:jar:2.23.4 (test),
> org.apache.directory.server:apacheds-test-framework:jar:2.0.0.AM25 (test),
> org.hamcrest:hamcrest-all:jar:1.3 (test),
> org.apache.iceberg:iceberg-api:jar:0.9.1.7.2.9.0-146 (compile),
> org.apache.iceberg:iceberg-hive-runtime:jar:0.9.1.7.2.9.0-146 (compile),
> org.apache.hive:hive-jdbc:jar:3.1.3000.7.2.9.0-146 (test),
> org.apache.hive:hive-classification:jar:3.1.3000.7.2.9.0-146 (compile),
> org.apache.hive:hive-standalone-metastore:jar:3.1.3000.7.2.9.0-146 (compile),
> org.apache.hadoop:hadoop-mapreduce-client-core:jar:3.1.1.7.2.9.0-146
> (compile), org.datanucleus:javax.jdo:jar:3.2.0-m3 (test),
> org.pac4j:pac4j-saml-opensamlv3:jar:4.0.3 (compile),
> org.bouncycastle:bcprov-jdk15on:jar:1.64 (compile),
> org.apache.santuario:xmlsec:jar:2.2.1 (compile),
> org.springframework:spring-core:jar:4.3.29.RELEASE (compile)]: Failed to read
> artifact descriptor for org.jamon:jamon-runtime:jar:2.3.1: Could not transfer
> artifact org.jamon:jamon-runtime:pom:2.3.1 from/to impala.cdp.repo
> (https://native-toolchain.s3.amazonaws.com/build/cdp_components/11920537/maven):
> Access denied to:
> https://native-toolchain.s3.amazonaws.com/build/cdp_components/11920537/maven/org/jamon/jamon-runtime/2.3.1/jamon-runtime-2.3.1.pom,
> ReasonPhrase: Forbidden. -> [Help 1]
> {noformat}
> I understand that Cloudera Impala might be restricted
[jira] [Commented] (IMPALA-13214) test_shell_commandline..test_removed_query_option failed with assertion failure
[
https://issues.apache.org/jira/browse/IMPALA-13214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17867074#comment-17867074
]
Michael Smith commented on IMPALA-13214:
I think what happened here is that {{impala-shell -q 'set
disable_cached_reads=true'}} ran too quickly for the ImpalaShell utility.
Unrelated to any recent code changes.
This test uses {{run_impala_shell_cmd}} to test that a set command generates a
warning. It does that by
# Creating a ImpalaShell object:
https://github.com/apache/impala/blob/4.4.0/tests/shell/util.py#L154
# which starts the new process:
https://github.com/apache/impala/blob/4.4.0/tests/shell/util.py#L230
# then waits for the connected message:
https://github.com/apache/impala/blob/4.4.0/tests/shell/util.py#L266
However it expects the process to still be running when it waits, and tests for
that via {{self.shell_process.poll()}}. When that returns a process result,
init considers that an error and asserts False. So we see
{code}
AssertionError: Impala shell exited with return code 0
{code}
but the return code is 0, meaning the process ran successfully.
I have some ideas for fixing up cases like this.
> test_shell_commandline..test_removed_query_option failed with assertion
> failure
> ---
>
> Key: IMPALA-13214
> URL: https://issues.apache.org/jira/browse/IMPALA-13214
> Project: IMPALA
> Issue Type: Bug
> Components: Clients
>Affects Versions: Impala 4.5.0
>Reporter: Laszlo Gaal
>Assignee: Michael Smith
>Priority: Blocker
>
> Happened during a recent s3-arm-data-cache build.
> Python backtrace:
> {code}
> /data/jenkins/workspace/impala-asf-master-core-s3-arm-data-cache/repos/Impala/tests/shell/test_shell_commandline.py:305:
> in test_removed_query_option
> expect_success=True)
> shell/util.py:135: in run_impala_shell_cmd
> stderr_file=stderr_file)
> shell/util.py:155: in run_impala_shell_cmd_no_expect
> stdout_file=stdout_file, stderr_file=stderr_file)
> shell/util.py:271: in __init__
> "Impala shell exited with return code {0}".format(process_status)
> E AssertionError: Impala shell exited with return code 0
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Work started] (IMPALA-13214) test_shell_commandline..test_removed_query_option failed with assertion failure
[
https://issues.apache.org/jira/browse/IMPALA-13214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on IMPALA-13214 started by Michael Smith.
--
> test_shell_commandline..test_removed_query_option failed with assertion
> failure
> ---
>
> Key: IMPALA-13214
> URL: https://issues.apache.org/jira/browse/IMPALA-13214
> Project: IMPALA
> Issue Type: Bug
> Components: Clients
>Affects Versions: Impala 4.5.0
>Reporter: Laszlo Gaal
>Assignee: Michael Smith
>Priority: Blocker
>
> Happened during a recent s3-arm-data-cache build.
> Python backtrace:
> {code}
> /data/jenkins/workspace/impala-asf-master-core-s3-arm-data-cache/repos/Impala/tests/shell/test_shell_commandline.py:305:
> in test_removed_query_option
> expect_success=True)
> shell/util.py:135: in run_impala_shell_cmd
> stderr_file=stderr_file)
> shell/util.py:155: in run_impala_shell_cmd_no_expect
> stdout_file=stdout_file, stderr_file=stderr_file)
> shell/util.py:271: in __init__
> "Impala shell exited with return code {0}".format(process_status)
> E AssertionError: Impala shell exited with return code 0
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Comment Edited] (IMPALA-13199) CLUSTER_TESTS on JDK11 failing with not being able to restart HMS
[
https://issues.apache.org/jira/browse/IMPALA-13199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17866873#comment-17866873
]
Michael Smith edited comment on IMPALA-13199 at 7/17/24 10:46 PM:
--
So far to reproduce this I need Ubuntu 20.04, and to run FE tests before a
failing suite like e.g. custom_cluster/test_catalog_hms_failures.py. Example:
https://jenkins.impala.io/job/ubuntu-20.04-from-scratch-java11/86/
was (Author: JIRAUSER288956):
So far to reproduce this I need Ubuntu 20.04, and to run FE tests before a
failing suite like e.g. custom_cluster/test_catalog_hms_failures.py.
> CLUSTER_TESTS on JDK11 failing with not being able to restart HMS
> -
>
> Key: IMPALA-13199
> URL: https://issues.apache.org/jira/browse/IMPALA-13199
> Project: IMPALA
> Issue Type: Bug
>Affects Versions: Impala 4.5.0
>Reporter: Laszlo Gaal
>Assignee: Michael Smith
>Priority: Critical
>
> The weekly test run on JDK 11 has been failing at least since August. Most
> failures happen during custom cluster tests, where multiple attempts to
> restart HSM fail with the following symptoms:
> Python backtrace:
> {code}
> common/custom_cluster_test_suite.py:174: in setup_method
> self._start_hive_service(method.__dict__[HIVE_CONF_DIR])
> common/custom_cluster_test_suite.py:281: in _start_hive_service
> raise RuntimeError("Unable to start Hive")
> E RuntimeError: Unable to start Hive
> {code}
> Console log:
> {code}
> 08:28:30.546 --- Captured stderr teardown
> ---
> 08:28:30.546 -- closing connection to: localhost:21000
> 08:28:30.546 -- closing connection to: localhost:21050
> 08:28:30.546 -- closing connection to: localhost:28000
> 08:28:30.546 Picked up JAVA_TOOL_OPTIONS:
> --add-opens=java.base/java.io=ALL-UNNAMED
> --add-opens=java.base/java.lang.invoke=ALL-UNNAMED
> --add-opens=java.base/java.lang.module=ALL-UNNAMED
> --add-opens=java.base/java.lang.ref=ALL-UNNAMED
> --add-opens=java.base/java.lang.reflect=ALL-UNNAMED
> --add-opens=java.base/java.lang=ALL-UNNAMED
> --add-opens=java.base/java.net=ALL-UNNAMED
> --add-opens=java.base/java.nio.charset=ALL-UNNAMED
> --add-opens=java.base/java.nio.file.attribute=ALL-UNNAMED
> --add-opens=java.base/java.nio=ALL-UNNAMED
> --add-opens=java.base/java.security=ALL-UNNAMED
> --add-opens=java.base/java.util.concurrent.locks=ALL-UNNAMED
> --add-opens=java.base/java.util.concurrent=ALL-UNNAMED
> --add-opens=java.base/java.util.jar=ALL-UNNAMED
> --add-opens=java.base/java.util.regex=ALL-UNNAMED
> --add-opens=java.base/java.util.zip=ALL-UNNAMED
> --add-opens=java.base/java.util=ALL-UNNAMED
> --add-opens=java.base/jdk.internal.loader=ALL-UNNAMED
> --add-opens=java.base/jdk.internal.math=ALL-UNNAMED
> --add-opens=java.base/jdk.internal.module=ALL-UNNAMED
> --add-opens=java.base/jdk.internal.perf=ALL-UNNAMED
> --add-opens=java.base/jdk.internal.platform=ALL-UNNAMED
> --add-opens=java.base/jdk.internal.platform.cgroupv1=ALL-UNNAMED
> --add-opens=java.base/jdk.internal.ref=ALL-UNNAMED
> --add-opens=java.base/jdk.internal.reflect=ALL-UNNAMED
> --add-opens=java.base/jdk.internal.util.jar=ALL-UNNAMED
> --add-opens=java.base/sun.net.www.protocol.jar=ALL-UNNAMED
> --add-opens=java.base/sun.nio.ch=ALL-UNNAMED
> --add-opens=java.base/sun.nio.fs=ALL-UNNAMED
> --add-opens=jdk.dynalink/jdk.dynalink.beans=ALL-UNNAMED
> --add-opens=jdk.dynalink/jdk.dynalink.linker.support=ALL-UNNAMED
> --add-opens=jdk.dynalink/jdk.dynalink.linker=ALL-UNNAMED
> --add-opens=jdk.dynalink/jdk.dynalink.support=ALL-UNNAMED
> --add-opens=jdk.dynalink/jdk.dynalink=ALL-UNNAMED
> --add-opens=jdk.management.jfr/jdk.management.jfr=ALL-UNNAMED
> --add-opens=jdk.management/com.sun.management.internal=ALL-UNNAMED
> -javaagent:/home/ubuntu/Impala/fe/target/dependency/jamm-0.4.0.jar
> 08:28:30.546 WARNING: Unknown module: jdk.dynalink specified to --add-opens
> 08:28:30.546 WARNING: Unknown module: jdk.dynalink specified to --add-opens
> 08:28:30.546 WARNING: Unknown module: jdk.dynalink specified to --add-opens
> 08:28:30.546 WARNING: Unknown module: jdk.dynalink specified to --add-opens
> 08:28:30.546 WARNING: Unknown module: jdk.dynalink specified to --add-opens
> 08:28:30.546 Exception in thread "main" java.lang.NoClassDefFoundError:
> sun/misc/Unsafe
> 08:28:30.546 at org.github.jamm.VM.loadUnsafe(VM.java:213)
> 08:28:30.546 at org.github.jamm.VM.(VM.java:26)
> 08:28:30.546 at
> org.github.jamm.accessors.FieldAccessor.newInstance(FieldAccessor.java:36)
> 08:28:30.546 at org.github.jamm.MemoryMeter.(MemoryMeter.java:184)
> 08:28:30.546 at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 08:28:30.546 at
>
[jira] [Commented] (IMPALA-13199) CLUSTER_TESTS on JDK11 failing with not being able to restart HMS
[
https://issues.apache.org/jira/browse/IMPALA-13199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17866873#comment-17866873
]
Michael Smith commented on IMPALA-13199:
So far to reproduce this I need Ubuntu 20.04, and to run FE tests before a
failing suite like e.g. custom_cluster/test_catalog_hms_failures.py.
> CLUSTER_TESTS on JDK11 failing with not being able to restart HMS
> -
>
> Key: IMPALA-13199
> URL: https://issues.apache.org/jira/browse/IMPALA-13199
> Project: IMPALA
> Issue Type: Bug
>Affects Versions: Impala 4.5.0
>Reporter: Laszlo Gaal
>Assignee: Michael Smith
>Priority: Critical
>
> The weekly test run on JDK 11 has been failing at least since August. Most
> failures happen during custom cluster tests, where multiple attempts to
> restart HSM fail with the following symptoms:
> Python backtrace:
> {code}
> common/custom_cluster_test_suite.py:174: in setup_method
> self._start_hive_service(method.__dict__[HIVE_CONF_DIR])
> common/custom_cluster_test_suite.py:281: in _start_hive_service
> raise RuntimeError("Unable to start Hive")
> E RuntimeError: Unable to start Hive
> {code}
> Console log:
> {code}
> 08:28:30.546 --- Captured stderr teardown
> ---
> 08:28:30.546 -- closing connection to: localhost:21000
> 08:28:30.546 -- closing connection to: localhost:21050
> 08:28:30.546 -- closing connection to: localhost:28000
> 08:28:30.546 Picked up JAVA_TOOL_OPTIONS:
> --add-opens=java.base/java.io=ALL-UNNAMED
> --add-opens=java.base/java.lang.invoke=ALL-UNNAMED
> --add-opens=java.base/java.lang.module=ALL-UNNAMED
> --add-opens=java.base/java.lang.ref=ALL-UNNAMED
> --add-opens=java.base/java.lang.reflect=ALL-UNNAMED
> --add-opens=java.base/java.lang=ALL-UNNAMED
> --add-opens=java.base/java.net=ALL-UNNAMED
> --add-opens=java.base/java.nio.charset=ALL-UNNAMED
> --add-opens=java.base/java.nio.file.attribute=ALL-UNNAMED
> --add-opens=java.base/java.nio=ALL-UNNAMED
> --add-opens=java.base/java.security=ALL-UNNAMED
> --add-opens=java.base/java.util.concurrent.locks=ALL-UNNAMED
> --add-opens=java.base/java.util.concurrent=ALL-UNNAMED
> --add-opens=java.base/java.util.jar=ALL-UNNAMED
> --add-opens=java.base/java.util.regex=ALL-UNNAMED
> --add-opens=java.base/java.util.zip=ALL-UNNAMED
> --add-opens=java.base/java.util=ALL-UNNAMED
> --add-opens=java.base/jdk.internal.loader=ALL-UNNAMED
> --add-opens=java.base/jdk.internal.math=ALL-UNNAMED
> --add-opens=java.base/jdk.internal.module=ALL-UNNAMED
> --add-opens=java.base/jdk.internal.perf=ALL-UNNAMED
> --add-opens=java.base/jdk.internal.platform=ALL-UNNAMED
> --add-opens=java.base/jdk.internal.platform.cgroupv1=ALL-UNNAMED
> --add-opens=java.base/jdk.internal.ref=ALL-UNNAMED
> --add-opens=java.base/jdk.internal.reflect=ALL-UNNAMED
> --add-opens=java.base/jdk.internal.util.jar=ALL-UNNAMED
> --add-opens=java.base/sun.net.www.protocol.jar=ALL-UNNAMED
> --add-opens=java.base/sun.nio.ch=ALL-UNNAMED
> --add-opens=java.base/sun.nio.fs=ALL-UNNAMED
> --add-opens=jdk.dynalink/jdk.dynalink.beans=ALL-UNNAMED
> --add-opens=jdk.dynalink/jdk.dynalink.linker.support=ALL-UNNAMED
> --add-opens=jdk.dynalink/jdk.dynalink.linker=ALL-UNNAMED
> --add-opens=jdk.dynalink/jdk.dynalink.support=ALL-UNNAMED
> --add-opens=jdk.dynalink/jdk.dynalink=ALL-UNNAMED
> --add-opens=jdk.management.jfr/jdk.management.jfr=ALL-UNNAMED
> --add-opens=jdk.management/com.sun.management.internal=ALL-UNNAMED
> -javaagent:/home/ubuntu/Impala/fe/target/dependency/jamm-0.4.0.jar
> 08:28:30.546 WARNING: Unknown module: jdk.dynalink specified to --add-opens
> 08:28:30.546 WARNING: Unknown module: jdk.dynalink specified to --add-opens
> 08:28:30.546 WARNING: Unknown module: jdk.dynalink specified to --add-opens
> 08:28:30.546 WARNING: Unknown module: jdk.dynalink specified to --add-opens
> 08:28:30.546 WARNING: Unknown module: jdk.dynalink specified to --add-opens
> 08:28:30.546 Exception in thread "main" java.lang.NoClassDefFoundError:
> sun/misc/Unsafe
> 08:28:30.546 at org.github.jamm.VM.loadUnsafe(VM.java:213)
> 08:28:30.546 at org.github.jamm.VM.(VM.java:26)
> 08:28:30.546 at
> org.github.jamm.accessors.FieldAccessor.newInstance(FieldAccessor.java:36)
> 08:28:30.546 at org.github.jamm.MemoryMeter.(MemoryMeter.java:184)
> 08:28:30.546 at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 08:28:30.546 at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> 08:28:30.546 at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 08:28:30.546 at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> 08:28:30.546 at
>
[jira] [Updated] (IMPALA-13235) Integrate OTel C++ SDK into Impala Backend
[ https://issues.apache.org/jira/browse/IMPALA-13235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Smith updated IMPALA-13235: --- Component/s: Toolchain > Integrate OTel C++ SDK into Impala Backend > -- > > Key: IMPALA-13235 > URL: https://issues.apache.org/jira/browse/IMPALA-13235 > Project: IMPALA > Issue Type: Improvement > Components: Backend, Toolchain >Reporter: Jason Fehr >Priority: Critical > Labels: observability > > Integrate the Otel SDK for C++ into Impala. This work involves: > 1. updating Impala to include the Otel SDK in the build process > 2. adding any necessary code to enable its use > 3. supporting authentication with OTel systems -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-11998) The iterator provided by ImpalaServer::GetQueryRecord() may become invalid
[
https://issues.apache.org/jira/browse/IMPALA-11998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Smith updated IMPALA-11998:
---
Component/s: Backend
> The iterator provided by ImpalaServer::GetQueryRecord() may become invalid
> --
>
> Key: IMPALA-11998
> URL: https://issues.apache.org/jira/browse/IMPALA-11998
> Project: IMPALA
> Issue Type: Bug
> Components: Backend, be
>Affects Versions: Impala 4.0.0, Impala 3.4.0, Impala 3.4.1, Impala 4.1.0,
> Impala 4.2.0, Impala 4.1.1, Impala 4.1.2
>Reporter: Zihao Ye
>Assignee: Zihao Ye
>Priority: Critical
> Fix For: Impala 4.3.0
>
> Attachments: hs_err_pid108619.log
>
>
> {code:c++}
> Status ImpalaServer::GetQueryRecord(const TUniqueId& query_id,
> QueryLogIndex::const_iterator* query_record) {
> lock_guard l(query_log_lock_);
> *query_record = query_log_index_.find(query_id);
> ...
> return Status::OK();
> }
> {code}
> This may cause the caller to access invalid iterators, although the function
> locks query_log_lock_ in the execution, the query_record it provides cannot
> guarantee to be valid, because it is out of the protection of query_log_lock_
> after returning, if query_log_index_ just deletes the corresponding record at
> this time, then query_record will be an invalid iterator.
> There is a very small probability that this issue may cause impalad to crash:
> {code:c++}
> Stack: [0x7f5be789f000,0x7f5be809f000], sp=0x7f5be8099a00, free
> space=8170k
> Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native
> code)
> C [impalad+0x118b08d]
> impala::ImpalaServer::GetRuntimeProfileOutput(impala::TUniqueId const&,
> std::__cxx11::basic_string, std::allocator
> > const&, impala::TRuntimeProfileFormat::type,
> impala::ImpalaServer::RuntimeProfileOutput*)+0x1dd
> C [impalad+0x1167213]
> impala::ImpalaHttpHandler::QueryProfileHelper(kudu::WebCallbackRegistry::WebRequest
> const&, rapidjson::GenericDocument,
> rapidjson::MemoryPoolAllocator,
> rapidjson::CrtAllocator>*, impala::TRuntimeProfileFormat::type)+0x53d
> C [impalad+0x116774a]
> impala::ImpalaHttpHandler::QueryProfileTextHandler(kudu::WebCallbackRegistry::WebRequest
> const&, rapidjson::GenericDocument,
> rapidjson::MemoryPoolAllocator,
> rapidjson::CrtAllocator>*)+0x16
> C [impalad+0x115bd0d] std::__cxx11::basic_string std::char_traits, std::allocator >
> apache::thrift::to_string >
> >(std::vector >,
> std::allocator > > > const&)+0x141
> C [impalad+0x14d158b]
> impala::Webserver::RenderUrlWithTemplate(sq_connection const*,
> kudu::WebCallbackRegistry::WebRequest const&, impala::Webserver::UrlHandler
> const&, std::__cxx11::basic_stringstream,
> std::allocator >*, impala::ContentType*,
> std::__cxx11::basic_string, std::allocator
> > const&)+0x177
> C [impalad+0x14d5423]
> impala::Webserver::BeginRequestCallback(sq_connection*,
> sq_request_info*)+0x1c71
> C [impalad+0x14d5d52]
> impala::Webserver::BeginRequestCallbackStatic(sq_connection*)+0x20
> C [impalad+0x14e5a87] impala::ScanRangesPB::~ScanRangesPB()+0x111
> C [impalad+0x14e7d28]
> impala::ScanRangeParamsPB::_InternalSerialize(unsigned char*,
> google::protobuf::io::EpsCopyOutputStream*) const+0x13e
> C [impalad+0x14e83ec]
> impala::PlanFragmentInstanceCtxPB::_InternalSerialize(unsigned char*,
> google::protobuf::io::EpsCopyOutputStream*) const+0x3ce
> {code}
> To trigger this issue, I looped get the oldest record on the page using a
> script:
> {code:python}
> #!/usr/bin/env python
> import requests
> from bs4 import BeautifulSoup
> import time
> root = "http://localhost:25000/;
> queries = root + "queries"
> profile = root + "query_profile_plain_text?query_id="
> while True:
> response = requests.get(queries)
> soup = BeautifulSoup(response.content, "html.parser")
> details_links = soup.find_all("a", text="Details")
> last_details_link = details_links[-1]
> details_url = last_details_link["href"]
> query_id = details_url[-33:]
> response = requests.get(profile + query_id)
> content = response.content[0:44]
> print content {code}
> At the same time, I constantly executed select 1 using another script, To
> increase the probability of triggering, I added a small delay after the
> GetQueryRecord() call. Then it was easy to trigger the crash.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12167) TestJvmMemTracker.test_jvm_mem_tracking crashing impalad during startup
[
https://issues.apache.org/jira/browse/IMPALA-12167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Smith updated IMPALA-12167:
---
Component/s: Backend
> TestJvmMemTracker.test_jvm_mem_tracking crashing impalad during startup
> ---
>
> Key: IMPALA-12167
> URL: https://issues.apache.org/jira/browse/IMPALA-12167
> Project: IMPALA
> Issue Type: Bug
> Components: Backend, be
>Affects Versions: Impala 4.3.0
>Reporter: David Rorke
>Assignee: Michael Smith
>Priority: Critical
> Fix For: Impala 4.3.0
>
> Attachments: ba01e6dd-9f36-44d6-9887dcb4-2df3f965.dmp_dumped.gz
>
>
> test_jvm_mem_tracking is failing and causing the impalad to abort/crash early
> during startup. Minidump is attached.
> The test starts impala with the following arguments:
> {noformat}
> start-impala-cluster.py
> '--state_store_args=--statestore_update_frequency_ms=50
> --statestore_priority_update_frequency_ms=50
> --statestore_heartbeat_frequency_ms=50' --cluster_size=1 --num_coordinators=1
> --log_dir=/data/jenkins/workspace/impala-asf-master-exhaustive/repos/Impala/logs/custom_cluster_tests
> --log_level=1 '--impalad_args=--mem_limit_includes_jvm=true
> --codegen_cache_capacity=0 ' '--state_store_args=None ' --jvm_args=-Xmx1g
> --impalad_args=–default_query_options={noformat}
> This test is unusual in setting --mem_limit_includes_jvm=true in the startup
> flags, so it seems likely the failure is related to use of this flag. There
> was also recent change to the Java startup options in IMPALA-11260 (to "add
> add-opens to JAVA_TOOL_OPTIONS on startup") that could be related.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12127) ImpalaD crashes during UBSAN core build
[
https://issues.apache.org/jira/browse/IMPALA-12127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Smith updated IMPALA-12127:
---
Component/s: Backend
> ImpalaD crashes during UBSAN core build
> ---
>
> Key: IMPALA-12127
> URL: https://issues.apache.org/jira/browse/IMPALA-12127
> Project: IMPALA
> Issue Type: Bug
> Components: Backend, be
>Affects Versions: Impala 4.3.0
>Reporter: Tamas Mate
>Priority: Major
> Labels: build-failure
> Attachments: 14fe9a0a-7490-4730-e77b1d9f-0627e8f7.dmp_dumped
>
>
> ImpalaD crashes during UBSAN core build.
> {code:none}
> query_test.test_parquet_stats.TestParquetStats.test_page_index[mt_dop: 2 |
> protocol: beeswax | exec_option: {'test_replan': 1, 'batch_size': 0,
> 'num_nodes': 0, 'disable_codegen_rows_threshold': 0, 'disable_codegen':
> False, 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} |
> table_format: parquet/none]
> {code}
> Related minidump part (attached as well):
> {code:none}
> 1 impalad!orc::readFooter(orc::InputStream*, orc::DataBuffer const*,
> unsigned long, orc::proto::PostScript const&, orc::MemoryPool&) [Reader.cc :
> 1317 + 0xc]
> 2 impalad!orc::DataBuffer::reserve(unsigned long) [MemoryPool.cc : 110
> + 0x2]
> 3 impalad!orc::createReader(std::unique_ptr std::default_delete >, orc::ReaderOptions const&)
> [Reader.cc : 1377 + 0x28]
> 4 impalad!std::_Head_base<0ul, orc::InputStream*,
> false>::_Head_base(orc::InputStream*&&) [tuple : 143 +
> 0x15]
> 5 impalad!std::_Tuple_impl<0ul, orc::InputStream*,
> std::default_delete >::_M_head(std::_Tuple_impl<0ul,
> orc::InputStream*, std::default_delete >&) [tuple : 211 +
> 0x15]
> 6 impalad!orc::InputStream*& std::__get_helper<0ul, orc::InputStream*,
> std::default_delete >(std::_Tuple_impl<0ul,
> orc::InputStream*, std::default_delete >&) [tuple : 1301 +
> 0x15]
> 7 impalad!std::tuple_element<0ul, std::tuple std::default_delete > >::type& std::get<0ul,
> orc::InputStream*, std::default_delete
> >(std::tuple >&)
> [tuple : 1312 + 0x15]
> 8 impalad!std::__uniq_ptr_impl std::default_delete >::_M_ptr() [unique_ptr.h : 172 + 0x15]
> 9 impalad!std::__uniq_ptr_impl std::default_delete
> >::__uniq_ptr_impl(std::__uniq_ptr_impl std::default_delete >&&) [unique_ptr.h : 163 + 0x5]
> 10 impalad!std::__uniq_ptr_data std::default_delete, true,
> true>::__uniq_ptr_data(std::__uniq_ptr_data std::default_delete, true, true>&&) [unique_ptr.h : 211 +
> 0x19]
> 11 impalad!impala::HdfsOrcScanner::ProcessFileTail() [hdfs-orc-scanner.cc :
> 510 + 0xc]
> 12
> impalad!__gnu_cxx::__aligned_membuf std::char_traits, std::allocator > const,
> impala::RuntimeProfileBase::Counter*> >::_M_ptr() [aligned_buffer.h : 73 +
> 0x9]
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12254) Inconsistent empty string handling in CSV parser
[
https://issues.apache.org/jira/browse/IMPALA-12254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Smith updated IMPALA-12254:
---
Component/s: Backend
> Inconsistent empty string handling in CSV parser
>
>
> Key: IMPALA-12254
> URL: https://issues.apache.org/jira/browse/IMPALA-12254
> Project: IMPALA
> Issue Type: Bug
> Components: Backend, be
>Reporter: Felix Bier
>Priority: Minor
>
> h3. Steps to reproduce:
> CSV table:
> {code:java}
> ,
> ,, {code}
> DDL:
>
>
> {code:java}
> CREATE EXTERNAL TABLE tab1
> (
>col_1 VARCHAR(256),
>col_2 VARCHAR(256),
>col_3 VARCHAR(256)
> )
> ROW FORMAT DELIMITED FIELDS TERMINATED BY ','
> LOCATION '/tmp/tab1'; {code}
> h3. Expected behavior:
> Querying tab1 should consistently either give all nulls or all empty strings.
> h3. Actual behavior:
> Some fields are NULL and some fields are empty string.
>
> {code:java}
> [localhost.localdomain:21050] default> select col1 is NULL, col2 is NULL,
> col3 is NULL from tab1;
> +---+---+---+
> | col_1 is null | col_2 is null | col_3 is null |
> +---+---+---+
> | false | true | false |
> | false | false | true |
> | false | false | true |
> +---+---+---+
> {code}
> h3. Analysis:
> Problem 1:
>
> DelimitedTextParser has function FillColumns for filling missing
> columns. This function creates a local variable for representing the fill
> value:
> {code:java}
> char* dummy = NULL;
> if (last_column == NULL) last_column = {code}
> The last_column pointer is passed to AddColumns, which will derefence the
> pointer and perform increment. As last_column points to dummy, the increment
> is applied to dummy, so after the call to AddColumns, dummy is NULL +1. Each
> further iteration in the loop in FillColumns will further increment dummy.
> This results in non-null pointers being set as the column value, which
> results in WriteSlot not entering the NULL case and writing an empty string
> instead.
>
> Problem 2:
> {color:#268bd2}ParseFieldLocations{color} makes calls to AddColumn with
> len=0, but non-null
> column pointers. These are interpreted as empty string instead of SQL-NULL
> in the result.
>
> h3. Suggested fix:
>
> Problem 1: The pointer increment in AddColumn could be guarded so that it is
> only performed on non-NULL pointers:
> {code:java}
> diff --git a/be/src/exec/delimited-text-parser.inline.h
> b/be/src/exec/delimited-text-parser.inline.h
> index 266fa06dd..868f4e9bc 100644
> --- a/be/src/exec/delimited-text-parser.inline.h
> +++ b/be/src/exec/delimited-text-parser.inline.h
> @@ -70,7 +70,9 @@ inline Status
> DelimitedTextParser::AddColumn(int64_t len,
> ++(*num_fields);
> }
> if (PROCESS_ESCAPES) current_column_has_escape_ = false;
> - *next_column_start += len + 1;
> + if (*next_column_start != NULL) {
> + *next_column_start += len + 1;
> + }
> ++column_idx_;
> return Status::OK();
> }
> {code}
> With this fix, querying the example tables gives:
> {code:java}
> [localhost.localdomain:21050] default> select col_1 is NULL, col_2 is NULL,
> col_3 is NULL from tab1;
> +---+---+---+
> | col_1 is null | col_2 is null | col_3 is null |
> +---+---+---+
> | false | true | true |
> | false | false | true |
> | false | false | true |
> +---+---+---+
> {code}
> Problem 2: Here it is not clear to what degree that is expected behavior. If
> this is not expected behavior, I would expect that some calls to AddColumn
> would have to be changed such that they pass a nullptr for the current column
> position if len=0.
> If the behavior is intended, the rule that is applied is not clear to me. For
> example, if the rule is that each empty token should be represented as empty
> string and each delimiter starts a new token, then the first two lines in the
> output (after applying fix for problem 1) make sense. First line in CSV file
> is empty, so it is interpreted as one empty token and then two NULL values
> are filled in. Second line contains " token>", so one NULL value gets filled in. But then the third line is not
> consistent, as it contains "
> ", so I would expect all three values to be
> non-NULL if this rule is followed.
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail:
[jira] [Updated] (IMPALA-12348) Segfault when selecting >500 columns with a filter
[
https://issues.apache.org/jira/browse/IMPALA-12348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Smith updated IMPALA-12348:
---
Component/s: Backend
> Segfault when selecting >500 columns with a filter
> --
>
> Key: IMPALA-12348
> URL: https://issues.apache.org/jira/browse/IMPALA-12348
> Project: IMPALA
> Issue Type: Bug
> Components: Backend, be
>Affects Versions: Impala 4.2.0
>Reporter: Saulius Valatka
>Priority: Major
> Attachments: 89da31ae-cc29-4394-e8c984a6-f9d9eeb3.dmp, gdb.txt,
> part-0-50adce0a-aaff-4030-9f9e-9c2093562f87.c000.snappy.parquet
>
>
> Impala daemon consistently crashes with a segfault when selecting a few
> hundred columns with a filter from a Parquet based table, i.e.: a query like:
> {{select x1, x2, ..., x500 from table where x1 = 'value' limit 100}}
> It doesn't seem to matter what columns are selected/filtered, as long as a
> certain threshold is reached. Interestingly it does not crash without the
> filter. Same queries did not crash with Impala 3.2.0.
> Attached a gdb backtrace from a debug build. Also attached a minidump, but I
> see it's only showing ?? backtraces for some reason.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-11979) Support option for consistent scan range scheduling across executor invocations with changing IP address
[ https://issues.apache.org/jira/browse/IMPALA-11979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Smith updated IMPALA-11979: --- Component/s: Backend > Support option for consistent scan range scheduling across executor > invocations with changing IP address > > > Key: IMPALA-11979 > URL: https://issues.apache.org/jira/browse/IMPALA-11979 > Project: IMPALA > Issue Type: Improvement > Components: Backend, be >Affects Versions: Impala 4.2.0 >Reporter: David Rorke >Priority: Major > > The scheduler currently assigns scan ranges to executor hosts using a hash > ring with the executor IP used as the hash key. This results in varying scan > range assignments across multiple instantiations of an executor if the > executor IP changes. This makes it more difficult to get repeatable > scheduling behavior when debugging or performance testing in an environment > where executor IPs can be assigned dynamically. > We should support a mode where the scheduler's executor hashing uses a fixed > executor identifier that we specify (unique within an executor group). This > could also be useful for potential future optimizations like cache prewarming > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12232) Verify JWT Audience and Issuer Claims
[
https://issues.apache.org/jira/browse/IMPALA-12232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Smith updated IMPALA-12232:
---
Component/s: Backend
> Verify JWT Audience and Issuer Claims
> -
>
> Key: IMPALA-12232
> URL: https://issues.apache.org/jira/browse/IMPALA-12232
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend, be, Security
>Reporter: Jason Fehr
>Assignee: Jason Fehr
>Priority: Major
> Labels: Impala, JWT, impala, jwt, security
>
> RFC 8725 contains JWT best practices that state the audience ("AUD") and
> issuer ("ISS") claims from a JWT should be validated if they are present.
> Impala currently has no mechanism to validate these claims.
> Implement [ISS claim
> validation|https://datatracker.ietf.org/doc/html/rfc8725#name-validate-issuer-and-subject]
> and [AUD claim
> validation|https://datatracker.ietf.org/doc/html/rfc8725#name-use-and-validate-audience].
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-11949) Bump AVRO version
[ https://issues.apache.org/jira/browse/IMPALA-11949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Smith updated IMPALA-11949: --- Component/s: Backend > Bump AVRO version > - > > Key: IMPALA-11949 > URL: https://issues.apache.org/jira/browse/IMPALA-11949 > Project: IMPALA > Issue Type: Sub-task > Components: Backend, be >Affects Versions: Impala 4.3.0 >Reporter: Tamas Mate >Assignee: Tamas Mate >Priority: Major > Labels: avro, impala-iceberg > > The current AVRO C version does not support logical types, while 1.11.1 C++ > one does. Additionally, the C library feels discontinued, while the C++ still > improves, we should use C++ library in the future to keep up with the new > functionalities. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12263) Update CMake Avro module with C++ lib
[ https://issues.apache.org/jira/browse/IMPALA-12263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Smith updated IMPALA-12263: --- Component/s: Backend > Update CMake Avro module with C++ lib > - > > Key: IMPALA-12263 > URL: https://issues.apache.org/jira/browse/IMPALA-12263 > Project: IMPALA > Issue Type: Sub-task > Components: Backend, be >Affects Versions: Impala 4.2.0 >Reporter: Tamas Mate >Assignee: Tamas Mate >Priority: Major > Fix For: Impala 4.3.0 > > > The C++ avro library is available in the toolchain and can be downloaded. > We should update the FindAvro CMake module to use the C++ library when > USE_AVRO_CPP is true. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12111) Speed up DATE to STRING conversion
[ https://issues.apache.org/jira/browse/IMPALA-12111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Smith updated IMPALA-12111: --- Component/s: Backend > Speed up DATE to STRING conversion > -- > > Key: IMPALA-12111 > URL: https://issues.apache.org/jira/browse/IMPALA-12111 > Project: IMPALA > Issue Type: Improvement > Components: Backend, be >Reporter: Csaba Ringhofer >Priority: Major > Fix For: Impala 4.3.0 > > > Converting DATE to STRING is slower than converting TIMESTAMP to STRING which > is weird as TIMESTAMP handling should need more work. The culprit seems to be > using stringstream in DateValue.ToString(): > https://github.com/apache/impala/blob/14698c8b99b80db7e6fd99900e32b6742bef1662/be/src/runtime/date-value.cc#L433 > As dates are returned as string to the client this also slow down the > materialization of results on the coordinator. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12191) Add hardware and OS details to runtime profile
[ https://issues.apache.org/jira/browse/IMPALA-12191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Smith updated IMPALA-12191: --- Component/s: Backend > Add hardware and OS details to runtime profile > -- > > Key: IMPALA-12191 > URL: https://issues.apache.org/jira/browse/IMPALA-12191 > Project: IMPALA > Issue Type: Improvement > Components: Backend, be >Reporter: David Rorke >Assignee: David Rorke >Priority: Major > Labels: ramp-up > > The runtime profiles are currently lacking any details about the hardware the > query ran on (CPU model and core count, cache sizes, etc) OS versions, etc > which may all be relevant when analyzing performance issues, comparing > performance metrics across different profiles, etc. > We should add relevant hardware and OS details to the profile. The > information currently displayed at the root Impalad web UI page (Hardware > Info, OS Info) would be a good starting point. > IMPALA-12118 is also relevant if we want to cover ARM processors. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-11966) Enable cache_ozone_file_handles by default
[ https://issues.apache.org/jira/browse/IMPALA-11966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Smith updated IMPALA-11966: --- Component/s: Backend > Enable cache_ozone_file_handles by default > -- > > Key: IMPALA-11966 > URL: https://issues.apache.org/jira/browse/IMPALA-11966 > Project: IMPALA > Issue Type: Improvement > Components: Backend, be >Reporter: David Rorke >Assignee: Michael Smith >Priority: Major > Labels: ozone > Fix For: Impala 4.3.0 > > > Ozone file handle caching support was added in IMPALA-10214 but disabled by > default initially. Subsequent performance testing with TPC-DS at 10TB scale > has shown that enabling Ozone file handle caching improves TPC-DS geomean > query time by 10%. The cache_ozone_file_handles parameter should be enabled > by default. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12085) The description in cluster-membership-mgr.h should be updated
[ https://issues.apache.org/jira/browse/IMPALA-12085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Smith updated IMPALA-12085: --- Component/s: Backend > The description in cluster-membership-mgr.h should be updated > - > > Key: IMPALA-12085 > URL: https://issues.apache.org/jira/browse/IMPALA-12085 > Project: IMPALA > Issue Type: Improvement > Components: Backend, be >Reporter: YifanZhang >Assignee: YifanZhang >Priority: Trivial > Fix For: Impala 4.3.0 > > > Since IMPALA-8484 is resolved, the comments about executor groups should be > removed:;[https://github.com/apache/impala/blob/2750ce59d2e0d0d52f791792d5316face3bbab08/be/src/scheduling/cluster-membership-mgr.h#L59-L61] > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12403) Kerberos authentication fails when connecting with a proxy user that passes LDAP user and group filters but does not delegate another user
[ https://issues.apache.org/jira/browse/IMPALA-12403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Smith updated IMPALA-12403: --- Component/s: Backend > Kerberos authentication fails when connecting with a proxy user that passes > LDAP user and group filters but does not delegate another user > -- > > Key: IMPALA-12403 > URL: https://issues.apache.org/jira/browse/IMPALA-12403 > Project: IMPALA > Issue Type: Bug > Components: Backend, be >Reporter: Gergely Farkas >Assignee: Gergely Farkas >Priority: Major > Fix For: Impala 4.3.0 > > > When connecting with a proxy user without _doAs_ request parameter or > _impala.doas.user_ connection config then the filters are executed with the > authenticated user itself, however, in case of Kerberos auth, the > authenticated user is a Kerberos user principal which will definitely not > pass the LDAP checks, because LDAP filters here need to be checked with a > short username (that needs to be extracted from the Kerberos user principal). > During the Kerberos authentication process, the short username is checked ( > see > [https://github.com/apache/impala/blob/master/be/src/rpc/authentication.cc#L757-L764]), > , the only point where it doesn't work like that is this: > [https://github.com/apache/impala/blob/master/be/src/service/impala-hs2-server.cc#L394-L403] > [https://github.com/apache/impala/blob/master/be/src/util/auth-util.cc#L43-L52] > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12431) Support reading compressed JSON file
[ https://issues.apache.org/jira/browse/IMPALA-12431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Smith updated IMPALA-12431: --- Component/s: Backend > Support reading compressed JSON file > > > Key: IMPALA-12431 > URL: https://issues.apache.org/jira/browse/IMPALA-12431 > Project: IMPALA > Issue Type: Sub-task > Components: Backend, be >Affects Versions: Impala 4.3.0 >Reporter: Zihao Ye >Assignee: Zihao Ye >Priority: Major > Fix For: Impala 4.4.0 > > > We have already supported reading uncompressed JSON files, but given that > JSON files have low storage efficiency and are generally stored compressed to > save space, it is important to support reading compressed JSON files as well. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12411) TSAN ThreadSanitizer: data race during expr-test teardown
[
https://issues.apache.org/jira/browse/IMPALA-12411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Smith updated IMPALA-12411:
---
Component/s: Backend
> TSAN ThreadSanitizer: data race during expr-test teardown
> -
>
> Key: IMPALA-12411
> URL: https://issues.apache.org/jira/browse/IMPALA-12411
> Project: IMPALA
> Issue Type: Task
> Components: Backend, be
>Affects Versions: Impala 4.3.0
>Reporter: Andrew Sherman
>Assignee: Michael Smith
>Priority: Major
> Fix For: Impala 4.3.0
>
> Attachments: expr-test-tsan-failure.log
>
>
> The racing threads are
> {code:java}
> 20:14:05 Read of size 8 at 0x0a8d3348 by main thread:
> 20:14:05 #0 std::vector std::allocator >::~vector()
> /data/jenkins/workspace/impala-cdw-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/gcc-10.4.0/lib/gcc/x86_64-pc-linux-gnu/10.4.0/../../../../include/c++/10.4.0/bits/stl_vector.h:680:54
> (unifiedbetests+0x3fcd9b9)
> 20:14:05 #1
> impala::TGetJvmMemoryMetricsResponse::~TGetJvmMemoryMetricsResponse()
> /data/jenkins/workspace/impala-cdw-master-core-tsan/repos/Impala/be/generated-sources/gen-cpp/Frontend_types.cpp:4158:1
> (unifiedbetests+0x3fc1397)
> 20:14:05 #2 impala::JvmMetricCache::~JvmMetricCache()
> /data/jenkins/workspace/impala-cdw-master-core-tsan/repos/Impala/be/src/util/memory-metrics.h:170:7
> (unifiedbetests+0x4b2989d)
> 20:14:05 #3 at_exit_wrapper(void*)
> /mnt/source/llvm/llvm-5.0.1.src-p7/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors.cc:361:31
> (unifiedbetests+0x21b3554)
> {code}
> and
> {code:java}
> 20:14:05 Previous write of size 8 at 0x0a8d3348 by thread T586:
> 20:14:05 #0 std::vector std::allocator
> >::_M_erase_at_end(impala::TJvmMemoryPool*)
> /data/jenkins/workspace/impala-cdw-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/gcc-10.4.0/lib/gcc/x86_64-pc-linux-gnu/10.4.0/../../../../include/c++/10.4.0/bits/stl_vector.h:1798:30
> (unifiedbetests+0x4afabcc)
> 20:14:05 #1 std::vector std::allocator >::clear()
> /data/jenkins/workspace/impala-cdw-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/gcc-10.4.0/lib/gcc/x86_64-pc-linux-gnu/10.4.0/../../../../include/c++/10.4.0/bits/stl_vector.h:1499:9
> (unifiedbetests+0x4afa4b4)
> 20:14:05 #2 unsigned int
> impala::TGetJvmMemoryMetricsResponse::read(apache::thrift::protocol::TProtocol*)
>
> /data/jenkins/workspace/impala-cdw-master-core-tsan/repos/Impala/be/generated-sources/gen-cpp/Frontend_types.tcc:5673:32
> (unifiedbetests+0x4afa21d)
> 20:14:05 #3 impala::Status
> impala::DeserializeThriftMsg(unsigned
> char const*, unsigned int*, bool, impala::TGetJvmMemoryMetricsResponse*)
> /data/jenkins/workspace/impala-cdw-master-core-tsan/repos/Impala/be/src/rpc/thrift-util.h:136:23
> (unifiedbetests+0x4af9da6)
> 20:14:05 #4 impala::Status
> impala::DeserializeThriftMsg(JNIEnv_*,
> _jbyteArray*, impala::TGetJvmMemoryMetricsResponse*)
> /data/jenkins/workspace/impala-cdw-master-core-tsan/repos/Impala/be/src/rpc/jni-thrift-util.h:61:3
> (unifiedbetests+0x4af9c62)
> 20:14:05 #5 impala::Status
> impala::JniCall::ObjectToResult(_jobject*,
> impala::TGetJvmMemoryMetricsResponse*)
> /data/jenkins/workspace/impala-cdw-master-core-tsan/repos/Impala/be/src/util/jni-util.h:493:3
> (unifiedbetests+0x4af9b24)
> 20:14:05 #6 impala::Status
> impala::JniCall::Call(impala::TGetJvmMemoryMetricsResponse*)
>
> /data/jenkins/workspace/impala-cdw-master-core-tsan/repos/Impala/be/src/util/jni-util.h:486:3
> (unifiedbetests+0x4af92a6)
> 20:14:05 #7
> impala::JniUtil::GetJvmMemoryMetrics(impala::TGetJvmMemoryMetricsResponse*)
> /data/jenkins/workspace/impala-cdw-master-core-tsan/repos/Impala/be/src/util/jni-util.cc:299:72
> (unifiedbetests+0x4af89a3)
> 20:14:05 #8 impala::JvmMetricCache::GrabMetricsIfNecessary()
> /data/jenkins/workspace/impala-cdw-master-core-tsan/repos/Impala/be/src/util/memory-metrics.cc:294:19
> (unifiedbetests+0x4b2780d)
> 20:14:05 #9 impala::JvmMetricCache::GetCounterMetric(long
> (*)(impala::TGetJvmMemoryMetricsResponse const&))
> /data/jenkins/workspace/impala-cdw-master-core-tsan/repos/Impala/be/src/util/memory-metrics.cc:305:3
> (unifiedbetests+0x4b27711)
> 20:14:05 #10 impala::JvmMemoryCounterMetric::GetValue()
> /data/jenkins/workspace/impala-cdw-master-core-tsan/repos/Impala/be/src/util/memory-metrics.cc:270:41
> (unifiedbetests+0x4b276bf)
> 20:14:05 #11
> impala::QueryState::Init(impala::ExecQueryFInstancesRequestPB const*,
> impala::TExecPlanFragmentInfo const&)::$_3::operator()() const
> /data/jenkins/workspace/impala-cdw-master-core-tsan/repos/Impala/be/src/runtime/query-state.cc:185:50
> (unifiedbetests+0x455bc15)
> 20:14:05 #12
>
[jira] [Reopened] (IMPALA-12624) Create a System Database for Impala
[ https://issues.apache.org/jira/browse/IMPALA-12624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Smith reopened IMPALA-12624: > Create a System Database for Impala > --- > > Key: IMPALA-12624 > URL: https://issues.apache.org/jira/browse/IMPALA-12624 > Project: IMPALA > Issue Type: Improvement > Components: be >Reporter: Jason Fehr >Assignee: Michael Smith >Priority: Major > Labels: workload-management > > Impala must create a database named "system" for storing Impala related > tables (such as query history). -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12581) ILIKE and IREGEXP don't work with non-const argument values
[
https://issues.apache.org/jira/browse/IMPALA-12581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Smith updated IMPALA-12581:
---
Component/s: Backend
> ILIKE and IREGEXP don't work with non-const argument values
> ---
>
> Key: IMPALA-12581
> URL: https://issues.apache.org/jira/browse/IMPALA-12581
> Project: IMPALA
> Issue Type: Bug
> Components: Backend, be
>Affects Versions: Impala 4.3.0
>Reporter: Peter Rozsa
>Assignee: Zihao Ye
>Priority: Critical
> Fix For: Impala 4.4.0
>
>
> ILIKE and IREGEXP's backend implementation does not set the case-insensitive
> flag when the argument value is non-constant.
> Example:
> {code:java}
> create table ilike_test (x string, y string);
> insert into ilike_test values ('ABC','b');
> select x ilike concat('%', y, '%') from ilike_test;{code}
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12625) Expire Old Query Log Records
[ https://issues.apache.org/jira/browse/IMPALA-12625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Smith updated IMPALA-12625: --- Component/s: Backend > Expire Old Query Log Records > > > Key: IMPALA-12625 > URL: https://issues.apache.org/jira/browse/IMPALA-12625 > Project: IMPALA > Issue Type: Improvement > Components: Backend, be >Reporter: Jason Fehr >Priority: Major > Labels: workload-management > > Add the ability for users to specify a number of days after which records > should be purged from the query log table (that stores completed queries). > Update Impala to automatically purge records older than the set threshold. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12559) Support x5c Parameter in JSON Web Keys (JWK)
[ https://issues.apache.org/jira/browse/IMPALA-12559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Smith updated IMPALA-12559: --- Component/s: Backend > Support x5c Parameter in JSON Web Keys (JWK) > > > Key: IMPALA-12559 > URL: https://issues.apache.org/jira/browse/IMPALA-12559 > Project: IMPALA > Issue Type: Bug > Components: Backend, be, Security >Reporter: Jason Fehr >Assignee: gaurav singh >Priority: Critical > Labels: JWT, jwt, security > > The ["x5u"|https://datatracker.ietf.org/doc/html/rfc7517#section-4.6], > ["x5c"|https://datatracker.ietf.org/doc/html/rfc7517#section-4.7], > ["x5t"|https://datatracker.ietf.org/doc/html/rfc7517#section-4.8], and > ["x5t#S256|https://datatracker.ietf.org/doc/html/rfc7517#section-4.9] > parameters in JWKs is not supported by Impala. Implement support for this > parameter using the available methods in the [Thalhammer/jwt-cpp > library|https://github.com/Thalhammer/jwt-cpp/blob/ce1f9df3a9f861d136d6f0c93a6f811c364d1d3d/example/jwks-verify.cpp]. > Note: If the "alg" property is specified and so is "x5u" or "x5c", then the > value of the "alg" property must match the algorithm on the certificate from > the "x5u" or "x5c" property. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12576) Query timeline not working when enable gen_experimental_profile
[ https://issues.apache.org/jira/browse/IMPALA-12576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Smith updated IMPALA-12576: --- Component/s: Backend > Query timeline not working when enable gen_experimental_profile > --- > > Key: IMPALA-12576 > URL: https://issues.apache.org/jira/browse/IMPALA-12576 > Project: IMPALA > Issue Type: Bug > Components: Backend, be >Affects Versions: Impala 4.3.0 >Reporter: Zihao Ye >Priority: Major > > When set startup flag gen_experimental_profile, the timeline page of the > query will be empty. It seems due to that the aggregated profile does not > provide event sequences. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12626) Include List of Referenced Tables/Views in Query Log Table
[ https://issues.apache.org/jira/browse/IMPALA-12626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Smith updated IMPALA-12626: --- Component/s: Backend > Include List of Referenced Tables/Views in Query Log Table > -- > > Key: IMPALA-12626 > URL: https://issues.apache.org/jira/browse/IMPALA-12626 > Project: IMPALA > Issue Type: Improvement > Components: Backend, be >Reporter: Jason Fehr >Assignee: Michael Smith >Priority: Major > Labels: workload-management > Fix For: Impala 4.4.0 > > > In the Impala query log table where completed queries are stored, add a list > of all tables and views that were referenced in the query. The purpose > behind this functionality is for end users to be able to analyze how often > each table is used in queries. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12604) Hit DCHECK in raw-value after IMPALA-12159
[
https://issues.apache.org/jira/browse/IMPALA-12604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Smith updated IMPALA-12604:
---
Component/s: Backend
> Hit DCHECK in raw-value after IMPALA-12159
> --
>
> Key: IMPALA-12604
> URL: https://issues.apache.org/jira/browse/IMPALA-12604
> Project: IMPALA
> Issue Type: Bug
> Components: Backend, be
>Affects Versions: Impala 4.4.0
>Reporter: Peter Rozsa
>Assignee: Daniel Becker
>Priority: Major
>
> After IMPALA-12159, core test suite hits a DCHECK in raw-value.cc
> Log entry:
> {code:java}
> F1207 01:12:57.681260 17760 raw-value.cc:516]
> b44210f83bc16917:55b6589e0001] Check failed: false Unknown type:
> STRUCT{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12717) Fix error message for missing table descriptor
[ https://issues.apache.org/jira/browse/IMPALA-12717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Smith updated IMPALA-12717: --- Component/s: Backend (was: be) > Fix error message for missing table descriptor > -- > > Key: IMPALA-12717 > URL: https://issues.apache.org/jira/browse/IMPALA-12717 > Project: IMPALA > Issue Type: Bug > Components: Backend >Reporter: Noemi Pap-Takacs >Assignee: Noemi Pap-Takacs >Priority: Minor > Labels: ErrorMessage, error_message_improvement > > HdfsTableSink::Prepare throws an error if the table descriptor is missing. > The error message should be something like: ‘ERROR: Failed to get table > descriptor for table id: 0’ > Instead it looks like: ‘ERROR: 0ailed to get table descriptor for table id:’ > It happens because stringstream operator << overwrites the string in the > constructor, in this case the beginning of the message with the table id. > See: [Difference between constructor and operator << > |https://stackoverflow.com/questions/49704052/c-stdstringstream-operator-vs-constructor] > It would be nice to fix these error messages in the Backend. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12627) Include Expanded SQL in Query Log Table
[ https://issues.apache.org/jira/browse/IMPALA-12627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Smith updated IMPALA-12627: --- Component/s: Backend (was: be) > Include Expanded SQL in Query Log Table > --- > > Key: IMPALA-12627 > URL: https://issues.apache.org/jira/browse/IMPALA-12627 > Project: IMPALA > Issue Type: Improvement > Components: Backend >Reporter: Jason Fehr >Priority: Major > > The current sql stored in the query log table does not have views expanded. > Include another sql statement that replaces each referenced view with that > view's sql statement. This sql statement must recursively expand the views. > If a view references another view, then expand the other view into it's sql > statement. > Also, add another list of all references tables which will contain all tables > referenced in the original query plus all tables referenced in the views. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12768) Set iceberg.engine.hive.lock-enabled To false By Default
[
https://issues.apache.org/jira/browse/IMPALA-12768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Smith updated IMPALA-12768:
---
Component/s: Catalog
(was: Backend)
> Set iceberg.engine.hive.lock-enabled To false By Default
> -
>
> Key: IMPALA-12768
> URL: https://issues.apache.org/jira/browse/IMPALA-12768
> Project: IMPALA
> Issue Type: Improvement
> Components: Catalog
>Reporter: Jason Fehr
>Priority: Major
> Labels: backend, iceberg, impala
>
> Iceberg tables are currently prone to leaving a permanent lock in HMS if
> Impala shuts down while the lock is held. This was observed intermittently
> during {{sys.impala_query_log}} testing and noted in
> https://github.com/apache/iceberg/issues/2301. It's a short lock, so it's
> fairly rare, but could happen and the only recourse is to delete rows from
> HIVE_LOCKS in HMS's database.
> https://github.com/apache/iceberg/pull/6570 introduced an alternative update
> mechanism in Iceberg 1.3 that depends on HIVE-26682. [Iceberg
> documentation|https://iceberg.apache.org/docs/latest/configuration/#hadoop-configuration]
> says to set the property "iceberg.engine.hive.lock-enabled
> {noformat}
> Warn: Setting iceberg.engine.hive.lock-enabled=false will cause HiveCatalog
> to commit to tables without using Hive locks. This should only be set to
> false if all following conditions are met:
> * HIVE-26882 is available on the Hive Metastore server
> * All other HiveCatalogs committing to tables that this HiveCatalog commits
> to are also on Iceberg 1.3 or later
> * All other HiveCatalogs committing to tables that this HiveCatalog commits
> to have also disabled Hive locks on commit.
> Failing to ensure these conditions risks corrupting the table.
> {noformat}
> Setting this property to false uses a new Hive atomic operation and avoids
> taking a lock, so it can't get stuck if Impala shuts down at the wrong time.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12768) Set iceberg.engine.hive.lock-enabled To false By Default
[
https://issues.apache.org/jira/browse/IMPALA-12768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Smith updated IMPALA-12768:
---
Component/s: Backend
(was: be)
> Set iceberg.engine.hive.lock-enabled To false By Default
> -
>
> Key: IMPALA-12768
> URL: https://issues.apache.org/jira/browse/IMPALA-12768
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
>Reporter: Jason Fehr
>Priority: Major
> Labels: backend, iceberg, impala
>
> Iceberg tables are currently prone to leaving a permanent lock in HMS if
> Impala shuts down while the lock is held. This was observed intermittently
> during {{sys.impala_query_log}} testing and noted in
> https://github.com/apache/iceberg/issues/2301. It's a short lock, so it's
> fairly rare, but could happen and the only recourse is to delete rows from
> HIVE_LOCKS in HMS's database.
> https://github.com/apache/iceberg/pull/6570 introduced an alternative update
> mechanism in Iceberg 1.3 that depends on HIVE-26682. [Iceberg
> documentation|https://iceberg.apache.org/docs/latest/configuration/#hadoop-configuration]
> says to set the property "iceberg.engine.hive.lock-enabled
> {noformat}
> Warn: Setting iceberg.engine.hive.lock-enabled=false will cause HiveCatalog
> to commit to tables without using Hive locks. This should only be set to
> false if all following conditions are met:
> * HIVE-26882 is available on the Hive Metastore server
> * All other HiveCatalogs committing to tables that this HiveCatalog commits
> to are also on Iceberg 1.3 or later
> * All other HiveCatalogs committing to tables that this HiveCatalog commits
> to have also disabled Hive locks on commit.
> Failing to ensure these conditions risks corrupting the table.
> {noformat}
> Setting this property to false uses a new Hive atomic operation and avoids
> taking a lock, so it can't get stuck if Impala shuts down at the wrong time.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12665) Adjust complete_micro_batch_ length to new scratch_batch_->capacity after ScratchTupleBatch::Reset
[
https://issues.apache.org/jira/browse/IMPALA-12665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Smith updated IMPALA-12665:
---
Component/s: Backend
(was: be)
> Adjust complete_micro_batch_ length to new scratch_batch_->capacity after
> ScratchTupleBatch::Reset
> --
>
> Key: IMPALA-12665
> URL: https://issues.apache.org/jira/browse/IMPALA-12665
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 4.3.0
>Reporter: Zinway
>Assignee: Zinway
>Priority: Major
> Fix For: Impala 4.4.0
>
>
> {panel}
> *Happened when parquet table scanning where row_size > 4096 bytes and row
> batch > 1024.*
> {panel}
> h3. Log with AddressSanitizer
>
> {code:java}
> ==557405==ERROR: AddressSanitizer: heap-buffer-overflow on address
> 0x7fa162333408 at pc 0x0413a68c bp 0x7fa162f2fc10 sp 0x7fa162f2fc08
> WRITE of size 4 at 0x7fa162333408 thread T559
> #0 0x413a68b (/usr/lib/impala/sbin/impalad+0x413a68b)# addr2line =>
> apache-impala-4.3.0/be/src/exec/parquet/parquet-common.h:570
> #1 0x419b76f (/usr/lib/impala/sbin/impalad+0x419b76f)# addr2line =>
> apache-impala-4.3.0/be/src/exec/parquet/parquet-common.h:616
> #2 0x4199769 (/usr/lib/impala/sbin/impalad+0x4199769)# addr2line =>
> apache-impala-4.3.0/be/src/exec/parquet/parquet-column-readers.cc:864
> #3 0x4195e74 (/usr/lib/impala/sbin/impalad+0x4195e74)# addr2line =>
> apache-impala-4.3.0/be/src/exec/parquet/parquet-column-readers.cc:663
> #4 0x419f719 (/usr/lib/impala/sbin/impalad+0x419f719)# addr2line =>
> apache-impala-4.3.0/be/src/exec/parquet/parquet-column-readers.cc:496
> #5 0x38876d4 (/usr/lib/impala/sbin/impalad+0x38876d4)# addr2line =>
> apache-impala-4.3.0/be/src/exec/parquet/hdfs-parquet-scanner.cc:?
> #6 0x388ef4f (/usr/lib/impala/sbin/impalad+0x388ef4f)# addr2line =>
> apache-impala-4.3.0/be/src/exec/parquet/hdfs-parquet-scanner.cc:2370
> #7 0x386db0d (/usr/lib/impala/sbin/impalad+0x386db0d)# addr2line =>
> apache-impala-4.3.0/be/src/exec/parquet/hdfs-parquet-scanner.cc:532
> #8 0x386b7d1 (/usr/lib/impala/sbin/impalad+0x386b7d1)# addr2line =>
> apache-impala-4.3.0/be/src/exec/parquet/hdfs-parquet-scanner.cc:416
> #9 0x3742adf (/usr/lib/impala/sbin/impalad+0x3742adf)# addr2line =>
> apache-impala-4.3.0/be/src/exec/hdfs-scan-node.cc:495
> #10 0x37418b8 (/usr/lib/impala/sbin/impalad+0x37418b8)# addr2line =>
> apache-impala-4.3.0/be/src/exec/hdfs-scan-node.cc:413
> #11 0x28720f6 (/usr/lib/impala/sbin/impalad+0x28720f6)
> #12 0x33db1ef (/usr/lib/impala/sbin/impalad+0x33db1ef)
> #13 0x33e74f8 (/usr/lib/impala/sbin/impalad+0x33e74f8)
> #14 0x33e734b (/usr/lib/impala/sbin/impalad+0x33e734b)
> #15 0x4b016f6 (/usr/lib/impala/sbin/impalad+0x4b016f6)
> #16 0x7fa5a4d1cdd4 (/lib64/libpthread.so.0+0x7dd4)
> #17 0x7fa5a1d0102c (/lib64/libc.so.6+0xfe02c)
> 0x7fa162333408 is located 8 bytes to the right of 4193280-byte region
> [0x7fa161f33800,0x7fa162333400)
> allocated by thread T559 here:
> #0 0x1eb956f (/usr/lib/impala/sbin/impalad+0x1eb956f)# addr2line =>
> ??:?
> #1 0x28fe1c3 (/usr/lib/impala/sbin/impalad+0x28fe1c3)# addr2line =>
> apache-impala-4.3.0/be/src/runtime/mem-pool.cc:132
> #2 0x2966b08 (/usr/lib/impala/sbin/impalad+0x2966b08)# addr2line =>
> apache-impala-4.3.0/be/src/runtime/mem-pool.h:295
> #3 0x2961bfd (/usr/lib/impala/sbin/impalad+0x2961bfd)# addr2line =>
> apache-impala-4.3.0/be/src/runtime/row-batch.cc:528
> #4 0x3818295 (/usr/lib/impala/sbin/impalad+0x3818295)# addr2line =>
> apache-impala-4.3.0/be/src/exec/scratch-tuple-batch.h:92
> #5 0x388ee46 (/usr/lib/impala/sbin/impalad+0x388ee46)# addr2line =>
> apache-impala-4.3.0/be/src/exec/parquet/hdfs-parquet-scanner.cc:2363
> #6 0x386db0d (/usr/lib/impala/sbin/impalad+0x386db0d)# addr2line =>
> apache-impala-4.3.0/be/src/exec/parquet/hdfs-parquet-scanner.cc:532
> #7 0x386b7d1 (/usr/lib/impala/sbin/impalad+0x386b7d1)# addr2line =>
> apache-impala-4.3.0/be/src/exec/parquet/hdfs-parquet-scanner.cc:416
> #8 0x3742adf (/usr/lib/impala/sbin/impalad+0x3742adf)# addr2line =>
> apache-impala-4.3.0/be/src/exec/hdfs-scan-node.cc:495
> #9 0x37418b8 (/usr/lib/impala/sbin/impalad+0x37418b8)# addr2line =>
> apache-impala-4.3.0/be/src/exec/hdfs-scan-node.cc:413
> #10 0x28720f6 (/usr/lib/impala/sbin/impalad+0x28720f6)
> #11 0x33db1ef (/usr/lib/impala/sbin/impalad+0x33db1ef)
> #12 0x33e74f8 (/usr/lib/impala/sbin/impalad+0x33e74f8)
> #13 0x33e734b (/usr/lib/impala/sbin/impalad+0x33e734b)
> #14 0x4b016f6
[jira] [Resolved] (IMPALA-12824) Add Built-in Functions to Pretty Print Duration and Bytes
[ https://issues.apache.org/jira/browse/IMPALA-12824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Smith resolved IMPALA-12824. Fix Version/s: Impala 4.4.0 Resolution: Fixed > Add Built-in Functions to Pretty Print Duration and Bytes > - > > Key: IMPALA-12824 > URL: https://issues.apache.org/jira/browse/IMPALA-12824 > Project: IMPALA > Issue Type: Improvement > Components: be >Reporter: Jason Fehr >Assignee: Jason Fehr >Priority: Major > Labels: backend, functions > Fix For: Impala 4.4.0 > > > Implement new built-in Impala string functions to pretty print time duration > from an input of nanoseconds and pretty print a memory value from an input of > bytes. > For example, pretty printing a duration of 2147483648 nanoseconds would > output "2s147ms". Pretty printing a memory value of 32768 would output > "32.00 KB". -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12786) Optimize count(*) for JSON scans
[ https://issues.apache.org/jira/browse/IMPALA-12786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Smith updated IMPALA-12786: --- Component/s: Backend (was: be) > Optimize count(*) for JSON scans > > > Key: IMPALA-12786 > URL: https://issues.apache.org/jira/browse/IMPALA-12786 > Project: IMPALA > Issue Type: Sub-task > Components: Backend >Affects Versions: Impala 4.3.0 >Reporter: Zihao Ye >Assignee: Zihao Ye >Priority: Major > > The current JSON scanner does not optimize the scanning of count(*) (zero > slots). Even if we do not need the actual data of any fields, the JSON parser > will still parse and copy strings during the scanning process. Skipping these > parsing and copying operations during the scanning of zero slots can > significantly improve performance. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12824) Add Built-in Functions to Pretty Print Duration and Bytes
[ https://issues.apache.org/jira/browse/IMPALA-12824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Smith updated IMPALA-12824: --- Component/s: Backend (was: be) > Add Built-in Functions to Pretty Print Duration and Bytes > - > > Key: IMPALA-12824 > URL: https://issues.apache.org/jira/browse/IMPALA-12824 > Project: IMPALA > Issue Type: Improvement > Components: Backend >Reporter: Jason Fehr >Assignee: Jason Fehr >Priority: Major > Labels: backend, functions > Fix For: Impala 4.4.0 > > > Implement new built-in Impala string functions to pretty print time duration > from an input of nanoseconds and pretty print a memory value from an input of > bytes. > For example, pretty printing a duration of 2147483648 nanoseconds would > output "2s147ms". Pretty printing a memory value of 32768 would output > "32.00 KB". -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12652) Limit Length of Completed Queries Insert DML
[ https://issues.apache.org/jira/browse/IMPALA-12652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Smith updated IMPALA-12652: --- Component/s: Backend (was: be) > Limit Length of Completed Queries Insert DML > > > Key: IMPALA-12652 > URL: https://issues.apache.org/jira/browse/IMPALA-12652 > Project: IMPALA > Issue Type: Improvement > Components: Backend >Reporter: Jason Fehr >Assignee: Jason Fehr >Priority: Major > Labels: backend, workload-management > > Implement a coordinator startup flag that limits the max length (number of > characters) of the insert DML statement that inserts records into the > impala_query_log table. > The purpose of this flag is to ensure that workload management does not > generate an insert DML statement that exceeds Impala's max length for a sql > statement (approximately 16 megabytes or 16 million characters). -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Reopened] (IMPALA-12824) Add Built-in Functions to Pretty Print Duration and Bytes
[ https://issues.apache.org/jira/browse/IMPALA-12824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Smith reopened IMPALA-12824: > Add Built-in Functions to Pretty Print Duration and Bytes > - > > Key: IMPALA-12824 > URL: https://issues.apache.org/jira/browse/IMPALA-12824 > Project: IMPALA > Issue Type: Improvement > Components: be >Reporter: Jason Fehr >Assignee: Jason Fehr >Priority: Major > Labels: backend, functions > > Implement new built-in Impala string functions to pretty print time duration > from an input of nanoseconds and pretty print a memory value from an input of > bytes. > For example, pretty printing a duration of 2147483648 nanoseconds would > output "2s147ms". Pretty printing a memory value of 32768 would output > "32.00 KB". -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12841) Write-Ahead Log for Completed Queries
[ https://issues.apache.org/jira/browse/IMPALA-12841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Smith updated IMPALA-12841: --- Component/s: Backend (was: be) > Write-Ahead Log for Completed Queries > - > > Key: IMPALA-12841 > URL: https://issues.apache.org/jira/browse/IMPALA-12841 > Project: IMPALA > Issue Type: Improvement > Components: Backend >Reporter: Jason Fehr >Priority: Major > Labels: workload-management > > Add the ability to set up a write-ahead log (WAL) for the queued up completed > queries that have not yet been inserted into the completed queries table. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12932) SPOOL_QUERY_OPTION Default Value Mismatch Between Code and Docs
[ https://issues.apache.org/jira/browse/IMPALA-12932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Smith updated IMPALA-12932: --- Component/s: Docs (was: Backend) > SPOOL_QUERY_OPTION Default Value Mismatch Between Code and Docs > --- > > Key: IMPALA-12932 > URL: https://issues.apache.org/jira/browse/IMPALA-12932 > Project: IMPALA > Issue Type: Bug > Components: Docs >Reporter: Jason Fehr >Assignee: Saurabh Katiyal >Priority: Major > > The Impala documentation is written to state that the SPOOL_QUERY_RESULTS > query option defaults to false. A few examples: > * > https://impala.apache.org/docs/build/html/topics/impala_spool_query_results.html > * > https://impala.apache.org/docs/build/html/topics/impala_query_results_spooling.html#data_sink > * https://impala.apache.org/docs/build/html/topics/impala_new_features.html > The code sets this query option to true by default -- > https://github.com/apache/impala/blob/1e67480ba6cd4a6f01b369eb11cc2b7330554bde/common/thrift/Query.thrift#L411 > Either the docs or the code needs to be modified to fix this mismatch. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12941) Implement the prettyprint_duration Built-in Function
[
https://issues.apache.org/jira/browse/IMPALA-12941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Smith updated IMPALA-12941:
---
Component/s: Backend
(was: be)
> Implement the prettyprint_duration Built-in Function
>
>
> Key: IMPALA-12941
> URL: https://issues.apache.org/jira/browse/IMPALA-12941
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
>Reporter: Jason Fehr
>Assignee: Jason Fehr
>Priority: Minor
> Labels: workload-management
>
> Implement a new Impala built-in function named prettyprint_duration. This
> function will take in two inputs -- a duration (all integer/floating
> point/decimal types) and a unit (nanoseconds, microseconds, milliseconds, and
> seconds) and output a prettyprinted, human readable string.
> For example:
> {noformat}
> select prettyprint_duration(2147483647, 'nanoseconds,);
> output: 2s147ms
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12897) Add Ability to set Memory Limit for Completed Queries Queue
[ https://issues.apache.org/jira/browse/IMPALA-12897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Smith updated IMPALA-12897: --- Component/s: Backend (was: be) > Add Ability to set Memory Limit for Completed Queries Queue > --- > > Key: IMPALA-12897 > URL: https://issues.apache.org/jira/browse/IMPALA-12897 > Project: IMPALA > Issue Type: Improvement > Components: Backend >Reporter: Jason Fehr >Priority: Major > Labels: impala, workload-management > > The in-memory queue that holds completed queries until they are written to > the completed queries table can grow unbounded in terms of memory usage. Add > a flag that sets a limit on the amount of memory this queue can use before it > is drained to the completed queries table. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12913) Refactor Workload Management Custom Cluster Tests
[ https://issues.apache.org/jira/browse/IMPALA-12913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Smith updated IMPALA-12913: --- Component/s: Test (was: be) > Refactor Workload Management Custom Cluster Tests > - > > Key: IMPALA-12913 > URL: https://issues.apache.org/jira/browse/IMPALA-12913 > Project: IMPALA > Issue Type: Task > Components: Test >Reporter: Jason Fehr >Assignee: Jason Fehr >Priority: Minor > Labels: test, workload-management > Fix For: Impala 4.4.0 > > > The custom cluster test_query_log.py tests can be refactored to leverage the > available tables in the "functional" database. Where custom tables > absolutely must be used, leverage the unique_database and unique_name test > fixtures. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12932) SPOOL_QUERY_OPTION Default Value Mismatch Between Code and Docs
[ https://issues.apache.org/jira/browse/IMPALA-12932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Smith updated IMPALA-12932: --- Component/s: Backend (was: be) > SPOOL_QUERY_OPTION Default Value Mismatch Between Code and Docs > --- > > Key: IMPALA-12932 > URL: https://issues.apache.org/jira/browse/IMPALA-12932 > Project: IMPALA > Issue Type: Bug > Components: Backend >Reporter: Jason Fehr >Assignee: Saurabh Katiyal >Priority: Major > > The Impala documentation is written to state that the SPOOL_QUERY_RESULTS > query option defaults to false. A few examples: > * > https://impala.apache.org/docs/build/html/topics/impala_spool_query_results.html > * > https://impala.apache.org/docs/build/html/topics/impala_query_results_spooling.html#data_sink > * https://impala.apache.org/docs/build/html/topics/impala_new_features.html > The code sets this query option to true by default -- > https://github.com/apache/impala/blob/1e67480ba6cd4a6f01b369eb11cc2b7330554bde/common/thrift/Query.thrift#L411 > Either the docs or the code needs to be modified to fix this mismatch. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-13003) Server exits early failing to create impala_query_log with AlreadyExistsException
[
https://issues.apache.org/jira/browse/IMPALA-13003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Smith updated IMPALA-13003:
---
Component/s: Backend
(was: be)
> Server exits early failing to create impala_query_log with
> AlreadyExistsException
> -
>
> Key: IMPALA-13003
> URL: https://issues.apache.org/jira/browse/IMPALA-13003
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 4.4.0
>Reporter: Andrew Sherman
>Assignee: Michael Smith
>Priority: Critical
> Labels: iceberg, workload-management
> Fix For: Impala 4.4.0
>
>
> At startup workload management tries to create the query log table here:
> {code:java}
> // The initialization code only works when run in a separate thread for
> reasons unknown.
> ABORT_IF_ERROR(SetupDbTable(internal_server_.get(), table_name));
> {code}
> This code is exiting:
> {code:java}
> I0413 23:40:05.183876 21006 client-request-state.cc:1348]
> 1d4878dbc9214c81:6dc8cc2e] ImpalaRuntimeException: Error making
> 'createTable' RPC to Hive Metastore:
> CAUSED BY: AlreadyExistsException: Table was created concurrently:
> sys.impala_query_log
> I0413 23:40:05.184055 20955 impala-server.cc:2582] Connection
> 27432606d99dcdae:218860164eb206bb from client in-memory.localhost:0 to server
> internal-server closed. The connection had 1 associated session(s).
> I0413 23:40:05.184067 20955 impala-server.cc:1780] Closing session:
> 27432606d99dcdae:218860164eb206bb
> I0413 23:40:05.184083 20955 impala-server.cc:1836] Closed session:
> 27432606d99dcdae:218860164eb206bb, client address: .
> F0413 23:40:05.184111 20955 workload-management.cc:304] query timed out
> waiting for results
> . Impalad exiting.
> I0413 23:40:05.184728 20883 impala-server.cc:1564] Query successfully
> unregistered: query_id=1d4878dbc9214c81:6dc8cc2e
> Minidump in thread [20955]completed-queries running query
> :, fragment instance
> :
> Wrote minidump to
> /data/jenkins/workspace/impala-cdw-master-core-ubsan/repos/Impala/logs/custom_cluster_tests/minidumps/impalad/402f37cc-4663-4c78-086ca295-a9e5943c.dmp
> {code}
> with stack
> {code:java}
> F0413 23:40:05.184111 20955 workload-management.cc:304] query timed out
> waiting for results
> . Impalad exiting.
> *** Check failure stack trace: ***
> @ 0x8e96a4d google::LogMessage::Fail()
> @ 0x8e98984 google::LogMessage::SendToLog()
> @ 0x8e9642c google::LogMessage::Flush()
> @ 0x8e98ea9 google::LogMessageFatal::~LogMessageFatal()
> @ 0x3da3a9a impala::ImpalaServer::CompletedQueriesThread()
> @ 0x3a8df93 boost::_mfi::mf0<>::operator()()
> @ 0x3a8de97 boost::_bi::list1<>::operator()<>()
> @ 0x3a8dd77 boost::_bi::bind_t<>::operator()()
> @ 0x3a8d672
> boost::detail::function::void_function_obj_invoker0<>::invoke()
> @ 0x301e7d0 boost::function0<>::operator()()
> @ 0x43ce415 impala::Thread::SuperviseThread()
> @ 0x43e2dc7 boost::_bi::list5<>::operator()<>()
> @ 0x43e29e7 boost::_bi::bind_t<>::operator()()
> @ 0x43e21c5 boost::detail::thread_data<>::run()
> @ 0x7984c37 thread_proxy
> @ 0x7f75b6982ea5 start_thread
> @ 0x7f75b36a7b0d __clone
> Picked up JAVA_TOOL_OPTIONS:
> -agentlib:jdwp=transport=dt_socket,address=3,server=y,suspend=n
> -Dsun.java.command=impalad
> Minidump in thread [20955]completed-queries running query
> :, fragment instance
> :
> {code}
> I think the key error is
> {code}
> CAUSED BY: AlreadyExistsException: Table was created concurrently:
> sys.impala_query_log
> {code}
> which suggests that creating the table with "if not exists" is not sufficient
> to protect against concurrent creations.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12948) Add Setting to Only Store Queries With Longer Durations
[ https://issues.apache.org/jira/browse/IMPALA-12948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Smith updated IMPALA-12948: --- Component/s: Backend (was: be) > Add Setting to Only Store Queries With Longer Durations > --- > > Key: IMPALA-12948 > URL: https://issues.apache.org/jira/browse/IMPALA-12948 > Project: IMPALA > Issue Type: Improvement > Components: Backend >Reporter: Jason Fehr >Priority: Major > Labels: workload-management > > Add an Impala coordinator flag that specifies a time duration, and if this > flag is set, do not insert queries with shorter durations than the setting > into the completed queries table. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-13004) heap-use-after-free error in ExprTest AiFunctionsTest
[
https://issues.apache.org/jira/browse/IMPALA-13004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Smith updated IMPALA-13004:
---
Component/s: Backend
(was: be)
> heap-use-after-free error in ExprTest AiFunctionsTest
> -
>
> Key: IMPALA-13004
> URL: https://issues.apache.org/jira/browse/IMPALA-13004
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 4.4.0
>Reporter: Andrew Sherman
>Assignee: Yida Wu
>Priority: Critical
> Fix For: Impala 4.4.0
>
>
> In an ASAN test, expr-test fails:
> {code}
> ==1601==ERROR: AddressSanitizer: heap-use-after-free on address
> 0x63100152c826 at pc 0x0298f841 bp 0x7ffc91fff460 sp 0x7ffc91fff458
> READ of size 2 at 0x63100152c826 thread T0
> #0 0x298f840 in rapidjson::GenericValue,
> rapidjson::MemoryPoolAllocator >::GetType() const
> /data/jenkins/workspace/impala-cdw-master-core-asan/Impala-Toolchain/toolchain-packages-gcc10.4.0/rapidjson-1.1.0/include/rapidjson/document.h:936:62
> #1 0x298d852 in bool rapidjson::GenericValue,
> rapidjson::MemoryPoolAllocator
> >::Accept,
> rapidjson::CrtAllocator>, rapidjson::UTF8, rapidjson::UTF8,
> rapidjson::CrtAllocator, 0u>
> >(rapidjson::Writer,
> rapidjson::CrtAllocator>, rapidjson::UTF8, rapidjson::UTF8,
> rapidjson::CrtAllocator, 0u>&) const
> /data/jenkins/workspace/impala-cdw-master-core-asan/Impala-Toolchain/toolchain-packages-gcc10.4.0/rapidjson-1.1.0/include/rapidjson/document.h:1769:16
> #2 0x298d8d0 in bool rapidjson::GenericValue,
> rapidjson::MemoryPoolAllocator
> >::Accept,
> rapidjson::CrtAllocator>, rapidjson::UTF8, rapidjson::UTF8,
> rapidjson::CrtAllocator, 0u>
> >(rapidjson::Writer,
> rapidjson::CrtAllocator>, rapidjson::UTF8, rapidjson::UTF8,
> rapidjson::CrtAllocator, 0u>&) const
> /data/jenkins/workspace/impala-cdw-master-core-asan/Impala-Toolchain/toolchain-packages-gcc10.4.0/rapidjson-1.1.0/include/rapidjson/document.h:1790:21
> #3 0x298d9e8 in bool rapidjson::GenericValue,
> rapidjson::MemoryPoolAllocator
> >::Accept,
> rapidjson::CrtAllocator>, rapidjson::UTF8, rapidjson::UTF8,
> rapidjson::CrtAllocator, 0u>
> >(rapidjson::Writer,
> rapidjson::CrtAllocator>, rapidjson::UTF8, rapidjson::UTF8,
> rapidjson::CrtAllocator, 0u>&) const
> /data/jenkins/workspace/impala-cdw-master-core-asan/Impala-Toolchain/toolchain-packages-gcc10.4.0/rapidjson-1.1.0/include/rapidjson/document.h:1781:21
> #4 0x28a0707 in impala_udf::StringVal
> impala::AiFunctions::AiGenerateTextInternal(impala_udf::FunctionContext*,
> impala_udf::StringVal const&, impala_udf::StringVal const&,
> impala_udf::StringVal const&, impala_udf::StringVal const&,
> impala_udf::StringVal const&, bool)
> /data/jenkins/workspace/impala-cdw-master-core-asan/repos/Impala/be/src/exprs/ai-functions.inline.h:140:11
> #5 0x286087e in impala::ExprTest_AiFunctionsTest_Test::TestBody()
> /data/jenkins/workspace/impala-cdw-master-core-asan/repos/Impala/be/src/exprs/expr-test.cc:11254:12
> #6 0x8aeaa4c in void
> testing::internal::HandleExceptionsInMethodIfSupported void>(testing::Test*, void (testing::Test::*)(), char const*)
> (/data0/jenkins/workspace/impala-cdw-master-core-asan/repos/Impala/be/build/debug/service/unifiedbetests+0x8aeaa4c)
> #7 0x8ae3ec4 in testing::Test::Run()
> (/data0/jenkins/workspace/impala-cdw-master-core-asan/repos/Impala/be/build/debug/service/unifiedbetests+0x8ae3ec4)
> #8 0x8ae4007 in testing::TestInfo::Run()
> (/data0/jenkins/workspace/impala-cdw-master-core-asan/repos/Impala/be/build/debug/service/unifiedbetests+0x8ae4007)
> #9 0x8ae40e4 in testing::TestCase::Run()
> (/data0/jenkins/workspace/impala-cdw-master-core-asan/repos/Impala/be/build/debug/service/unifiedbetests+0x8ae40e4)
> #10 0x8ae45db in testing::internal::UnitTestImpl::RunAllTests()
> (/data0/jenkins/workspace/impala-cdw-master-core-asan/repos/Impala/be/build/debug/service/unifiedbetests+0x8ae45db)
> #11 0x8ae4682 in testing::UnitTest::Run()
> (/data0/jenkins/workspace/impala-cdw-master-core-asan/repos/Impala/be/build/debug/service/unifiedbetests+0x8ae4682)
> #12 0x249ac19 in main
> /data/jenkins/workspace/impala-cdw-master-core-asan/repos/Impala/be/src/service/unified-betest-main.cc:48:10
> #13 0x7f4b0b911554 in __libc_start_main (/lib64/libc.so.6+0x22554)
> #14 0x2396af6 in _start
> (/data0/jenkins/workspace/impala-cdw-master-core-asan/repos/Impala/be/build/debug/service/unifiedbetests+0x2396af6)
> 0x63100152c826 is located 38 bytes inside of 65560-byte region
> [0x63100152c800,0x63100153c818)
> freed by thread T0 here:
> #0 0x2466ea7 in __interceptor_free
> (/data0/jenkins/workspace/impala-cdw-master-core-asan/repos/Impala/be/build/debug/service/unifiedbetests+0x2466ea7)
>
[jira] [Resolved] (IMPALA-13137) Add additional client fetch metrics columns to the queries page
[ https://issues.apache.org/jira/browse/IMPALA-13137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Smith resolved IMPALA-13137. Fix Version/s: Impala 4.5.0 Resolution: Fixed > Add additional client fetch metrics columns to the queries page > --- > > Key: IMPALA-13137 > URL: https://issues.apache.org/jira/browse/IMPALA-13137 > Project: IMPALA > Issue Type: New Feature > Components: Backend, be >Reporter: Surya Hebbar >Assignee: Surya Hebbar >Priority: Major > Fix For: Impala 4.5.0 > > Attachments: completed_queries_new_column_names.png, > completed_query.png, in_flight_queries_new_column_names.png, > in_flight_query_1.png, in_flight_query_2.png, in_flight_query_3.png, > very_short_fetch_timer.png > > > For helping users to better understand query execution times, it would be > helpful to add the following columns on the queries page. > * First row fetched time - Time taken for the client to fetch the first row > * Client fetch wait time - Time taken for the client to fetch all rows > Additional details - > https://jira.cloudera.com/browse/DWX-18295 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-13078) Unsupported column type: 8 error during query on `files` metadata table
[ https://issues.apache.org/jira/browse/IMPALA-13078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Smith updated IMPALA-13078: --- Component/s: Backend (was: be) > Unsupported column type: 8 error during query on `files` metadata table > --- > > Key: IMPALA-13078 > URL: https://issues.apache.org/jira/browse/IMPALA-13078 > Project: IMPALA > Issue Type: Bug > Components: Backend >Affects Versions: Impala 4.3.0 >Reporter: Peter Rozsa >Assignee: Daniel Becker >Priority: Minor > Labels: iceberg, impala-iceberg > > Query on metadata table `files` fails with the following error message: > F0514 13:50:15.044785 142755 iceberg-row-reader.cc:138] > 4c4ac446b97bbadc:d4bf9495] Check failed: false Unsupported column > type: 8 > Any table that contains double/float column could be used to reproduce the > issue. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-13133) Failed LOAD DATA Statements Not Cleaning Up
[ https://issues.apache.org/jira/browse/IMPALA-13133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Smith updated IMPALA-13133: --- Component/s: Backend (was: be) > Failed LOAD DATA Statements Not Cleaning Up > --- > > Key: IMPALA-13133 > URL: https://issues.apache.org/jira/browse/IMPALA-13133 > Project: IMPALA > Issue Type: Bug > Components: Backend >Reporter: Jason Fehr >Priority: Major > > When a LOAD DATA statement fails (for example, if there is a mismatch in the > number of columns between the parquet files and the destination table), > Impala is not cleaning up the table or the temporary hdfs directory. Thus, a > "show tables" command still shows the temporary table, and the warehouse hdfs > directory still contains a temporary folder. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Reopened] (IMPALA-13137) Add additional client fetch metrics columns to the queries page
[ https://issues.apache.org/jira/browse/IMPALA-13137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Smith reopened IMPALA-13137: > Add additional client fetch metrics columns to the queries page > --- > > Key: IMPALA-13137 > URL: https://issues.apache.org/jira/browse/IMPALA-13137 > Project: IMPALA > Issue Type: New Feature > Components: Backend, be >Reporter: Surya Hebbar >Assignee: Surya Hebbar >Priority: Major > Attachments: completed_queries_new_column_names.png, > completed_query.png, in_flight_queries_new_column_names.png, > in_flight_query_1.png, in_flight_query_2.png, in_flight_query_3.png, > very_short_fetch_timer.png > > > For helping users to better understand query execution times, it would be > helpful to add the following columns on the queries page. > * First row fetched time - Time taken for the client to fetch the first row > * Client fetch wait time - Time taken for the client to fetch all rows > Additional details - > https://jira.cloudera.com/browse/DWX-18295 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-13137) Add additional client fetch metrics columns to the queries page
[ https://issues.apache.org/jira/browse/IMPALA-13137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Smith updated IMPALA-13137: --- Component/s: (was: be) > Add additional client fetch metrics columns to the queries page > --- > > Key: IMPALA-13137 > URL: https://issues.apache.org/jira/browse/IMPALA-13137 > Project: IMPALA > Issue Type: New Feature > Components: Backend >Reporter: Surya Hebbar >Assignee: Surya Hebbar >Priority: Major > Fix For: Impala 4.5.0 > > Attachments: completed_queries_new_column_names.png, > completed_query.png, in_flight_queries_new_column_names.png, > in_flight_query_1.png, in_flight_query_2.png, in_flight_query_3.png, > very_short_fetch_timer.png > > > For helping users to better understand query execution times, it would be > helpful to add the following columns on the queries page. > * First row fetched time - Time taken for the client to fetch the first row > * Client fetch wait time - Time taken for the client to fetch all rows > Additional details - > https://jira.cloudera.com/browse/DWX-18295 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-11810) The mem estimate is incorrect in HdfsScanNode
[
https://issues.apache.org/jira/browse/IMPALA-11810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Smith updated IMPALA-11810:
---
Component/s: Frontend
(was: fe)
> The mem estimate is incorrect in HdfsScanNode
> --
>
> Key: IMPALA-11810
> URL: https://issues.apache.org/jira/browse/IMPALA-11810
> Project: IMPALA
> Issue Type: Improvement
> Components: Frontend
>Affects Versions: Impala 4.2.0, Impala 4.1.1
>Reporter: jhkcool
>Priority: Major
> Attachments: after_used_materialized.png, current_totalBytes.png
>
>
> About perInstanceMemEstimate calculate in the below method of the
> HdfsScanNode class:
> {code:java}
> @Override
> public void computeNodeResourceProfile(TQueryOptions queryOptions) {
> ...
> long avgScanRangeBytes = (long) Math.ceil(sumValues(totalBytesPerFs_) /
> (double) scanRangeSize);
> ...
> }{code}
> All table data file sizes are used in the calculation of scan hdfs memory
> consumption, it is unreasonable. Because not all data in the table needs to
> be read into the memory, but only the materialized fields are read, so only
> the data file size occupied by the materialized field needs to be calculated.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-11750) Iceberg LOAD DATA INPATH partition clause support
[
https://issues.apache.org/jira/browse/IMPALA-11750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Smith updated IMPALA-11750:
---
Component/s: Frontend
(was: fe)
> Iceberg LOAD DATA INPATH partition clause support
> -
>
> Key: IMPALA-11750
> URL: https://issues.apache.org/jira/browse/IMPALA-11750
> Project: IMPALA
> Issue Type: Improvement
> Components: Frontend
>Affects Versions: Impala 4.2.0
>Reporter: Tamas Mate
>Priority: Major
>
> To support the PARTITION clause of LOAD DATA INPATH the provided
> PartitionSpec should be analysed or worked around. Once that is done, a
> possible solution could be to modify the INSERT INTO SELECT statement to
> ingest the specified partition value, ie.:
> {code:none}
> INSERT INTO ... SELECT part_1, * FROM tmp_table;
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-13173) Redundant Catalog Update Check in Coordinator
[ https://issues.apache.org/jira/browse/IMPALA-13173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Smith updated IMPALA-13173: --- Component/s: (was: be) > Redundant Catalog Update Check in Coordinator > - > > Key: IMPALA-13173 > URL: https://issues.apache.org/jira/browse/IMPALA-13173 > Project: IMPALA > Issue Type: Bug > Components: Backend >Reporter: Noemi Pap-Takacs >Assignee: Noemi Pap-Takacs >Priority: Major > Labels: impala-iceberg > > In case of DML operations, the Coordinator sends an update to the Catalog > about the files changed in the table. Before sending the update, we check if > any file was created. If no files were added or deleted, we skip the catalog > update. See the logic in _'DmlExecState::PrepareCatalogUpdate'._ > However, in case of unpartitioned Iceberg tables, the check in > _'DmlExecState::PrepareCatalogUpdate'_ always returns true, and updates the > Catalog even if no files were added. Currently, this does not cause incorrect > behavior because the presence of created files is double-checked later in > client-request-state.cc. > On the other hand, there are cases, when not writing any files does not equal > a NO-OP. For example overwriting a table with empty content or an OPTIMIZE > TABLE that merges delete files. The Catalog needs to be informed about the > changes in such cases. > We should filter NO-OP DMLs correctly in the Coordinator, eliminating false > positive and false negative updates as well. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12186) Iceberg LOAD DATA INPATH should set TEMPORARY table property
[
https://issues.apache.org/jira/browse/IMPALA-12186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Smith updated IMPALA-12186:
---
Component/s: (was: fe)
Frontend
> Iceberg LOAD DATA INPATH should set TEMPORARY table property
>
>
> Key: IMPALA-12186
> URL: https://issues.apache.org/jira/browse/IMPALA-12186
> Project: IMPALA
> Issue Type: Improvement
> Components: Frontend
>Affects Versions: Impala 4.2.0
>Reporter: Tamas Mate
>Assignee: Tamas Mate
>Priority: Major
> Labels: impala-iceberg
>
> Apache Atlas looks for {{TEMPORARY}} table property to display items. The
> LOAD DATA operation creates a temporary table and for nicer lineage display
> this table should have {{TEMPORARY=true}} property set.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-11918) Fix test_java_udfs_from_impala after IMPALA-11745
[ https://issues.apache.org/jira/browse/IMPALA-11918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Smith updated IMPALA-11918: --- Component/s: Frontend Test (was: fe) > Fix test_java_udfs_from_impala after IMPALA-11745 > - > > Key: IMPALA-11918 > URL: https://issues.apache.org/jira/browse/IMPALA-11918 > Project: IMPALA > Issue Type: Bug > Components: Frontend, Test >Affects Versions: Impala 4.3.0 >Reporter: Peter Rozsa >Assignee: Peter Rozsa >Priority: Major > Labels: broken-build > > IMPALA-11745 changed the error message regarding failed method extraction for > Hive UDFs, and an exhaustive test case remained unchanged, causing failure in > exhaustive builds. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12121) Add non empty check for UDF location in getLastModifiedTime method
[
https://issues.apache.org/jira/browse/IMPALA-12121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Smith updated IMPALA-12121:
---
Component/s: Frontend
(was: fe)
> Add non empty check for UDF location in getLastModifiedTime method
> --
>
> Key: IMPALA-12121
> URL: https://issues.apache.org/jira/browse/IMPALA-12121
> Project: IMPALA
> Issue Type: Improvement
> Components: Frontend
>Reporter: Soumyakanti Das
>Assignee: Soumyakanti Das
>Priority: Major
> Fix For: Impala 4.3.0
>
>
> {{getLastModifiedTime()}} of {{Function.java}} assumes that if the UDF is not
> a BUILTIN and if the location is not null, it is a non-empty string. However,
> we may have UDFs with an empty location if they are in the install location
> and loaded at init time.
> We should also add a non empty check for the location in the if condition.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-11811) Avoid storing unregistered predicate objects in a Map
[ https://issues.apache.org/jira/browse/IMPALA-11811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Smith updated IMPALA-11811: --- Component/s: Frontend (was: fe) > Avoid storing unregistered predicate objects in a Map > - > > Key: IMPALA-11811 > URL: https://issues.apache.org/jira/browse/IMPALA-11811 > Project: IMPALA > Issue Type: Bug > Components: Frontend >Affects Versions: Impala 4.2.0 >Reporter: Andrew Sherman >Assignee: Andrew Sherman >Priority: Major > Fix For: Impala 4.3.0 > > > Within the extractIcebergConjuncts() method we are tracking conjuncts which > are identity conjuncts by storing them in a temporary Map. > The conjuncts are Expr objects which have a hashCode() method based on their > id_ field, which is only present when they are registered. > If the id_ field is null, then the hashCode() will throw, and hence > unregistered predicates cannot be stored in a Map. > This can happen if the conjuncts are bound predicates (produced by > getBoundPredicates()) which are explicitly not registered. > Change extractIcebergConjuncts() to track the identity conjuncts in another > way, perhaps using a boolean array which tracks the index of the identity > conjuncts in conjuncts_ List. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-11918) Fix test_java_udfs_from_impala after IMPALA-11745
[ https://issues.apache.org/jira/browse/IMPALA-11918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Smith updated IMPALA-11918: --- Component/s: (was: Frontend) > Fix test_java_udfs_from_impala after IMPALA-11745 > - > > Key: IMPALA-11918 > URL: https://issues.apache.org/jira/browse/IMPALA-11918 > Project: IMPALA > Issue Type: Bug > Components: Test >Affects Versions: Impala 4.3.0 >Reporter: Peter Rozsa >Assignee: Peter Rozsa >Priority: Major > Labels: broken-build > > IMPALA-11745 changed the error message regarding failed method extraction for > Hive UDFs, and an exhaustive test case remained unchanged, causing failure in > exhaustive builds. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12062) TestUdfExecution.test_java_udfs fails intermittently
[
https://issues.apache.org/jira/browse/IMPALA-12062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Smith updated IMPALA-12062:
---
Component/s: Frontend
Test
(was: fe)
> TestUdfExecution.test_java_udfs fails intermittently
>
>
> Key: IMPALA-12062
> URL: https://issues.apache.org/jira/browse/IMPALA-12062
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend, Test
>Affects Versions: Impala 4.3.0
>Reporter: Peter Rozsa
>Assignee: Peter Rozsa
>Priority: Major
> Labels: flaky, test-fail
>
> TestUdfExecution.test_java_udfs fails intermittently in various cases, where
> the expected value is not NULL, but the execution returns NULL.
> NULL handling changed in IMPALA-11911 and String/Binary handling changed in
> IMPALA-11854, these changes may be related.
> Example error message:
> {code:java}
> query_test/test_udfs.py:344: in test_java_udfs
> self.run_test_case('QueryTest/java-udf', vector, use_db=unique_database)
> common/impala_test_suite.py:749: in run_test_case
> self.__verify_results_and_errors(vector, test_section, result, use_db)
> common/impala_test_suite.py:585: in __verify_results_and_errors
> replace_filenames_with_placeholder) common/test_result_verifier.py:486: in
> verify_raw_results VERIFIER_MAP[verifier](expected, actual)
> common/test_result_verifier.py:295: in verify_query_result_is_equal
> assert expected_results == actual_results E assert Comparing
> QueryTestResults (expected vs actual): E '' != 'NULL' {code}
> Statement:
> {code:java}
> select increment(""); {code}
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12042) Invalid casts in set operations calculation
[
https://issues.apache.org/jira/browse/IMPALA-12042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Smith updated IMPALA-12042:
---
Component/s: Frontend
(was: fe)
> Invalid casts in set operations calculation
> ---
>
> Key: IMPALA-12042
> URL: https://issues.apache.org/jira/browse/IMPALA-12042
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
>Affects Versions: Impala 4.3.0
>Reporter: Peter Rozsa
>Assignee: Peter Rozsa
>Priority: Major
> Fix For: Impala 4.3.0
>
>
> Invalid implicit casts are produced when a set operation contains transitive
> compatibility, for example:
> {code:java}
> values (true), (123), (111.0){code}
> where boolean(true) is compatible with tinyint(123) and tinyint is
> compatible with decimal(111.0) but boolean is not compatible with decimal
> nonetheless 'castToSetOpCompatibleTypes' produces decimal as a compatible
> type.
>
> If the condition described above exists, FE fails with
> {code:java}
> IllegalStateException: cast BOOLEAN to DECIMAL(4,1){code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12177) The exception in CatalogOpExecutor.unsetPartitionsColStats() was not thrown
[ https://issues.apache.org/jira/browse/IMPALA-12177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Smith updated IMPALA-12177: --- Component/s: Frontend (was: fe) > The exception in CatalogOpExecutor.unsetPartitionsColStats() was not thrown > --- > > Key: IMPALA-12177 > URL: https://issues.apache.org/jira/browse/IMPALA-12177 > Project: IMPALA > Issue Type: Bug > Components: Frontend >Reporter: Zihao Ye >Assignee: Zihao Ye >Priority: Major > Fix For: Impala 4.3.0 > > > The throw keyword was omitted when capturing TException and creating > ImpalaRuntimeException in CatalogOpExecutor.unsetPartitionsColStats(), > causing the newly created exception to not be thrown. This may result in the > method returning normally in exceptional cases, which could mask errors. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12161) Add query hints for memory limits
[
https://issues.apache.org/jira/browse/IMPALA-12161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Smith updated IMPALA-12161:
---
Component/s: Frontend
(was: fe)
> Add query hints for memory limits
> -
>
> Key: IMPALA-12161
> URL: https://issues.apache.org/jira/browse/IMPALA-12161
> Project: IMPALA
> Issue Type: New Feature
> Components: Frontend
>Affects Versions: Impala 4.1.2
>Reporter: Peter Ebert
>Priority: Major
>
> Support for a query hint to specify memory limits would make implementing
> memory limits in applications easier where multiple queries need different
> memory limits but are run through the same connection, e.g.: {{/*
> +mem_limit=4g */ select * from table}}
> ---
> Session settings for mem_limit are critical for getting large queries to
> avoid OOM when using admission control pools (for multi tenancy) and the
> memory estimation happens to severely under-estimate (raising the minimum
> memory for the pool would also reduce concurrency). However, for
> applications that run multiple queries through a single connection (e.g. data
> visualization tools) this pattern is cumbersome:
> # Set the memory limit
> # Run the large query
> # Reset the memory limit
> # Resume other queries with good estimates
> Due to this, some data visualization tools only support session settings at
> the connection level. If you need GBs for your worst query and MBs for your
> average query, this greatly limits the concurrency in that pool using a 'one
> size fits all' memory limit or it requires maintaining many connections of
> varying size memory limits to attempt to use memory efficiently.
> In one production use case this would limit concurrency by 10x or more.
>
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12279) Bump CDP_BUILD_NUMBER for Iceberg 1.3
[ https://issues.apache.org/jira/browse/IMPALA-12279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Smith updated IMPALA-12279: --- Component/s: Frontend (was: fe) > Bump CDP_BUILD_NUMBER for Iceberg 1.3 > - > > Key: IMPALA-12279 > URL: https://issues.apache.org/jira/browse/IMPALA-12279 > Project: IMPALA > Issue Type: Task > Components: Frontend >Reporter: Tamas Mate >Assignee: Tamas Mate >Priority: Major > Fix For: Impala 4.4.0 > > > Iceberg 1.3 is available in the CPD build, we should bump the CDP build > version. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-12285) Type mismatch in set operations with ALLOW_UNSAFE_CASTS enabled
[
https://issues.apache.org/jira/browse/IMPALA-12285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Smith resolved IMPALA-12285.
Fix Version/s: Impala 4.4.0
Resolution: Fixed
> Type mismatch in set operations with ALLOW_UNSAFE_CASTS enabled
> ---
>
> Key: IMPALA-12285
> URL: https://issues.apache.org/jira/browse/IMPALA-12285
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
>Reporter: Peter Rozsa
>Assignee: Peter Rozsa
>Priority: Major
> Fix For: Impala 4.4.0
>
>
> {code:java}
> INSERT INTO iobt(c0) VALUES ("0"), (15629);{code}
> hit DCHECK in tuple.cc:233; the common type calculation resolves to smallint
> due to 15629, but literal casting does not respect the target type, and
> transforms the string literal to tinyint type.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Reopened] (IMPALA-12285) Type mismatch in set operations with ALLOW_UNSAFE_CASTS enabled
[
https://issues.apache.org/jira/browse/IMPALA-12285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Smith reopened IMPALA-12285:
> Type mismatch in set operations with ALLOW_UNSAFE_CASTS enabled
> ---
>
> Key: IMPALA-12285
> URL: https://issues.apache.org/jira/browse/IMPALA-12285
> Project: IMPALA
> Issue Type: Bug
> Components: fe
>Reporter: Peter Rozsa
>Assignee: Peter Rozsa
>Priority: Major
>
> {code:java}
> INSERT INTO iobt(c0) VALUES ("0"), (15629);{code}
> hit DCHECK in tuple.cc:233; the common type calculation resolves to smallint
> due to 15629, but literal casting does not respect the target type, and
> transforms the string literal to tinyint type.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12266) Sporadic failure after migrating a table to Iceberg
[
https://issues.apache.org/jira/browse/IMPALA-12266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Smith updated IMPALA-12266:
---
Component/s: Test
> Sporadic failure after migrating a table to Iceberg
> ---
>
> Key: IMPALA-12266
> URL: https://issues.apache.org/jira/browse/IMPALA-12266
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend, Test
>Affects Versions: Impala 4.2.0
>Reporter: Tamas Mate
>Assignee: Gabor Kaszab
>Priority: Critical
> Labels: impala-iceberg
> Attachments:
> catalogd.bd40020df22b.invalid-user.log.INFO.20230704-181939.1,
> impalad.6c0f48d9ce66.invalid-user.log.INFO.20230704-181940.1
>
>
> TestIcebergTable.test_convert_table test failed in a recent verify job's
> dockerised tests:
> https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/7629
> {code:none}
> E ImpalaBeeswaxException: ImpalaBeeswaxException:
> EINNER EXCEPTION:
> EMESSAGE: AnalysisException: Failed to load metadata for table:
> 'parquet_nopartitioned'
> E CAUSED BY: TableLoadingException: Could not load table
> test_convert_table_cdba7383.parquet_nopartitioned from catalog
> E CAUSED BY: TException:
> TGetPartialCatalogObjectResponse(status:TStatus(status_code:GENERAL,
> error_msgs:[NullPointerException: null]), lookup_status:OK)
> {code}
> {code:none}
> E0704 19:09:22.980131 833 JniUtil.java:183]
> 7145c21173f2c47b:2579db55] Error in Getting partial catalog object of
> TABLE:test_convert_table_cdba7383.parquet_nopartitioned. Time spent: 49ms
> I0704 19:09:22.980309 833 jni-util.cc:288]
> 7145c21173f2c47b:2579db55] java.lang.NullPointerException
> at
> org.apache.impala.catalog.CatalogServiceCatalog.replaceTableIfUnchanged(CatalogServiceCatalog.java:2357)
> at
> org.apache.impala.catalog.CatalogServiceCatalog.getOrLoadTable(CatalogServiceCatalog.java:2300)
> at
> org.apache.impala.catalog.CatalogServiceCatalog.doGetPartialCatalogObject(CatalogServiceCatalog.java:3587)
> at
> org.apache.impala.catalog.CatalogServiceCatalog.getPartialCatalogObject(CatalogServiceCatalog.java:3513)
> at
> org.apache.impala.catalog.CatalogServiceCatalog.getPartialCatalogObject(CatalogServiceCatalog.java:3480)
> at
> org.apache.impala.service.JniCatalog.lambda$getPartialCatalogObject$11(JniCatalog.java:397)
> at
> org.apache.impala.service.JniCatalogOp.lambda$execAndSerialize$1(JniCatalogOp.java:90)
> at org.apache.impala.service.JniCatalogOp.execOp(JniCatalogOp.java:58)
> at
> org.apache.impala.service.JniCatalogOp.execAndSerialize(JniCatalogOp.java:89)
> at
> org.apache.impala.service.JniCatalogOp.execAndSerializeSilentStartAndFinish(JniCatalogOp.java:109)
> at
> org.apache.impala.service.JniCatalog.execAndSerializeSilentStartAndFinish(JniCatalog.java:238)
> at
> org.apache.impala.service.JniCatalog.getPartialCatalogObject(JniCatalog.java:396)
> I0704 19:09:22.980324 833 status.cc:129] 7145c21173f2c47b:2579db55]
> NullPointerException: null
> @ 0x1012f9f impala::Status::Status()
> @ 0x187f964 impala::JniUtil::GetJniExceptionMsg()
> @ 0xfee920 impala::JniCall::Call<>()
> @ 0xfccd0f impala::Catalog::GetPartialCatalogObject()
> @ 0xfb55a5
> impala::CatalogServiceThriftIf::GetPartialCatalogObject()
> @ 0xf7a691
> impala::CatalogServiceProcessorT<>::process_GetPartialCatalogObject()
> @ 0xf82151 impala::CatalogServiceProcessorT<>::dispatchCall()
> @ 0xee330f apache::thrift::TDispatchProcessor::process()
> @ 0x1329246
> apache::thrift::server::TAcceptQueueServer::Task::run()
> @ 0x1315a89 impala::ThriftThread::RunRunnable()
> @ 0x131773d
> boost::detail::function::void_function_obj_invoker0<>::invoke()
> @ 0x195ba8c impala::Thread::SuperviseThread()
> @ 0x195c895 boost::detail::thread_data<>::run()
> @ 0x23a03a7 thread_proxy
> @ 0x7faaad2a66ba start_thread
> @ 0x7f2c151d clone
> E0704 19:09:23.006968 833 catalog-server.cc:278]
> 7145c21173f2c47b:2579db55] NullPointerException: null
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Closed] (IMPALA-12280) Add storate_handler to Atlas lineage log
[
https://issues.apache.org/jira/browse/IMPALA-12280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Smith closed IMPALA-12280.
--
Resolution: Incomplete
It's not clear from the ticket why this is necessary, and has had no updates in
a year.
> Add storate_handler to Atlas lineage log
>
>
> Key: IMPALA-12280
> URL: https://issues.apache.org/jira/browse/IMPALA-12280
> Project: IMPALA
> Issue Type: New Feature
> Components: Frontend
>Affects Versions: Impala 4.2.0
>Reporter: Tamas Mate
>Priority: Major
>
> Atlas lineage report should have a {{storage_handler}} property as well.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12266) Sporadic failure after migrating a table to Iceberg
[
https://issues.apache.org/jira/browse/IMPALA-12266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Smith updated IMPALA-12266:
---
Component/s: Frontend
(was: fe)
> Sporadic failure after migrating a table to Iceberg
> ---
>
> Key: IMPALA-12266
> URL: https://issues.apache.org/jira/browse/IMPALA-12266
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
>Affects Versions: Impala 4.2.0
>Reporter: Tamas Mate
>Assignee: Gabor Kaszab
>Priority: Critical
> Labels: impala-iceberg
> Attachments:
> catalogd.bd40020df22b.invalid-user.log.INFO.20230704-181939.1,
> impalad.6c0f48d9ce66.invalid-user.log.INFO.20230704-181940.1
>
>
> TestIcebergTable.test_convert_table test failed in a recent verify job's
> dockerised tests:
> https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/7629
> {code:none}
> E ImpalaBeeswaxException: ImpalaBeeswaxException:
> EINNER EXCEPTION:
> EMESSAGE: AnalysisException: Failed to load metadata for table:
> 'parquet_nopartitioned'
> E CAUSED BY: TableLoadingException: Could not load table
> test_convert_table_cdba7383.parquet_nopartitioned from catalog
> E CAUSED BY: TException:
> TGetPartialCatalogObjectResponse(status:TStatus(status_code:GENERAL,
> error_msgs:[NullPointerException: null]), lookup_status:OK)
> {code}
> {code:none}
> E0704 19:09:22.980131 833 JniUtil.java:183]
> 7145c21173f2c47b:2579db55] Error in Getting partial catalog object of
> TABLE:test_convert_table_cdba7383.parquet_nopartitioned. Time spent: 49ms
> I0704 19:09:22.980309 833 jni-util.cc:288]
> 7145c21173f2c47b:2579db55] java.lang.NullPointerException
> at
> org.apache.impala.catalog.CatalogServiceCatalog.replaceTableIfUnchanged(CatalogServiceCatalog.java:2357)
> at
> org.apache.impala.catalog.CatalogServiceCatalog.getOrLoadTable(CatalogServiceCatalog.java:2300)
> at
> org.apache.impala.catalog.CatalogServiceCatalog.doGetPartialCatalogObject(CatalogServiceCatalog.java:3587)
> at
> org.apache.impala.catalog.CatalogServiceCatalog.getPartialCatalogObject(CatalogServiceCatalog.java:3513)
> at
> org.apache.impala.catalog.CatalogServiceCatalog.getPartialCatalogObject(CatalogServiceCatalog.java:3480)
> at
> org.apache.impala.service.JniCatalog.lambda$getPartialCatalogObject$11(JniCatalog.java:397)
> at
> org.apache.impala.service.JniCatalogOp.lambda$execAndSerialize$1(JniCatalogOp.java:90)
> at org.apache.impala.service.JniCatalogOp.execOp(JniCatalogOp.java:58)
> at
> org.apache.impala.service.JniCatalogOp.execAndSerialize(JniCatalogOp.java:89)
> at
> org.apache.impala.service.JniCatalogOp.execAndSerializeSilentStartAndFinish(JniCatalogOp.java:109)
> at
> org.apache.impala.service.JniCatalog.execAndSerializeSilentStartAndFinish(JniCatalog.java:238)
> at
> org.apache.impala.service.JniCatalog.getPartialCatalogObject(JniCatalog.java:396)
> I0704 19:09:22.980324 833 status.cc:129] 7145c21173f2c47b:2579db55]
> NullPointerException: null
> @ 0x1012f9f impala::Status::Status()
> @ 0x187f964 impala::JniUtil::GetJniExceptionMsg()
> @ 0xfee920 impala::JniCall::Call<>()
> @ 0xfccd0f impala::Catalog::GetPartialCatalogObject()
> @ 0xfb55a5
> impala::CatalogServiceThriftIf::GetPartialCatalogObject()
> @ 0xf7a691
> impala::CatalogServiceProcessorT<>::process_GetPartialCatalogObject()
> @ 0xf82151 impala::CatalogServiceProcessorT<>::dispatchCall()
> @ 0xee330f apache::thrift::TDispatchProcessor::process()
> @ 0x1329246
> apache::thrift::server::TAcceptQueueServer::Task::run()
> @ 0x1315a89 impala::ThriftThread::RunRunnable()
> @ 0x131773d
> boost::detail::function::void_function_obj_invoker0<>::invoke()
> @ 0x195ba8c impala::Thread::SuperviseThread()
> @ 0x195c895 boost::detail::thread_data<>::run()
> @ 0x23a03a7 thread_proxy
> @ 0x7faaad2a66ba start_thread
> @ 0x7f2c151d clone
> E0704 19:09:23.006968 833 catalog-server.cc:278]
> 7145c21173f2c47b:2579db55] NullPointerException: null
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12285) Type mismatch in set operations with ALLOW_UNSAFE_CASTS enabled
[
https://issues.apache.org/jira/browse/IMPALA-12285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Smith updated IMPALA-12285:
---
Component/s: Frontend
(was: fe)
> Type mismatch in set operations with ALLOW_UNSAFE_CASTS enabled
> ---
>
> Key: IMPALA-12285
> URL: https://issues.apache.org/jira/browse/IMPALA-12285
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
>Reporter: Peter Rozsa
>Assignee: Peter Rozsa
>Priority: Major
>
> {code:java}
> INSERT INTO iobt(c0) VALUES ("0"), (15629);{code}
> hit DCHECK in tuple.cc:233; the common type calculation resolves to smallint
> due to 15629, but literal casting does not respect the target type, and
> transforms the string literal to tinyint type.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12291) Insert statement fails even if hdfs ranger policy allows it
[
https://issues.apache.org/jira/browse/IMPALA-12291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Smith updated IMPALA-12291:
---
Component/s: Frontend
(was: fe)
> Insert statement fails even if hdfs ranger policy allows it
> ---
>
> Key: IMPALA-12291
> URL: https://issues.apache.org/jira/browse/IMPALA-12291
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend, Security
> Environment: - Impala Version (4.1.0)
> - Ranger admin version (2.0)
> - Hive version (3.1.2)
>Reporter: halim kim
>Assignee: halim kim
>Priority: Major
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Apache Ranger is framework for providing security and authorization in hadoop
> platform.
> Impala can also utilize apache ranger via ranger hive policy.
> The thing is that insert or some other query is not executed even If you
> enable ranger hdfs plugin and set proper allow condition for impala query
> excuting.
> you can see error log like below.
> {code:java}
> AnalysisException: Unable to INSERT into target table (testdb.testtable)
> because Impala does not have WRITE access to HDFS location:
> hdfs://testcluster/warehouse/testdb.db/testtable
> {code}
> This happens when ranger hdfs plugin is enabled but impala doesn't have
> permission for hdfs POSIX permission.
> For example, In the case that DB file owner, group and permission is set as
> hdfs:hdfs r-xr-xr-- and ranger plugin policy(hdfs, hive and impala) allows
> impala to execute query, Insert Query will be fail.
> In my opinion, The main cause is impala fe component doesn't check ranger
> policy but hdfs POSIX model permissions.
> Similar issue : https://issues.apache.org/jira/browse/IMPALA-10272
> I'm working on resolving this issue by adding hdfs ranger policy checking
> code.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12395) Planner overestimates scan cardinality for queries using count star optimization
[
https://issues.apache.org/jira/browse/IMPALA-12395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Smith updated IMPALA-12395:
---
Component/s: Frontend
(was: fe)
> Planner overestimates scan cardinality for queries using count star
> optimization
>
>
> Key: IMPALA-12395
> URL: https://issues.apache.org/jira/browse/IMPALA-12395
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
>Reporter: David Rorke
>Assignee: Riza Suminto
>Priority: Critical
> Fix For: Impala 4.3.0
>
>
> The scan cardinality estimate for count(*) queries doesn't account for the
> fact that the count(*) optimization only scans metadata and not the actual
> columns.
> Scan for a count(*) query on Parquet store_sales:
>
> {noformat}
> Operator #Hosts #Inst Avg Time Max Time #Rows Est. #Rows Peak Mem Est. Peak
> Mem Detail
> -
> 00:SCAN S3 6 72 8s131ms 8s496ms 2.71K 8.64B 128.00 KB 88.00 MB
> tpcds_3000_string_parquet_managed.store_sales
> {noformat}
>
> This is a problem with all file/table formats that implement count(*)
> optimizations (Parquet and also probably ORC and Iceberg).
> This problem is more serious than it was in the past because with
> IMPALA-12091 we now rely on scan cardinality estimates for executor group
> assignments so count(*) queries are likely to get assigned to a larger
> executor group than needed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12384) Restore NullLiteral's uncheckedCastTo function signature
[ https://issues.apache.org/jira/browse/IMPALA-12384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Smith updated IMPALA-12384: --- Component/s: Frontend (was: fe) > Restore NullLiteral's uncheckedCastTo function signature > > > Key: IMPALA-12384 > URL: https://issues.apache.org/jira/browse/IMPALA-12384 > Project: IMPALA > Issue Type: Bug > Components: Frontend >Affects Versions: Impala 4.3.0 >Reporter: Peter Rozsa >Assignee: Peter Rozsa >Priority: Minor > Fix For: Impala 4.3.0 > > > NullLiteral's uncheckedCastTo function should preserve its signature as it > was before IMPALA-10173 to maintain its external compatibility. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12386) NullExpr substitution failure with unsafe casts enabled
[
https://issues.apache.org/jira/browse/IMPALA-12386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Smith updated IMPALA-12386:
---
Component/s: Frontend
(was: fe)
> NullExpr substitution failure with unsafe casts enabled
> ---
>
> Key: IMPALA-12386
> URL: https://issues.apache.org/jira/browse/IMPALA-12386
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
>Affects Versions: Impala 4.3.0
>Reporter: Peter Rozsa
>Assignee: Peter Rozsa
>Priority: Major
> Fix For: Impala 4.3.0
>
>
> insert into t01(a, b) values(null, "23"), ("21", null) query fails with the
> following error:
> ERROR: IllegalStateException: Failed analysis after expr substitution.
> CAUSED BY: IllegalStateException: cast STRING to INT
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12402) Make CatalogdMetaProvider's cache concurrency level configurable
[ https://issues.apache.org/jira/browse/IMPALA-12402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Smith updated IMPALA-12402: --- Component/s: Frontend (was: fe) > Make CatalogdMetaProvider's cache concurrency level configurable > > > Key: IMPALA-12402 > URL: https://issues.apache.org/jira/browse/IMPALA-12402 > Project: IMPALA > Issue Type: Improvement > Components: Frontend >Reporter: Maxwell Guo >Assignee: Maxwell Guo >Priority: Minor > Labels: pull-request-available > Fix For: Impala 4.4.0 > > > when the cluster contains many db and tables such as if there are more than > 10 tables, and if we restart the impalad , the local cache_ > CatalogMetaProvider's need to doing some loading process. > As we know that the goole's guava cache 's concurrencyLevel os set to 4 by > default. > but if there is many tables the loading process will need more time and > increase the probability of lock contention, see > [here|https://github.com/google/guava/blob/master/guava/src/com/google/common/cache/CacheBuilder.java#L437]. > > So we propose to add some configurations here, the first is the concurrency > of cache. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12539) hive-4 compatibility with Impala
[ https://issues.apache.org/jira/browse/IMPALA-12539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Smith updated IMPALA-12539: --- Component/s: Frontend (was: fe) > hive-4 compatibility with Impala > > > Key: IMPALA-12539 > URL: https://issues.apache.org/jira/browse/IMPALA-12539 > Project: IMPALA > Issue Type: Task > Components: Frontend >Affects Versions: Impala 4.1.2 > Environment: Hive-4.0.0-alpha-1 > Impala-4.1.2 >Reporter: Basapuram Kumar >Priority: Major > > As of now , on "fe" it has compatibility with > [hive-3|https://github.com/apache/impala/tree/master/fe/src] only. > It will be nice to add the Hive-4.0.0-alpha-1 compatibility with Impala-4.1.2. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12338) Additional Authentication Test Cases
[ https://issues.apache.org/jira/browse/IMPALA-12338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Smith updated IMPALA-12338: --- Component/s: Frontend (was: fe) > Additional Authentication Test Cases > > > Key: IMPALA-12338 > URL: https://issues.apache.org/jira/browse/IMPALA-12338 > Project: IMPALA > Issue Type: Improvement > Components: Frontend, Security >Reporter: Jason Fehr >Priority: Minor > > The frontend Java unit tests exercise various different authentication > methods separately but do not fully exercise multiple authentication methods > together. Some tests check LDAP/Kerberos together, but more tests are needed > to cover the cases where LDAP, Kerberos, SAML, and JWT authentication are all > enabled at the same time. > These tests will need to ensure clients can authenticate with each method > thus demonstrating that the different authentication methods do not conflict > with each other. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12468) Add the ability to update EventProcessorStatus
[
https://issues.apache.org/jira/browse/IMPALA-12468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Smith updated IMPALA-12468:
---
Component/s: Backend
Frontend
(was: be)
(was: fe)
> Add the ability to update EventProcessorStatus
> --
>
> Key: IMPALA-12468
> URL: https://issues.apache.org/jira/browse/IMPALA-12468
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend, Catalog, Frontend
>Reporter: Maxwell Guo
>Assignee: Maxwell Guo
>Priority: Minor
>
> Once the impala and hive's status is missmatched , and the
> EventProcessorStatus become NEED_INVALIDATE， we usually use invalidate
> metadata to reset the catalog instance. And then impala will update the
> status to ACTIVE .
> But if impala contains many tables , the cost of invalidate is a bit high for
> a global invalidate. So we may invalidate metadata for tables one by one for
> these incremental changed table. For example , we have 1000,000,000,000
> tables but only some of the table event process occurs CatalogException and
> MetastoreNotificationNeedsInvalidateException was thrown. I think there is no
> need to invalidate all table caches in order to reset the catalog instance
> see [here
> |https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java#L2088].
>
> MetaStoresProcessor 's async update process will not update the currentStatus
> when the status is not ACTIVE, see
> [here|https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java#L876]
> So what about add a new SQL grammar : SET EVENT STATUS ${status} ?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12547) Set the default worker threads for compaction to 1
[
https://issues.apache.org/jira/browse/IMPALA-12547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Smith updated IMPALA-12547:
---
Component/s: Frontend
(was: fe)
> Set the default worker threads for compaction to 1
> --
>
> Key: IMPALA-12547
> URL: https://issues.apache.org/jira/browse/IMPALA-12547
> Project: IMPALA
> Issue Type: Wish
> Components: Frontend
>Reporter: Sai Hemanth Gantasala
>Assignee: Sai Hemanth Gantasala
>Priority: Major
>
> Set the default worker threads for compaction to 1.
> Currently worker threads for compaction to 4, as a result, in the external
> frontend mode, there are a lot of calls to HMS in idle state. So, it is
> flooding the catalogD log with unnecessary calls to HMS.
> {code:java}
> I0503 14:13:26.944278 15874 CatalogMetastoreServer.java:215] Invoking HMS
> API: get_current_notificationEventId
> I0503 14:13:26.948185 15874 CatalogMetastoreServer.java:215] Invoking HMS
> API: get_config_value
> I0503 14:13:26.949983 15874 CatalogMetastoreServer.java:215] Invoking HMS
> API: get_next_notification
> I0503 14:13:41.494956 15300 CatalogMetastoreServer.java:215] Invoking HMS
> API: find_next_compact2
> I0503 14:13:41.501145 15303 CatalogMetastoreServer.java:215] Invoking HMS
> API: find_next_compact2
> I0503 14:13:41.507638 15306 CatalogMetastoreServer.java:215] Invoking HMS
> API: find_next_compact2
> I0503 14:13:41.541407 15309 CatalogMetastoreServer.java:215] Invoking HMS
> API: find_next_compact2{code}
> After the above-proposed values for the configs, this is the log info in an
> idle state.
> {code:java}
> I1107 15:58:03.130424 533568 CatalogMetastoreServer.java:215] Invoking HMS
> API: get_next_notification
> I1107 15:58:03.157773 533460 CatalogMetastoreServer.java:215] Invoking HMS
> API: scheduled_query_poll
> I1107 15:58:03.178020 533480 CatalogMetastoreServer.java:215] Invoking HMS
> API: find_next_compact2 {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12589) NPE on querying external tables pointing to a single file
[
https://issues.apache.org/jira/browse/IMPALA-12589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Smith updated IMPALA-12589:
---
Component/s: Frontend
(was: fe)
> NPE on querying external tables pointing to a single file
> -
>
> Key: IMPALA-12589
> URL: https://issues.apache.org/jira/browse/IMPALA-12589
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
>Affects Versions: Impala 4.2.0, Impala 4.3.0
>Reporter: Peter Rozsa
>Assignee: Riza Suminto
>Priority: Major
> Fix For: Impala 4.4.0
>
>
> IMPALA-11507 brought in changes for relative/absolute paths, and made it
> impossible to query external tables pointing to a single file.
> HdfsPartition.getPath() returns null instead of empty path during
> computeEstimatedTableSize().
> Example:
> {code:java}
> create external table t1(c1 int) stored as textfile;
> insert into t1 values (1);
> show files in t1;
>
>
>
> invalidate metadata t1;
> select count(*) from t1; {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12662) Support whitelist for db and table
[ https://issues.apache.org/jira/browse/IMPALA-12662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Smith updated IMPALA-12662: --- Component/s: Frontend (was: be) (was: fe) > Support whitelist for db and table > -- > > Key: IMPALA-12662 > URL: https://issues.apache.org/jira/browse/IMPALA-12662 > Project: IMPALA > Issue Type: Improvement > Components: Frontend >Reporter: Maxwell Guo >Priority: Minor > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12704) NullPointerException occurs when quering metadata of empty Iceberg table
[
https://issues.apache.org/jira/browse/IMPALA-12704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Smith updated IMPALA-12704:
---
Component/s: Frontend
(was: fe)
> NullPointerException occurs when quering metadata of empty Iceberg table
> -
>
> Key: IMPALA-12704
> URL: https://issues.apache.org/jira/browse/IMPALA-12704
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
>Reporter: Zihao Ye
>Assignee: Zihao Ye
>Priority: Critical
> Fix For: Impala 4.4.0
>
>
> {code:sql}
> [4620ee0acf3b:21050] default> create table empty_ice_tbl (id int) stored as
> iceberg;
> Query: create table empty_ice_tbl (id int) stored as iceberg
> +-+
> | summary |
> +-+
> | Table has been created. |
> +-+
> Fetched 1 row(s) in 0.21s
> [4620ee0acf3b:21050] default> select * from default.empty_ice_tbl.entries;
> Query: select * from default.empty_ice_tbl.entries
> Query submitted at: 2024-01-11 10:51:05 (Coordinator:
> http://4620ee0acf3b:25000)
> Query state can be monitored at:
> http://4620ee0acf3b:25000/query_plan?query_id=544ab08315c17508:5d141184
> ERROR: NullPointerException: null
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
