[jira] [Commented] (IMPALA-12222) impala_shell-install-test.tar.gz pip3 could not be installed
[ https://issues.apache.org/jira/browse/IMPALA-1?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17810749#comment-17810749 ] Quanlong Huang commented on IMPALA-1: - I think this is resolved by IMPALA-11846. > impala_shell-install-test.tar.gz pip3 could not be installed > > > Key: IMPALA-1 > URL: https://issues.apache.org/jira/browse/IMPALA-1 > Project: IMPALA > Issue Type: Bug > Components: Distributed Exec >Affects Versions: Impala 4.2.0 >Reporter: bianqi >Priority: Major > > Pip3 prompts that the install-test version cannot be found。 > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12742) DELETE/UPDATE Iceberg table partitioned by DATE fails with error
[ https://issues.apache.org/jira/browse/IMPALA-12742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Noemi Pap-Takacs updated IMPALA-12742: -- Summary: DELETE/UPDATE Iceberg table partitioned by DATE fails with error (was: DELETE/UPDATE Iceberg table partitioned by DATE gives incorrect value) > DELETE/UPDATE Iceberg table partitioned by DATE fails with error > > > Key: IMPALA-12742 > URL: https://issues.apache.org/jira/browse/IMPALA-12742 > Project: IMPALA > Issue Type: Bug > Components: Backend, Catalog >Reporter: Noemi Pap-Takacs >Priority: Major > Labels: impala-iceberg > > Iceberg tables can be identity partitioned by any type, e.g. int, date and > even float. > If a table is partitioned, the file path contains the partition value in > human readable form. When an UPDATE or DELETE command is executed, the delete > file contains the file path to the referenced data file. It seems that DATE > type is converted to this form incorrectly, and cannot be parsed by the > Catalog and throws an error. > > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-12709) Hierarchical metastore event processing
[ https://issues.apache.org/jira/browse/IMPALA-12709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17810774#comment-17810774 ] Venugopal Reddy K commented on IMPALA-12709: [~maxwellguo] I am doing a small proof of concept based on the design doc. Actively working on it. Once it is done, will add the poc code link to the design doc for reference. Parallely working on the review comments in the design doc as well. Will keep updating the status of it periodically in this jira. > Hierarchical metastore event processing > --- > > Key: IMPALA-12709 > URL: https://issues.apache.org/jira/browse/IMPALA-12709 > Project: IMPALA > Issue Type: Improvement > Components: Catalog >Reporter: Venugopal Reddy K >Assignee: Venugopal Reddy K >Priority: Major > Attachments: Hierarchical metastore event processing.docx > > > *Current Issue:* > At present, metastore event processor is single threaded. Notification events > are processed sequentially with a maximum limit of 1000 events fetched and > processed in a single batch. Multiple locks are used to address the > concurrency issues that may arise when catalog DDL operation processing and > metastore event processing tries to access/update the catalog objects > concurrently. Waiting for a lock or file metadata loading of a table can slow > the event processing and can affect the processing of other events following > it. Those events may not be dependent on the previous event. Altogether it > takes a very long time to synchronize all the HMS events. > *Proposal:* > Existing metastore event processing can be turned into multi-level event > processing. Idea is to segregate the events based on their dependency, > maintain the order of events as they occur within the dependency and process > them independently as much as possible: > # All the events of a table are processed in the same order they have > actually occurred. > # Events of different tables are processed in parallel. > # When a database is altered, all the events relating to the database(i.e., > for all its tables) occurring after the alter db event are processed only > after the alter database event is processed ensuring the order. > Have attached an initial proposal design document > https://docs.google.com/document/d/1KZ-ANko-qn5CYmY13m4OVJXAYjLaS1VP-c64Pumipq8/edit?pli=1#heading=h.qyk8qz8ez37b -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Assigned] (IMPALA-12742) DELETE/UPDATE Iceberg table partitioned by DATE fails with error
[ https://issues.apache.org/jira/browse/IMPALA-12742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltán Borók-Nagy reassigned IMPALA-12742: -- Assignee: Zoltán Borók-Nagy > DELETE/UPDATE Iceberg table partitioned by DATE fails with error > > > Key: IMPALA-12742 > URL: https://issues.apache.org/jira/browse/IMPALA-12742 > Project: IMPALA > Issue Type: Bug > Components: Backend, Catalog >Reporter: Noemi Pap-Takacs >Assignee: Zoltán Borók-Nagy >Priority: Major > Labels: impala-iceberg > > Iceberg tables can be identity partitioned by any type, e.g. int, date and > even float. > If a table is partitioned, the file path contains the partition value in > human readable form. When an UPDATE or DELETE command is executed, the delete > file contains the file path to the referenced data file. It seems that DATE > type is converted to this form incorrectly, and cannot be parsed by the > Catalog and throws an error. > > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-12709) Hierarchical metastore event processing
[ https://issues.apache.org/jira/browse/IMPALA-12709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17810813#comment-17810813 ] Venugopal Reddy K commented on IMPALA-12709: [~rizaon] Thats exactly what i am trying to do. Group 4(db event) in the example is a barrier event. All the table events(of the db) occurred prior to the barrier event are ensured to be processed before processing it. And, all the table events(of the db) occurred after the barrier event are ensured to be processed only after barrier event processing. > Hierarchical metastore event processing > --- > > Key: IMPALA-12709 > URL: https://issues.apache.org/jira/browse/IMPALA-12709 > Project: IMPALA > Issue Type: Improvement > Components: Catalog >Reporter: Venugopal Reddy K >Assignee: Venugopal Reddy K >Priority: Major > Attachments: Hierarchical metastore event processing.docx > > > *Current Issue:* > At present, metastore event processor is single threaded. Notification events > are processed sequentially with a maximum limit of 1000 events fetched and > processed in a single batch. Multiple locks are used to address the > concurrency issues that may arise when catalog DDL operation processing and > metastore event processing tries to access/update the catalog objects > concurrently. Waiting for a lock or file metadata loading of a table can slow > the event processing and can affect the processing of other events following > it. Those events may not be dependent on the previous event. Altogether it > takes a very long time to synchronize all the HMS events. > *Proposal:* > Existing metastore event processing can be turned into multi-level event > processing. Idea is to segregate the events based on their dependency, > maintain the order of events as they occur within the dependency and process > them independently as much as possible: > # All the events of a table are processed in the same order they have > actually occurred. > # Events of different tables are processed in parallel. > # When a database is altered, all the events relating to the database(i.e., > for all its tables) occurring after the alter db event are processed only > after the alter database event is processed ensuring the order. > Have attached an initial proposal design document > https://docs.google.com/document/d/1KZ-ANko-qn5CYmY13m4OVJXAYjLaS1VP-c64Pumipq8/edit?pli=1#heading=h.qyk8qz8ez37b -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Work started] (IMPALA-12600) Support equality deletes when table has partition or schema evolution
[ https://issues.apache.org/jira/browse/IMPALA-12600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on IMPALA-12600 started by Gabor Kaszab. - > Support equality deletes when table has partition or schema evolution > - > > Key: IMPALA-12600 > URL: https://issues.apache.org/jira/browse/IMPALA-12600 > Project: IMPALA > Issue Type: Sub-task >Reporter: Gabor Kaszab >Assignee: Gabor Kaszab >Priority: Major > > With adding the basic equality delete read support, we reject queries for > Iceberg tables that has equality delete files and has partition or schema > evolution. This ticket is to enhance this functionality. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-12709) Hierarchical metastore event processing
[ https://issues.apache.org/jira/browse/IMPALA-12709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17810952#comment-17810952 ] Riza Suminto commented on IMPALA-12709: --- [~VenuReddy] what I want to point out is, you might be able to avoid having 2 different thread pool and event routing between them if you pre-process the incoming batch into a DAG of non-overlapping mini-batches. You can process each DAG node that does not have predecessor in parallel and remove them from the DAG upon completion. Then, repeat the process until all mini-batches removed from DAG. That way, you just need 1 thread pool and no event routing. > Hierarchical metastore event processing > --- > > Key: IMPALA-12709 > URL: https://issues.apache.org/jira/browse/IMPALA-12709 > Project: IMPALA > Issue Type: Improvement > Components: Catalog >Reporter: Venugopal Reddy K >Assignee: Venugopal Reddy K >Priority: Major > Attachments: Hierarchical metastore event processing.docx > > > *Current Issue:* > At present, metastore event processor is single threaded. Notification events > are processed sequentially with a maximum limit of 1000 events fetched and > processed in a single batch. Multiple locks are used to address the > concurrency issues that may arise when catalog DDL operation processing and > metastore event processing tries to access/update the catalog objects > concurrently. Waiting for a lock or file metadata loading of a table can slow > the event processing and can affect the processing of other events following > it. Those events may not be dependent on the previous event. Altogether it > takes a very long time to synchronize all the HMS events. > *Proposal:* > Existing metastore event processing can be turned into multi-level event > processing. Idea is to segregate the events based on their dependency, > maintain the order of events as they occur within the dependency and process > them independently as much as possible: > # All the events of a table are processed in the same order they have > actually occurred. > # Events of different tables are processed in parallel. > # When a database is altered, all the events relating to the database(i.e., > for all its tables) occurring after the alter db event are processed only > after the alter database event is processed ensuring the order. > Have attached an initial proposal design document > https://docs.google.com/document/d/1KZ-ANko-qn5CYmY13m4OVJXAYjLaS1VP-c64Pumipq8/edit?pli=1#heading=h.qyk8qz8ez37b -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-12709) Hierarchical metastore event processing
[ https://issues.apache.org/jira/browse/IMPALA-12709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17810960#comment-17810960 ] Maxwell Guo commented on IMPALA-12709: -- [~VenuReddy]thank for your reply ,looking forward to your update. > Hierarchical metastore event processing > --- > > Key: IMPALA-12709 > URL: https://issues.apache.org/jira/browse/IMPALA-12709 > Project: IMPALA > Issue Type: Improvement > Components: Catalog >Reporter: Venugopal Reddy K >Assignee: Venugopal Reddy K >Priority: Major > Attachments: Hierarchical metastore event processing.docx > > > *Current Issue:* > At present, metastore event processor is single threaded. Notification events > are processed sequentially with a maximum limit of 1000 events fetched and > processed in a single batch. Multiple locks are used to address the > concurrency issues that may arise when catalog DDL operation processing and > metastore event processing tries to access/update the catalog objects > concurrently. Waiting for a lock or file metadata loading of a table can slow > the event processing and can affect the processing of other events following > it. Those events may not be dependent on the previous event. Altogether it > takes a very long time to synchronize all the HMS events. > *Proposal:* > Existing metastore event processing can be turned into multi-level event > processing. Idea is to segregate the events based on their dependency, > maintain the order of events as they occur within the dependency and process > them independently as much as possible: > # All the events of a table are processed in the same order they have > actually occurred. > # Events of different tables are processed in parallel. > # When a database is altered, all the events relating to the database(i.e., > for all its tables) occurring after the alter db event are processed only > after the alter database event is processed ensuring the order. > Have attached an initial proposal design document > https://docs.google.com/document/d/1KZ-ANko-qn5CYmY13m4OVJXAYjLaS1VP-c64Pumipq8/edit?pli=1#heading=h.qyk8qz8ez37b -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-12746) Bump jackson-databind version to 2.15
[ https://issues.apache.org/jira/browse/IMPALA-12746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17810991#comment-17810991 ] ASF subversion and git services commented on IMPALA-12746: -- Commit c14156eb3a95036d07ae684a046cf15c220066a5 in impala's branch refs/heads/master from Csaba Ringhofer [ https://gitbox.apache.org/repos/asf?p=impala.git;h=c14156eb3 ] IMPALA-12746: Bump jackson.databind to 2.15.3 Also sets dependencyManagement to force using the same version for jackson-databind, jackson-core and jackon-annotations. This is needed because datagenerator depends on kitesdk, which would pull in a very old jackson-core version (2.3.1) and lead to build failures with the newer jackson.databind. Change-Id: I8440426da1395045cf149aca0044286015861e5f Reviewed-on: http://gerrit.cloudera.org:8080/20914 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins > Bump jackson-databind version to 2.15 > - > > Key: IMPALA-12746 > URL: https://issues.apache.org/jira/browse/IMPALA-12746 > Project: IMPALA > Issue Type: Task >Reporter: Csaba Ringhofer >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-12564) ARM TSAN test run fails to start Hive Server
[
https://issues.apache.org/jira/browse/IMPALA-12564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17810995#comment-17810995
]
Michael Smith commented on IMPALA-12564:
This might not be an ARM issue. I was running this test on Ubuntu 22.04. After
recently upgrading my AMD64 workstation to Ubuntu 22.04 I also run into this. I
found
{code}
/usr/lib/jvm/java-8-openjdk-amd64/bin/java: symbol lookup error:
/home/michael/Impala/be/build/debug/service/libfesupport.so: undefined symbol:
__tsan_init
{code}
in logs/cluster/hive/hive-server2.out.
> ARM TSAN test run fails to start Hive Server
>
>
> Key: IMPALA-12564
> URL: https://issues.apache.org/jira/browse/IMPALA-12564
> Project: IMPALA
> Issue Type: Bug
>Reporter: Michael Smith
>Assignee: Michael Smith
>Priority: Major
>
> Hive Server startup hangs during ARM TSAN test runs. Logs were not
> particularly illuminating.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12564) TSAN test run fails to start Hive Server on Ubuntu 22.04
[
https://issues.apache.org/jira/browse/IMPALA-12564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Smith updated IMPALA-12564:
---
Summary: TSAN test run fails to start Hive Server on Ubuntu 22.04 (was:
ARM TSAN test run fails to start Hive Server)
> TSAN test run fails to start Hive Server on Ubuntu 22.04
>
>
> Key: IMPALA-12564
> URL: https://issues.apache.org/jira/browse/IMPALA-12564
> Project: IMPALA
> Issue Type: Bug
>Reporter: Michael Smith
>Assignee: Michael Smith
>Priority: Major
>
> Hive Server startup hangs during ARM TSAN test runs. Logs were not
> particularly illuminating.
> hive-server2.out shows
> {code}
> /usr/lib/jvm/java-8-openjdk-amd64/bin/java: symbol lookup error:
> /home/michael/Impala/be/build/debug/service/libfesupport.so: undefined
> symbol: __tsan_init
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12564) ARM TSAN test run fails to start Hive Server
[
https://issues.apache.org/jira/browse/IMPALA-12564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Smith updated IMPALA-12564:
---
Description:
Hive Server startup hangs during ARM TSAN test runs. Logs were not particularly
illuminating.
hive-server2.out shows
{code}
/usr/lib/jvm/java-8-openjdk-amd64/bin/java: symbol lookup error:
/home/michael/Impala/be/build/debug/service/libfesupport.so: undefined symbol:
__tsan_init
{code}
was:Hive Server startup hangs during ARM TSAN test runs. Logs were not
particularly illuminating.
> ARM TSAN test run fails to start Hive Server
>
>
> Key: IMPALA-12564
> URL: https://issues.apache.org/jira/browse/IMPALA-12564
> Project: IMPALA
> Issue Type: Bug
>Reporter: Michael Smith
>Assignee: Michael Smith
>Priority: Major
>
> Hive Server startup hangs during ARM TSAN test runs. Logs were not
> particularly illuminating.
> hive-server2.out shows
> {code}
> /usr/lib/jvm/java-8-openjdk-amd64/bin/java: symbol lookup error:
> /home/michael/Impala/be/build/debug/service/libfesupport.so: undefined
> symbol: __tsan_init
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12564) TSAN test run fails to start Hive Server on Ubuntu 22.04
[
https://issues.apache.org/jira/browse/IMPALA-12564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Smith updated IMPALA-12564:
---
Description:
Hive Server startup hangs during ARM TSAN test runs. Logs were not particularly
illuminating.
hive-server2.out shows
{code}
/usr/lib/jvm/java-8-openjdk-amd64/bin/java: symbol lookup error:
/home/michael/Impala/be/build/debug/service/libfesupport.so: undefined symbol:
__tsan_init
{code}
Can work around it locally by commenting out
https://github.com/apache/impala/blob/4.3.0/testdata/bin/run-hive-server.sh#L140-L146.
was:
Hive Server startup hangs during ARM TSAN test runs. Logs were not particularly
illuminating.
hive-server2.out shows
{code}
/usr/lib/jvm/java-8-openjdk-amd64/bin/java: symbol lookup error:
/home/michael/Impala/be/build/debug/service/libfesupport.so: undefined symbol:
__tsan_init
{code}
> TSAN test run fails to start Hive Server on Ubuntu 22.04
>
>
> Key: IMPALA-12564
> URL: https://issues.apache.org/jira/browse/IMPALA-12564
> Project: IMPALA
> Issue Type: Bug
>Reporter: Michael Smith
>Assignee: Michael Smith
>Priority: Major
>
> Hive Server startup hangs during ARM TSAN test runs. Logs were not
> particularly illuminating.
> hive-server2.out shows
> {code}
> /usr/lib/jvm/java-8-openjdk-amd64/bin/java: symbol lookup error:
> /home/michael/Impala/be/build/debug/service/libfesupport.so: undefined
> symbol: __tsan_init
> {code}
> Can work around it locally by commenting out
> https://github.com/apache/impala/blob/4.3.0/testdata/bin/run-hive-server.sh#L140-L146.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12747) internal-server-test crashes with SIGSEGV
[
https://issues.apache.org/jira/browse/IMPALA-12747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Laszlo Gaal updated IMPALA-12747:
-
Summary: internal-server-test crashes with SIGSEGV (was:
internal-server-test crashes with SIGSEGV on ARM in release build)
> internal-server-test crashes with SIGSEGV
> -
>
> Key: IMPALA-12747
> URL: https://issues.apache.org/jira/browse/IMPALA-12747
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 4.4.0
>Reporter: Laszlo Gaal
>Assignee: Michael Smith
>Priority: Blocker
> Labels: arm64, broken-build
>
> internal-server-test crashed in
> InternalServerTest.SimultaneousMultipleQueriesOneSession with minidumps in a
> release-mode build on ARM.
> The minidump was resolved to the below call stack:
> {code}
> Operating system: Linux
> 0.0.0 Linux 4.18.0-477.15.1.el8_8.aarch64 #1 SMP Fri Jun 2
> 08:39:44 EDT 2023 aarch64
> CPU: arm64
> 16 CPUs
> GPU: UNKNOWN
> Crash reason: SIGSEGV /SEGV_MAPERR
> Crash address: 0x8
> Process uptime: not available
> Thread 538 (crashed)
> 0 internal-server-test!impala::TPlanNode::TPlanNode(impala::TPlanNode
> const&) [PlanNodes_types.cpp : 3124 + 0x0]
> x0 = 0x4d600020x1 = 0x0020
> x2 = 0xx3 = 0x
> x4 = 0x4d600900x5 = 0x4d600a00
> x6 = 0xx7 = 0x4d600918
> x8 = 0x4d600060x9 = 0x4d600828
> x10 = 0x04a3ab48 x11 = 0x4d600838
> x12 = 0x4d600868 x13 = 0x4d6007f8
> x14 = 0x4d600818 x15 = 0x4d6007d0
> x16 = 0x4d600848 x17 = 0x4d600878
> x18 = 0x04a3ab98 x19 = 0x4d60
> x20 = 0x x21 = 0x4d600098
> x22 = 0x4d6001e0 x23 = 0x4d600038
> x24 = 0x4d600190 x25 = 0x4d6003d8
> x26 = 0x4d600560 x27 = 0x4d600080
> x28 = 0x4d6008b8fp = 0xfffe4df61990
> lr = 0x4d600798sp = 0xfffe4df61990
> pc = 0x01023dc4
> Found by: given as instruction pointer in context
> 1 internal-server-test!std::vector std::allocator >::operator=(std::vector std::allocator > const&) [stl_construct.h : 109 + 0x8]
> x19 = 0x x20 = 0x4d60
> x21 = 0x4497b038 x22 = 0x4d60
> x23 = 0x45a9bb30 x24 = 0x45a9bb30
> x25 = 0x4497b020 x26 = 0x390da838
> x27 = 0x461d7000 x28 = 0xfffe4df62018
> fp = 0xfffe4df61a50sp = 0xfffe4df61a50
> pc = 0x010242f4
> Found by: call frame info
> 2 internal-server-test!impala::TPlan::operator=(impala::TPlan const&)
> [PlanNodes_types.cpp : 3305 + 0x0]
> x19 = 0x4497b030 x20 = 0x390da800
> x21 = 0x4497b010 x22 = 0x4497b400
> x23 = 0x4497b050 x24 = 0x4497b328
> x25 = 0x4497b020 x26 = 0x4497b030
> x27 = 0x461d7000 x28 = 0xfffe4df62018
> fp = 0xfffe4df61aa0sp = 0xfffe4df61aa0
> pc = 0x010244ac
> Found by: call frame info
> 3
> internal-server-test!impala::TPlanFragment::TPlanFragment(impala::TPlanFragment
> const&) [Planner_types.cpp : 110 + 0x8]
> x19 = 0x4497b000 x20 = 0x390da800
> x21 = 0x4497b010 x22 = 0x4497b400
> x23 = 0x4497b050 x24 = 0x4497b328
> x25 = 0x4497b020 x26 = 0x4497b030
> x27 = 0x461d7000 x28 = 0xfffe4df62018
> fp = 0xfffe4df61ac0sp = 0xfffe4df61ac0
> pc = 0x0102cd40
> Found by: call frame info
> 4
> internal-server-test!impala::ImpalaServer::QueryStateRecord::Init(impala::ClientRequestState
> const&) [stl_construct.h : 109 + 0x8]
> x19 = 0x390da800 x20 = 0x4497b000
> x21 = 0x390daf30 x22 = 0x0730
> x23 = 0x4497b000 x24 = 0xfffe4df61d50
> x25 = 0x x26 = 0x462ed320
> x27 = 0x461d7000 x28 = 0xfffe4df62018
> fp = 0xfffe4df61b10sp = 0xfffe4df61b10
> pc = 0x014d80a4
> Found by: call frame info
> 5
> internal-server-test!impala::ImpalaServer::QueryStateRecord::QueryStateRecord(impala::ClientRequestState
> const&) [impala-server.cc : 2529 + 0x0]
> x19 = 0xfffe4df61d50 x20 = 0xfffe4df61eb8
> x21 = 0xfffe4df61df8 x22 = 0xfffe4df61e38
> x23 = 0xfffe4df61e48 x24 = 0xfffe4df61e98
> x25 = 0xfffe4df61ea8 x26 = 0xfffe4df62018
> x27 =
[jira] [Commented] (IMPALA-12747) internal-server-test crashes with SIGSEGV
[
https://issues.apache.org/jira/browse/IMPALA-12747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17811006#comment-17811006
]
Laszlo Gaal commented on IMPALA-12747:
--
Dropped "ARM" from the title: just saw the same crash in an amd64 running on
JRE 17.
> internal-server-test crashes with SIGSEGV
> -
>
> Key: IMPALA-12747
> URL: https://issues.apache.org/jira/browse/IMPALA-12747
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 4.4.0
>Reporter: Laszlo Gaal
>Assignee: Michael Smith
>Priority: Blocker
> Labels: arm64, broken-build
>
> internal-server-test crashed in
> InternalServerTest.SimultaneousMultipleQueriesOneSession with minidumps in a
> release-mode build on ARM.
> The minidump was resolved to the below call stack:
> {code}
> Operating system: Linux
> 0.0.0 Linux 4.18.0-477.15.1.el8_8.aarch64 #1 SMP Fri Jun 2
> 08:39:44 EDT 2023 aarch64
> CPU: arm64
> 16 CPUs
> GPU: UNKNOWN
> Crash reason: SIGSEGV /SEGV_MAPERR
> Crash address: 0x8
> Process uptime: not available
> Thread 538 (crashed)
> 0 internal-server-test!impala::TPlanNode::TPlanNode(impala::TPlanNode
> const&) [PlanNodes_types.cpp : 3124 + 0x0]
> x0 = 0x4d600020x1 = 0x0020
> x2 = 0xx3 = 0x
> x4 = 0x4d600900x5 = 0x4d600a00
> x6 = 0xx7 = 0x4d600918
> x8 = 0x4d600060x9 = 0x4d600828
> x10 = 0x04a3ab48 x11 = 0x4d600838
> x12 = 0x4d600868 x13 = 0x4d6007f8
> x14 = 0x4d600818 x15 = 0x4d6007d0
> x16 = 0x4d600848 x17 = 0x4d600878
> x18 = 0x04a3ab98 x19 = 0x4d60
> x20 = 0x x21 = 0x4d600098
> x22 = 0x4d6001e0 x23 = 0x4d600038
> x24 = 0x4d600190 x25 = 0x4d6003d8
> x26 = 0x4d600560 x27 = 0x4d600080
> x28 = 0x4d6008b8fp = 0xfffe4df61990
> lr = 0x4d600798sp = 0xfffe4df61990
> pc = 0x01023dc4
> Found by: given as instruction pointer in context
> 1 internal-server-test!std::vector std::allocator >::operator=(std::vector std::allocator > const&) [stl_construct.h : 109 + 0x8]
> x19 = 0x x20 = 0x4d60
> x21 = 0x4497b038 x22 = 0x4d60
> x23 = 0x45a9bb30 x24 = 0x45a9bb30
> x25 = 0x4497b020 x26 = 0x390da838
> x27 = 0x461d7000 x28 = 0xfffe4df62018
> fp = 0xfffe4df61a50sp = 0xfffe4df61a50
> pc = 0x010242f4
> Found by: call frame info
> 2 internal-server-test!impala::TPlan::operator=(impala::TPlan const&)
> [PlanNodes_types.cpp : 3305 + 0x0]
> x19 = 0x4497b030 x20 = 0x390da800
> x21 = 0x4497b010 x22 = 0x4497b400
> x23 = 0x4497b050 x24 = 0x4497b328
> x25 = 0x4497b020 x26 = 0x4497b030
> x27 = 0x461d7000 x28 = 0xfffe4df62018
> fp = 0xfffe4df61aa0sp = 0xfffe4df61aa0
> pc = 0x010244ac
> Found by: call frame info
> 3
> internal-server-test!impala::TPlanFragment::TPlanFragment(impala::TPlanFragment
> const&) [Planner_types.cpp : 110 + 0x8]
> x19 = 0x4497b000 x20 = 0x390da800
> x21 = 0x4497b010 x22 = 0x4497b400
> x23 = 0x4497b050 x24 = 0x4497b328
> x25 = 0x4497b020 x26 = 0x4497b030
> x27 = 0x461d7000 x28 = 0xfffe4df62018
> fp = 0xfffe4df61ac0sp = 0xfffe4df61ac0
> pc = 0x0102cd40
> Found by: call frame info
> 4
> internal-server-test!impala::ImpalaServer::QueryStateRecord::Init(impala::ClientRequestState
> const&) [stl_construct.h : 109 + 0x8]
> x19 = 0x390da800 x20 = 0x4497b000
> x21 = 0x390daf30 x22 = 0x0730
> x23 = 0x4497b000 x24 = 0xfffe4df61d50
> x25 = 0x x26 = 0x462ed320
> x27 = 0x461d7000 x28 = 0xfffe4df62018
> fp = 0xfffe4df61b10sp = 0xfffe4df61b10
> pc = 0x014d80a4
> Found by: call frame info
> 5
> internal-server-test!impala::ImpalaServer::QueryStateRecord::QueryStateRecord(impala::ClientRequestState
> const&) [impala-server.cc : 2529 + 0x0]
> x19 = 0xfffe4df61d50 x20 = 0xfffe4df61eb8
> x21 = 0xfffe4df61df8 x22 = 0xfffe4df61e38
> x23 = 0xfffe4df61e48 x24 = 0xfffe4df61e98
> x25 = 0xfffe4df61ea8 x26 = 0xfffe4df620
[jira] [Updated] (IMPALA-12542) test_query_cancel_created failed in ASAN build
[
https://issues.apache.org/jira/browse/IMPALA-12542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Laszlo Gaal updated IMPALA-12542:
-
Priority: Blocker (was: Major)
> test_query_cancel_created failed in ASAN build
> --
>
> Key: IMPALA-12542
> URL: https://issues.apache.org/jira/browse/IMPALA-12542
> Project: IMPALA
> Issue Type: Test
>Affects Versions: Impala 4.4.0
>Reporter: Riza Suminto
>Assignee: Michael Smith
>Priority: Blocker
> Labels: broken-build
> Fix For: Impala 4.4.0
>
>
> IMPALA-12493 add test_query_cancel_created. It is failing in ASAN build with
> following error:
> {code:java}
> Error Message
> assert 1 == 0
> Stacktrace
> webserver/test_web_pages.py:961: in test_query_cancel_created
> assert response_json['num_in_flight_queries'] == 0
> E assert 1 == 0 {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-12757) TSAN flags lock-order-inversion during internal-server-test
Michael Smith created IMPALA-12757:
--
Summary: TSAN flags lock-order-inversion during
internal-server-test
Key: IMPALA-12757
URL: https://issues.apache.org/jira/browse/IMPALA-12757
Project: IMPALA
Issue Type: Bug
Affects Versions: Impala 4.4.0
Reporter: Michael Smith
internal-server-test has a tight loop starting queries and fetching /queries.
That's led to identifying several latent threading issues. Once IMPALA-12747 is
addressed, TSAN identifies a new error:
{code}
$ run-jvm-binary.sh ./be/build/debug/service/internal-server-test
I20240125 10:44:44.971467 152733 openssl_util.cc:110] FIPS mode is disabled.
Picked up JAVA_TOOL_OPTIONS: -Dsun.java.command=internal-server-test
24/01/25 10:44:45 WARN fs.FileSystem: Cannot load filesystem:
java.util.ServiceConfigurationError: org.apache.hadoop.fs.FileSystem: Provider
org.apache.hadoop.hive.ql.io.NullScanFileSystem not found
24/01/25 10:44:45 WARN fs.FileSystem: Cannot load filesystem:
java.util.ServiceConfigurationError: org.apache.hadoop.fs.FileSystem: Provider
org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem not found
24/01/25 10:44:45 INFO util.JvmPauseMonitor: Starting JVM pause monitor
[==] Running 10 tests from 1 test case.
[--] Global test environment set-up.
[--] 10 tests from InternalServerTest
[ RUN ] InternalServerTest.QueryTimeout
[ OK ] InternalServerTest.QueryTimeout (10236 ms)
[ RUN ] InternalServerTest.InvalidQueryOption
[ OK ] InternalServerTest.InvalidQueryOption (76 ms)
[ RUN ] InternalServerTest.MultipleQueriesMultipleSessions
/home/michael/Impala/be/src/service/internal-server-test.cc:289: Failure
Value of: status_.ok()
Actual: false
Expected: true
Error: Failed due to unreachable impalad(s): michaelsmith-22742:27000
[ FAILED ] InternalServerTest.MultipleQueriesMultipleSessions (17225 ms)
[ RUN ] InternalServerTest.RetryFailedQuery
[ OK ] InternalServerTest.RetryFailedQuery (1206 ms)
[ RUN ] InternalServerTest.MultipleQueriesOneSession
^C==
WARNING: ThreadSanitizer: thread leak (pid=152733)
Thread T1086 (tid=154103, finished) created by main thread at:
#0 pthread_create (internal-server-test+0x203c383)
#1 boost::thread::start_thread_noexcept()
(internal-server-test+0x3bcd3fd)
#2 boost::thread::thread, std::allocator > const&,
std::__cxx11::basic_string, std::allocator >
const&, boost::function, impala::ThreadDebugInfo const*,
impala::Promise*),
std::__cxx11::basic_string, std::allocator
>, std::__cxx11::basic_string,
std::allocator >, boost::function, impala::ThreadDebugInfo*,
impala::Promise*>(void
(*)(std::__cxx11::basic_string,
std::allocator > const&, std::__cxx11::basic_string, std::allocator > const&, boost::function, impala::ThreadDebugInfo const*, impala::Promise*), std::__cxx11::basic_string, std::allocator >,
std::__cxx11::basic_string, std::allocator
>, boost::function, impala::ThreadDebugInfo*, impala::Promise*)
/home/michael/Impala/toolchain/toolchain-packages-gcc10.4.0/boost-1.74.0-p1/include/boost/thread/detail/thread.hpp:424:13
(internal-server-test+0x2d84514)
#3 impala::Thread::StartThread(std::__cxx11::basic_string, std::allocator > const&,
std::__cxx11::basic_string, std::allocator >
const&, boost::function const&, std::unique_ptr >*, bool)
/home/michael/Impala/be/src/util/thread.cc:317:13
(internal-server-test+0x2d8091c)
#4 impala::Status impala::Thread::Create(std::__cxx11::basic_string, std::allocator > const&,
std::__cxx11::basic_string, std::allocator >
const&, void (impala::ClientRequestState::* const&)(),
impala::ClientRequestState* const&, std::unique_ptr >*, bool)
/home/michael/Impala/be/src/util/thread.h:81:12 (internal-server-test+0x2b382f7)
#5 impala::ClientRequestState::WaitAsync()
/home/michael/Impala/be/src/service/client-request-state.cc:1126:10
(internal-server-test+0x2b2f9f4)
#6 impala::ImpalaServer::WaitForResults(impala::TUniqueId&)
/home/michael/Impala/be/src/service/internal-server.cc:156:3
(internal-server-test+0x2ad148e)
#7 non-virtual thunk to
impala::ImpalaServer::WaitForResults(impala::TUniqueId&)
/home/michael/Impala/be/src/service/internal-server.cc
(internal-server-test+0x2ad15fd)
#8
impala::internalservertest::InternalServerTest_MultipleQueriesOneSession_Test::TestBody()
/home/michael/Impala/be/src/service/internal-server-test.cc:380:3
(internal-server-test+0x209fb9c)
#9 void
testing::internal::HandleExceptionsInMethodIfSupported(testing::Test*, void (testing::Test::*)(), char const*)
(internal-server-test+0x5212c1c)
#10 (libc.so.6+0x29d8f)
SUMMARY: ThreadSanitizer: thread leak
(/home/michael/Impala/be/build/debug/service/internal-server-test+0x203c383) in
__interceptor_pthread_create
==
michael@michaelsmith-22742 ~/Impala ❯❯❯ TSAN_OPTIONS=second_deadlock_stack=1
run-j
[jira] [Created] (IMPALA-12759) Investigate missing alter table event for a truncate operation on Iceberg table
Sai Hemanth Gantasala created IMPALA-12759: -- Summary: Investigate missing alter table event for a truncate operation on Iceberg table Key: IMPALA-12759 URL: https://issues.apache.org/jira/browse/IMPALA-12759 Project: IMPALA Issue Type: Wish Reporter: Sai Hemanth Gantasala Assignee: Sai Hemanth Gantasala Currently, we are not firing an alter table event to metastore when there is a truncate operation on an Iceberg table. As a result, other impala clusters consuming the events from metastore are unaware of that truncate operation resulting in (meta)data sync issues. Need to investigate why the alter table event is not fired to metastore for a truncate table operation on Iceberg table. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12760) Investigate missing insert table event for an insert operation on Iceberg table
[ https://issues.apache.org/jira/browse/IMPALA-12760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sai Hemanth Gantasala updated IMPALA-12760: --- Description: Currently, we are not firing an insert table event to metastore when there is an insert operation on an Iceberg table. As a result, other impala clusters consuming the events from metastore are unaware of that insert operation resulting in (meta)data sync issues. Need to investigate why the insert table event is not fired to metastore for a insert table operation on Iceberg table. h4. was: Currently, we are not firing an insert table event to metastore when there is an insert operation on an Iceberg table. As a result, other impala clusters consuming the events from metastore are unaware of that insert operation resulting in (meta)data sync issues. Need to investigate why the alter table event is not fired to metastore for a truncate table operation on Iceberg table. h4. > Investigate missing insert table event for an insert operation on Iceberg > table > --- > > Key: IMPALA-12760 > URL: https://issues.apache.org/jira/browse/IMPALA-12760 > Project: IMPALA > Issue Type: Wish >Reporter: Sai Hemanth Gantasala >Assignee: Sai Hemanth Gantasala >Priority: Major > > Currently, we are not firing an insert table event to metastore when there is > an insert operation on an Iceberg table. As a result, other impala clusters > consuming the events from metastore are unaware of that insert operation > resulting in (meta)data sync issues. > Need to investigate why the insert table event is not fired to metastore for > a insert table operation on Iceberg table. > h4. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12760) Investigate missing insert table event for an insert operation on Iceberg table
[ https://issues.apache.org/jira/browse/IMPALA-12760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sai Hemanth Gantasala updated IMPALA-12760: --- Description: Currently, we are not firing an insert table event to metastore when there is an insert operation on an Iceberg table. As a result, other impala clusters consuming the events from metastore are unaware of that insert operation resulting in (meta)data sync issues. Need to investigate why the alter table event is not fired to metastore for a truncate table operation on Iceberg table. h4. was: Currently, we are not firing an alter table event to metastore when there is a truncate operation on an Iceberg table. As a result, other impala clusters consuming the events from metastore are unaware of that truncate operation resulting in (meta)data sync issues. Need to investigate why the alter table event is not fired to metastore for a truncate table operation on Iceberg table. h4. > Investigate missing insert table event for an insert operation on Iceberg > table > --- > > Key: IMPALA-12760 > URL: https://issues.apache.org/jira/browse/IMPALA-12760 > Project: IMPALA > Issue Type: Wish >Reporter: Sai Hemanth Gantasala >Assignee: Sai Hemanth Gantasala >Priority: Major > > Currently, we are not firing an insert table event to metastore when there is > an insert operation on an Iceberg table. As a result, other impala clusters > consuming the events from metastore are unaware of that insert operation > resulting in (meta)data sync issues. > Need to investigate why the alter table event is not fired to metastore for a > truncate table operation on Iceberg table. > h4. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12760) Investigate missing insert table event for an insert operation on Iceberg table
[ https://issues.apache.org/jira/browse/IMPALA-12760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sai Hemanth Gantasala updated IMPALA-12760: --- Summary: Investigate missing insert table event for an insert operation on Iceberg table (was: I) > Investigate missing insert table event for an insert operation on Iceberg > table > --- > > Key: IMPALA-12760 > URL: https://issues.apache.org/jira/browse/IMPALA-12760 > Project: IMPALA > Issue Type: Wish >Reporter: Sai Hemanth Gantasala >Assignee: Sai Hemanth Gantasala >Priority: Major > > Currently, we are not firing an alter table event to metastore when there is > a truncate operation on an Iceberg table. As a result, other impala clusters > consuming the events from metastore are unaware of that truncate operation > resulting in (meta)data sync issues. > Need to investigate why the alter table event is not fired to metastore for a > truncate table operation on Iceberg table. > h4. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12760) Investigate missing insert table event for an insert operation on Iceberg table
[ https://issues.apache.org/jira/browse/IMPALA-12760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sai Hemanth Gantasala updated IMPALA-12760: --- Description: Currently, we are not firing an insert table event to metastore when there is an insert operation on an Iceberg table. As a result, other impala clusters consuming the events from metastore are unaware of that insert operation resulting in (meta)data sync issues. Need to investigate why the insert table event is not fired to metastore for an insert table operation on Iceberg table. h4. was: Currently, we are not firing an insert table event to metastore when there is an insert operation on an Iceberg table. As a result, other impala clusters consuming the events from metastore are unaware of that insert operation resulting in (meta)data sync issues. Need to investigate why the insert table event is not fired to metastore for a insert table operation on Iceberg table. h4. > Investigate missing insert table event for an insert operation on Iceberg > table > --- > > Key: IMPALA-12760 > URL: https://issues.apache.org/jira/browse/IMPALA-12760 > Project: IMPALA > Issue Type: Wish >Reporter: Sai Hemanth Gantasala >Assignee: Sai Hemanth Gantasala >Priority: Major > > Currently, we are not firing an insert table event to metastore when there is > an insert operation on an Iceberg table. As a result, other impala clusters > consuming the events from metastore are unaware of that insert operation > resulting in (meta)data sync issues. > Need to investigate why the insert table event is not fired to metastore for > an insert table operation on Iceberg table. > h4. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12757) TSAN flags lock-order-inversion during internal-server-test
[
https://issues.apache.org/jira/browse/IMPALA-12757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Smith updated IMPALA-12757:
---
Description:
internal-server-test has a tight loop starting queries and fetching /queries.
That's led to identifying several latent threading issues. Once IMPALA-12747 is
addressed, TSAN identifies a new error:
{code}
$ run-jvm-binary.sh ./be/build/debug/service/internal-server-test
I20240125 10:44:44.971467 152733 openssl_util.cc:110] FIPS mode is disabled.
Picked up JAVA_TOOL_OPTIONS: -Dsun.java.command=internal-server-test
24/01/25 10:44:45 WARN fs.FileSystem: Cannot load filesystem:
java.util.ServiceConfigurationError: org.apache.hadoop.fs.FileSystem: Provider
org.apache.hadoop.hive.ql.io.NullScanFileSystem not found
24/01/25 10:44:45 WARN fs.FileSystem: Cannot load filesystem:
java.util.ServiceConfigurationError: org.apache.hadoop.fs.FileSystem: Provider
org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem not found
24/01/25 10:44:45 INFO util.JvmPauseMonitor: Starting JVM pause monitor
[==] Running 10 tests from 1 test case.
[--] Global test environment set-up.
[--] 10 tests from InternalServerTest
[ RUN ] InternalServerTest.QueryTimeout
[ OK ] InternalServerTest.QueryTimeout (10236 ms)
[ RUN ] InternalServerTest.InvalidQueryOption
[ OK ] InternalServerTest.InvalidQueryOption (76 ms)
[ RUN ] InternalServerTest.MultipleQueriesMultipleSessions
/home/michael/Impala/be/src/service/internal-server-test.cc:289: Failure
Value of: status_.ok()
Actual: false
Expected: true
Error: Failed due to unreachable impalad(s): michaelsmith-22742:27000
[ FAILED ] InternalServerTest.MultipleQueriesMultipleSessions (17225 ms)
[ RUN ] InternalServerTest.RetryFailedQuery
[ OK ] InternalServerTest.RetryFailedQuery (1206 ms)
[ RUN ] InternalServerTest.MultipleQueriesOneSession
^C==
WARNING: ThreadSanitizer: thread leak (pid=152733)
Thread T1086 (tid=154103, finished) created by main thread at:
#0 pthread_create (internal-server-test+0x203c383)
#1 boost::thread::start_thread_noexcept()
(internal-server-test+0x3bcd3fd)
#2 boost::thread::thread, std::allocator > const&,
std::__cxx11::basic_string, std::allocator >
const&, boost::function, impala::ThreadDebugInfo const*,
impala::Promise*),
std::__cxx11::basic_string, std::allocator
>, std::__cxx11::basic_string,
std::allocator >, boost::function, impala::ThreadDebugInfo*,
impala::Promise*>(void
(*)(std::__cxx11::basic_string,
std::allocator > const&, std::__cxx11::basic_string, std::allocator > const&, boost::function, impala::ThreadDebugInfo const*, impala::Promise*), std::__cxx11::basic_string, std::allocator >,
std::__cxx11::basic_string, std::allocator
>, boost::function, impala::ThreadDebugInfo*, impala::Promise*)
/home/michael/Impala/toolchain/toolchain-packages-gcc10.4.0/boost-1.74.0-p1/include/boost/thread/detail/thread.hpp:424:13
(internal-server-test+0x2d84514)
#3 impala::Thread::StartThread(std::__cxx11::basic_string, std::allocator > const&,
std::__cxx11::basic_string, std::allocator >
const&, boost::function const&, std::unique_ptr >*, bool)
/home/michael/Impala/be/src/util/thread.cc:317:13
(internal-server-test+0x2d8091c)
#4 impala::Status impala::Thread::Create(std::__cxx11::basic_string, std::allocator > const&,
std::__cxx11::basic_string, std::allocator >
const&, void (impala::ClientRequestState::* const&)(),
impala::ClientRequestState* const&, std::unique_ptr >*, bool)
/home/michael/Impala/be/src/util/thread.h:81:12 (internal-server-test+0x2b382f7)
#5 impala::ClientRequestState::WaitAsync()
/home/michael/Impala/be/src/service/client-request-state.cc:1126:10
(internal-server-test+0x2b2f9f4)
#6 impala::ImpalaServer::WaitForResults(impala::TUniqueId&)
/home/michael/Impala/be/src/service/internal-server.cc:156:3
(internal-server-test+0x2ad148e)
#7 non-virtual thunk to
impala::ImpalaServer::WaitForResults(impala::TUniqueId&)
/home/michael/Impala/be/src/service/internal-server.cc
(internal-server-test+0x2ad15fd)
#8
impala::internalservertest::InternalServerTest_MultipleQueriesOneSession_Test::TestBody()
/home/michael/Impala/be/src/service/internal-server-test.cc:380:3
(internal-server-test+0x209fb9c)
#9 void
testing::internal::HandleExceptionsInMethodIfSupported(testing::Test*, void (testing::Test::*)(), char const*)
(internal-server-test+0x5212c1c)
#10 (libc.so.6+0x29d8f)
SUMMARY: ThreadSanitizer: thread leak
(/home/michael/Impala/be/build/debug/service/internal-server-test+0x203c383) in
__interceptor_pthread_create
==
michael@michaelsmith-22742 ~/Impala ❯❯❯ TSAN_OPTIONS=second_deadlock_stack=1
run-jvm-binary.sh ./be/build/debug/service/internal-server-test
michael@michaelsmith-22742 ~/Impala ❯❯❯ export
TSAN_OPTIONS=second_deadlock_stack=1
[jira] [Created] (IMPALA-12761) impala-shell: --show_profiles should return the query profile for failed queries also
Abhishek Rawat created IMPALA-12761: --- Summary: impala-shell: --show_profiles should return the query profile for failed queries also Key: IMPALA-12761 URL: https://issues.apache.org/jira/browse/IMPALA-12761 Project: IMPALA Issue Type: Improvement Components: Clients Reporter: Abhishek Rawat The `–show_profiles` option returns query profile at the end of a query execution, but only if the query was successful. It will be useful to also return query profile for the failed query for debugging. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-12757) TSAN flags lock-order-inversion during internal-server-test
[
https://issues.apache.org/jira/browse/IMPALA-12757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17811064#comment-17811064
]
Michael Smith commented on IMPALA-12757:
https://github.com/google/sanitizers/issues/814 might also be related. I
haven't worked through the circumstances enough to tell yet.
> TSAN flags lock-order-inversion during internal-server-test
> ---
>
> Key: IMPALA-12757
> URL: https://issues.apache.org/jira/browse/IMPALA-12757
> Project: IMPALA
> Issue Type: Bug
>Affects Versions: Impala 4.4.0
>Reporter: Michael Smith
>Priority: Major
>
> internal-server-test has a tight loop starting queries and fetching /queries.
> That's led to identifying several latent threading issues. Once IMPALA-12747
> is addressed, TSAN identifies a new error:
> {code}
> $ run-jvm-binary.sh ./be/build/debug/service/internal-server-test
> I20240125 10:44:44.971467 152733 openssl_util.cc:110] FIPS mode is disabled.
> Picked up JAVA_TOOL_OPTIONS: -Dsun.java.command=internal-server-test
> 24/01/25 10:44:45 WARN fs.FileSystem: Cannot load filesystem:
> java.util.ServiceConfigurationError: org.apache.hadoop.fs.FileSystem:
> Provider org.apache.hadoop.hive.ql.io.NullScanFileSystem not found
> 24/01/25 10:44:45 WARN fs.FileSystem: Cannot load filesystem:
> java.util.ServiceConfigurationError: org.apache.hadoop.fs.FileSystem:
> Provider org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem not found
> 24/01/25 10:44:45 INFO util.JvmPauseMonitor: Starting JVM pause monitor
> [==] Running 10 tests from 1 test case.
> [--] Global test environment set-up.
> [--] 10 tests from InternalServerTest
> [ RUN ] InternalServerTest.QueryTimeout
> [ OK ] InternalServerTest.QueryTimeout (10236 ms)
> [ RUN ] InternalServerTest.InvalidQueryOption
> [ OK ] InternalServerTest.InvalidQueryOption (76 ms)
> [ RUN ] InternalServerTest.MultipleQueriesMultipleSessions
> /home/michael/Impala/be/src/service/internal-server-test.cc:289: Failure
> Value of: status_.ok()
> Actual: false
> Expected: true
> Error: Failed due to unreachable impalad(s): michaelsmith-22742:27000
> [ FAILED ] InternalServerTest.MultipleQueriesMultipleSessions (17225 ms)
> [ RUN ] InternalServerTest.RetryFailedQuery
> [ OK ] InternalServerTest.RetryFailedQuery (1206 ms)
> [ RUN ] InternalServerTest.MultipleQueriesOneSession
> ^C==
> WARNING: ThreadSanitizer: thread leak (pid=152733)
> Thread T1086 (tid=154103, finished) created by main thread at:
> #0 pthread_create (internal-server-test+0x203c383)
> #1 boost::thread::start_thread_noexcept()
> (internal-server-test+0x3bcd3fd)
> #2 boost::thread::thread std::char_traits, std::allocator > const&,
> std::__cxx11::basic_string, std::allocator
> > const&, boost::function, impala::ThreadDebugInfo const*,
> impala::Promise*),
> std::__cxx11::basic_string, std::allocator
> >, std::__cxx11::basic_string,
> std::allocator >, boost::function, impala::ThreadDebugInfo*,
> impala::Promise*>(void
> (*)(std::__cxx11::basic_string,
> std::allocator > const&, std::__cxx11::basic_string std::char_traits, std::allocator > const&, boost::function ()>, impala::ThreadDebugInfo const*, impala::Promise (impala::PromiseMode)0>*), std::__cxx11::basic_string std::char_traits, std::allocator >,
> std::__cxx11::basic_string, std::allocator
> >, boost::function, impala::ThreadDebugInfo*, impala::Promise (impala::PromiseMode)0>*)
> /home/michael/Impala/toolchain/toolchain-packages-gcc10.4.0/boost-1.74.0-p1/include/boost/thread/detail/thread.hpp:424:13
> (internal-server-test+0x2d84514)
> #3 impala::Thread::StartThread(std::__cxx11::basic_string std::char_traits, std::allocator > const&,
> std::__cxx11::basic_string, std::allocator
> > const&, boost::function const&, std::unique_ptr std::default_delete >*, bool)
> /home/michael/Impala/be/src/util/thread.cc:317:13
> (internal-server-test+0x2d8091c)
> #4 impala::Status impala::Thread::Create (impala::ClientRequestState::*)(),
> impala::ClientRequestState*>(std::__cxx11::basic_string std::char_traits, std::allocator > const&,
> std::__cxx11::basic_string, std::allocator
> > const&, void (impala::ClientRequestState::* const&)(),
> impala::ClientRequestState* const&, std::unique_ptr std::default_delete >*, bool)
> /home/michael/Impala/be/src/util/thread.h:81:12
> (internal-server-test+0x2b382f7)
> #5 impala::ClientRequestState::WaitAsync()
> /home/michael/Impala/be/src/service/client-request-state.cc:1126:10
> (internal-server-test+0x2b2f9f4)
> #6 impala::ImpalaServer::WaitForResults(impala::TUniqueId&)
> /home/michael/Impala/be/src/service/internal-server.cc:156:3
> (internal-server-test+0x2ad148e)
> #7 non-virtual thunk to
> impa
