[jira] [Commented] (IMPALA-11333) Error when compile
[ https://issues.apache.org/jira/browse/IMPALA-11333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17810251#comment-17810251 ] Quanlong Huang commented on IMPALA-11333: - [~MadBeeDo] There are some real errors before this. Could you scroll up and share them? > Error when compile > -- > > Key: IMPALA-11333 > URL: https://issues.apache.org/jira/browse/IMPALA-11333 > Project: IMPALA > Issue Type: Bug >Affects Versions: Impala 4.1.0 > Environment: cmake version: 3.23.2 > maven version: 3.5.4 > centos:7 > java:1.8.0_112 >Reporter: Pain Sun >Priority: Major > > Error reports when compile impala. > [ 83%] Built target Util > [ 83%] Linking CXX static library ../../build/release/runtime/libRuntime.a > [ 83%] Built target Runtime > make[1]: *** [CMakeFiles/notests_all_targets.dir/rule] Error 2 > make: *** [notests_all_targets] Error 2 > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-12221) Start Impalad java.lang.NoClassDefFoundError: javax/jdo/JDOException
[
https://issues.apache.org/jira/browse/IMPALA-12221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17810262#comment-17810262
]
Quanlong Huang commented on IMPALA-12221:
-
I think the issue comes from hive-site.xml.
In my dev env, when launching impalad, the logs show that hive-site.xml under
${IMPALA_HOME}/fe/src/test/resources/ is used.
{noformat}
I0122 12:44:46.118683 17717 HiveConf.java:293] Found configuration file
file:/home/quanlong/workspace/Impala/fe/src/test/resources/hive-site.xml
I0122 12:44:46.602887 17717 MetastoreConf.java:2035] Found configuration file
file:/home/quanlong/workspace/Impala/fe/src/test/resources/hive-site.xml
I0122 12:44:46.603626 17717 MetastoreConf.java:2028] Unable to find config file
hivemetastore-site.xml
I0122 12:44:46.603684 17717 MetastoreConf.java:2035] Found configuration file
null
I0122 12:44:46.604097 17717 MetastoreConf.java:2028] Unable to find config file
metastore-site.xml
I0122 12:44:46.604152 17717 MetastoreConf.java:2035] Found configuration file
null{noformat}
In your logs, it shows hive-site.xml in the toolchain is used:
/bianqi/impala-4.3.0/toolchain/cdp_components-40643771/apache-hive-3.1.3000.7.2.18.0-41-bin/conf/hive-site.xml
It might be the default configuration file that is missing some correct
configuration, e.g.
{code:xml}
javax.jdo.option.ConnectionDriverName
org.postgresql.Driver
javax.jdo.option.ConnectionPassword
password
javax.jdo.option.ConnectionURL
jdbc:postgresql://localhost:5432/HMS_home_quanlong_workspace_Impala_cdp
javax.jdo.option.ConnectionUserName
hiveuser
{code}
Did you customize hive-site.xml? If you want to use what the dev env
configures, you can run bin/create-test-configuration.sh to generate the xml
files under ${IMPALA_HOME}/fe/src/test/resources/
> Start Impalad java.lang.NoClassDefFoundError: javax/jdo/JDOException
>
>
> Key: IMPALA-12221
> URL: https://issues.apache.org/jira/browse/IMPALA-12221
> Project: IMPALA
> Issue Type: Bug
> Components: Clients
>Affects Versions: Impala 4.3.0
>Reporter: bianqi
>Priority: Blocker
> Fix For: Not Applicable
>
>
> When I execute/ Start-impala-cluster.py - s 1-- restart_ Impalad_ Only, the
> imported service did not start successfully; Observing the logs, it was found
> that there was a class conflict;
> {code:java}
> I0616 11:50:26.499475 32367 MetastoreConf.java:2025] Found configuration file
> file:/bianqi/impala-4.3.0/toolchain/cdp_components-40643771/apache-hive-3.1.3000.7.2.18.0-41-bin/conf/hive-site.xml
> I0616 11:50:26.499955 32367 MetastoreConf.java:2018] Unable to find config
> file hivemetastore-site.xml
> I0616 11:50:26.53 32367 MetastoreConf.java:2025] Found configuration file
> null
> I0616 11:50:26.500358 32367 MetastoreConf.java:2018] Unable to find config
> file metastore-site.xml
> I0616 11:50:26.500401 32367 MetastoreConf.java:2025] Found configuration file
> null
> I0616 11:50:26.571043 32367 QueryEventHookManager.java:94] QueryEventHook
> config:
> I0616 11:50:26.571125 32367 QueryEventHookManager.java:95] -
> query_event_hook_nthreads=1
> I0616 11:50:26.571159 32367 QueryEventHookManager.java:96] -
> query_event_hook_classes=
> I0616 11:50:26.634904 32367 HiveMetaStoreClient.java:272] HMS client
> filtering is enabled.
> W0616 11:50:26.698336 32367 MetaStoreClientPool.java:107] Failed to connect
> to Hive MetaStore. Retrying.
> Java exception follows:
> java.lang.RuntimeException: Unable to instantiate
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient
> at
> org.apache.hadoop.hive.metastore.utils.JavaUtils.newInstance(JavaUtils.java:86)
> at
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.(RetryingMetaStoreClient.java:98)
> at
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:151)
> at
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:122)
> at
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:115)
> at
> org.apache.impala.catalog.MetaStoreClientPool$MetaStoreClient.(MetaStoreClientPool.java:99)
> at
> org.apache.impala.catalog.MetaStoreClientPool$MetaStoreClient.(MetaStoreClientPool.java:78)
> at
> org.apache.impala.catalog.MetaStoreClientPool.initClients(MetaStoreClientPool.java:174)
> at
> org.apache.impala.catalog.MetaStoreClientPool.(MetaStoreClientPool.java:163)
> at
> org.apache.impala.catalog.MetaStoreClientPool.(MetaStoreClientPool.java:155)
> at org.apache.impala.service.Frontend.(Frontend.java:50
[jira] [Commented] (IMPALA-12222) impala_shell-install-test.tar.gz pip3 could not be installed
[ https://issues.apache.org/jira/browse/IMPALA-1?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17810265#comment-17810265 ] Quanlong Huang commented on IMPALA-1: - Could you share the commands to reproduce this? > impala_shell-install-test.tar.gz pip3 could not be installed > > > Key: IMPALA-1 > URL: https://issues.apache.org/jira/browse/IMPALA-1 > Project: IMPALA > Issue Type: Bug > Components: Distributed Exec >Affects Versions: Impala 4.2.0 >Reporter: bianqi >Priority: Major > > Pip3 prompts that the install-test version cannot be found。 > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-12750) Support multiple file system in one partition for Iceberg Table
ZhuMinghui created IMPALA-12750: --- Summary: Support multiple file system in one partition for Iceberg Table Key: IMPALA-12750 URL: https://issues.apache.org/jira/browse/IMPALA-12750 Project: IMPALA Issue Type: New Feature Reporter: ZhuMinghui Assignee: ZhuMinghui -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12750) Support multiple file system in one partition of the Iceberg Table
[ https://issues.apache.org/jira/browse/IMPALA-12750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZhuMinghui updated IMPALA-12750: Summary: Support multiple file system in one partition of the Iceberg Table (was: Support multiple file system in one partition for Iceberg Table) > Support multiple file system in one partition of the Iceberg Table > -- > > Key: IMPALA-12750 > URL: https://issues.apache.org/jira/browse/IMPALA-12750 > Project: IMPALA > Issue Type: New Feature >Reporter: ZhuMinghui >Assignee: ZhuMinghui >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-12751) Crash unnesting nested array coming from UNION ALL
Daniel Becker created IMPALA-12751:
--
Summary: Crash unnesting nested array coming from UNION ALL
Key: IMPALA-12751
URL: https://issues.apache.org/jira/browse/IMPALA-12751
Project: IMPALA
Issue Type: Bug
Components: Backend, Frontend
Reporter: Daniel Becker
Unnesting a 2D array coming from a UNION ALL crashes:
{code:java}
with v0 as (select arr_int_2d from collection_tbl
union all select arr_int_2d from collection_tbl)
select a.item middle from v0, v0.arr_int_2d a;{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12751) Crash unnesting nested array coming from UNION ALL
[
https://issues.apache.org/jira/browse/IMPALA-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Becker updated IMPALA-12751:
---
Epic Link: IMPALA-9494
> Crash unnesting nested array coming from UNION ALL
> --
>
> Key: IMPALA-12751
> URL: https://issues.apache.org/jira/browse/IMPALA-12751
> Project: IMPALA
> Issue Type: Bug
> Components: Backend, Frontend
>Reporter: Daniel Becker
>Priority: Major
>
> Unnesting a 2D array coming from a UNION ALL crashes:
> {code:java}
> with v0 as (select arr_int_2d from collection_tbl
> union all select arr_int_2d from collection_tbl)
> select a.item middle from v0, v0.arr_int_2d a;{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12695) Crash with UNION with complex types
[
https://issues.apache.org/jira/browse/IMPALA-12695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Becker updated IMPALA-12695:
---
Epic Link: IMPALA-9494
> Crash with UNION with complex types
> ---
>
> Key: IMPALA-12695
> URL: https://issues.apache.org/jira/browse/IMPALA-12695
> Project: IMPALA
> Issue Type: Bug
>Reporter: Daniel Becker
>Assignee: Daniel Becker
>Priority: Major
>
> The following query crashes:
> {code:java}
> with tmp as (select all_mix from collection_struct_mix
> union all select all_mix from collection_struct_mix)
> select value from tmp, tmp.all_mix am;{code}
> We should find out the cause and fix it.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-12752) Enable JS tests for webUI in jenkins
Surya Hebbar created IMPALA-12752: - Summary: Enable JS tests for webUI in jenkins Key: IMPALA-12752 URL: https://issues.apache.org/jira/browse/IMPALA-12752 Project: IMPALA Issue Type: Improvement Reporter: Surya Hebbar The JS tests have been integrated into Impala. But, it has not been integrated into jenkins builds. So, it is necessary to integrate with jenkins. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Work started] (IMPALA-12752) Enable JS tests for webUI in jenkins
[ https://issues.apache.org/jira/browse/IMPALA-12752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on IMPALA-12752 started by Surya Hebbar. - > Enable JS tests for webUI in jenkins > > > Key: IMPALA-12752 > URL: https://issues.apache.org/jira/browse/IMPALA-12752 > Project: IMPALA > Issue Type: Improvement >Reporter: Surya Hebbar >Assignee: Surya Hebbar >Priority: Major > > The JS tests have been integrated into Impala. But, it has not been > integrated into jenkins builds. So, it is necessary to integrate with jenkins. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-12753) Predicate missing on unnested collection from UNION
Daniel Becker created IMPALA-12753:
--
Summary: Predicate missing on unnested collection from UNION
Key: IMPALA-12753
URL: https://issues.apache.org/jira/browse/IMPALA-12753
Project: IMPALA
Issue Type: Bug
Components: Backend, Frontend
Reporter: Daniel Becker
When a collection value comes from a UNNION [ALL], and is later unnested, and
there is a predicate on the unnested values, the predicate is only applied to
one side of the union.
To reproduce, create the table in Hive and add a row:
{code:java}
create table tmp (arr ARRAY) stored as parquet;
insert into tmp values (array(1, 2, 3, 4, 5));{code}
Then from Impala:
{code:java}
with v as (select arr from tmp
union all select arr from tmp)
select item from v, v.arr am where item=1;
+--+
| item |
+--+
| 1 |
| 1 |
| 2 |
| 3 |
| 4 |
| 5 |
+--+{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12688) Support JSON profile imports in webUI
[ https://issues.apache.org/jira/browse/IMPALA-12688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Surya Hebbar updated IMPALA-12688: -- Summary: Support JSON profile imports in webUI (was: Support JSON profile imports for visualizing query timeline in webUI) > Support JSON profile imports in webUI > - > > Key: IMPALA-12688 > URL: https://issues.apache.org/jira/browse/IMPALA-12688 > Project: IMPALA > Issue Type: New Feature >Reporter: Surya Hebbar >Assignee: Surya Hebbar >Priority: Major > > It would be helpful for users to visualize the query timeline by selecting a > local JSON query profile. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12688) Support JSON profile imports in webUI
[ https://issues.apache.org/jira/browse/IMPALA-12688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Surya Hebbar updated IMPALA-12688: -- Attachment: imported_queries_button.png > Support JSON profile imports in webUI > - > > Key: IMPALA-12688 > URL: https://issues.apache.org/jira/browse/IMPALA-12688 > Project: IMPALA > Issue Type: New Feature >Reporter: Surya Hebbar >Assignee: Surya Hebbar >Priority: Major > Attachments: imported_queries_button.png > > > It would be helpful for users to visualize the query timeline by selecting a > local JSON query profile. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12688) Support JSON profile imports in webUI
[ https://issues.apache.org/jira/browse/IMPALA-12688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Surya Hebbar updated IMPALA-12688: -- Attachment: imported_queries_page.png > Support JSON profile imports in webUI > - > > Key: IMPALA-12688 > URL: https://issues.apache.org/jira/browse/IMPALA-12688 > Project: IMPALA > Issue Type: New Feature >Reporter: Surya Hebbar >Assignee: Surya Hebbar >Priority: Major > Attachments: imported_queries_button.png, imported_queries_page.png > > > It would be helpful for users to visualize the query timeline by selecting a > local JSON query profile. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-12688) Support JSON profile imports in webUI
[ https://issues.apache.org/jira/browse/IMPALA-12688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17810434#comment-17810434 ] Surya Hebbar commented on IMPALA-12688: --- The imported queries section is located below completed queries and above query locations. The section also displays number of imported queries. !imported_queries_page.png|width=933,height=456! > Support JSON profile imports in webUI > - > > Key: IMPALA-12688 > URL: https://issues.apache.org/jira/browse/IMPALA-12688 > Project: IMPALA > Issue Type: New Feature >Reporter: Surya Hebbar >Assignee: Surya Hebbar >Priority: Major > Attachments: imported_queries_button.png, imported_queries_page.png > > > It would be helpful for users to visualize the query timeline by selecting a > local JSON query profile. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12688) Support JSON profile imports in webUI
[ https://issues.apache.org/jira/browse/IMPALA-12688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Surya Hebbar updated IMPALA-12688: -- Attachment: imported_queries_list.png > Support JSON profile imports in webUI > - > > Key: IMPALA-12688 > URL: https://issues.apache.org/jira/browse/IMPALA-12688 > Project: IMPALA > Issue Type: New Feature >Reporter: Surya Hebbar >Assignee: Surya Hebbar >Priority: Major > Attachments: imported_queries_button.png, imported_queries_list.png, > imported_queries_page.png > > > It would be helpful for users to visualize the query timeline by selecting a > local JSON query profile. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Comment Edited] (IMPALA-12688) Support JSON profile imports in webUI
[ https://issues.apache.org/jira/browse/IMPALA-12688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17810435#comment-17810435 ] Surya Hebbar edited comment on IMPALA-12688 at 1/24/24 2:53 PM: The following columns are displayed for each imported query. Along with a delete button for each. - Query ID - User - Default Db - Query Type - Start Time - End Time - Duration - Bytes Read - Bytes Sent - Query State - No. of rows fetched - Resource Pool - Query Statement !imported_queries_list.png|width=973,height=250! was (Author: JIRAUSER299620): The following columns are displayed for each imported query. - Query ID - User - Default Db - Query Type - Start Time - End Time - Duration - Bytes Read - Bytes Sent - Query State - No. of rows fetched - Resource Pool - Query Statement !imported_queries_list.png|width=973,height=250! > Support JSON profile imports in webUI > - > > Key: IMPALA-12688 > URL: https://issues.apache.org/jira/browse/IMPALA-12688 > Project: IMPALA > Issue Type: New Feature >Reporter: Surya Hebbar >Assignee: Surya Hebbar >Priority: Major > Attachments: imported_queries_button.png, imported_queries_list.png, > imported_queries_page.png > > > It would be helpful for users to visualize the query timeline by selecting a > local JSON query profile. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-12688) Support JSON profile imports in webUI
[ https://issues.apache.org/jira/browse/IMPALA-12688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17810435#comment-17810435 ] Surya Hebbar commented on IMPALA-12688: --- The following columns are displayed for each imported query. - Query ID - User - Default Db - Query Type - Start Time - End Time - Duration - Bytes Read - Bytes Sent - Query State - No. of rows fetched - Resource Pool - Query Statement !imported_queries_list.png|width=973,height=250! > Support JSON profile imports in webUI > - > > Key: IMPALA-12688 > URL: https://issues.apache.org/jira/browse/IMPALA-12688 > Project: IMPALA > Issue Type: New Feature >Reporter: Surya Hebbar >Assignee: Surya Hebbar >Priority: Major > Attachments: imported_queries_button.png, imported_queries_list.png, > imported_queries_page.png > > > It would be helpful for users to visualize the query timeline by selecting a > local JSON query profile. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Comment Edited] (IMPALA-12688) Support JSON profile imports in webUI
[ https://issues.apache.org/jira/browse/IMPALA-12688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17810435#comment-17810435 ] Surya Hebbar edited comment on IMPALA-12688 at 1/24/24 2:55 PM: The following columns are displayed for each imported query. Along with a delete button for each imported query. - Query ID - User - Default Db - Query Type - Start Time - End Time - Duration - Bytes Read - Bytes Sent - Query State - No. of rows fetched - Resource Pool - Query Statement !imported_queries_list.png|width=973,height=250! was (Author: JIRAUSER299620): The following columns are displayed for each imported query. Along with a delete button for each. - Query ID - User - Default Db - Query Type - Start Time - End Time - Duration - Bytes Read - Bytes Sent - Query State - No. of rows fetched - Resource Pool - Query Statement !imported_queries_list.png|width=973,height=250! > Support JSON profile imports in webUI > - > > Key: IMPALA-12688 > URL: https://issues.apache.org/jira/browse/IMPALA-12688 > Project: IMPALA > Issue Type: New Feature >Reporter: Surya Hebbar >Assignee: Surya Hebbar >Priority: Major > Attachments: imported_queries_button.png, imported_queries_list.png, > imported_queries_page.png > > > It would be helpful for users to visualize the query timeline by selecting a > local JSON query profile. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12688) Support JSON profile imports in webUI
[ https://issues.apache.org/jira/browse/IMPALA-12688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Surya Hebbar updated IMPALA-12688: -- Attachment: imported_query_text_plan.png imported_query_timeline.png imported_query_statement.png > Support JSON profile imports in webUI > - > > Key: IMPALA-12688 > URL: https://issues.apache.org/jira/browse/IMPALA-12688 > Project: IMPALA > Issue Type: New Feature >Reporter: Surya Hebbar >Assignee: Surya Hebbar >Priority: Major > Attachments: imported_queries_button.png, imported_queries_list.png, > imported_queries_page.png, imported_query_statement.png, > imported_query_text_plan.png, imported_query_timeline.png > > > It would be helpful for users to visualize the query timeline by selecting a > local JSON query profile. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12688) Support JSON profile imports in webUI
[ https://issues.apache.org/jira/browse/IMPALA-12688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Surya Hebbar updated IMPALA-12688: -- Attachment: imported_query_timeline-1.png > Support JSON profile imports in webUI > - > > Key: IMPALA-12688 > URL: https://issues.apache.org/jira/browse/IMPALA-12688 > Project: IMPALA > Issue Type: New Feature >Reporter: Surya Hebbar >Assignee: Surya Hebbar >Priority: Major > Attachments: imported_queries_button.png, imported_queries_list.png, > imported_queries_page.png, imported_query_statement.png, > imported_query_text_plan.png, imported_query_timeline-1.png, > imported_query_timeline.png > > > It would be helpful for users to visualize the query timeline by selecting a > local JSON query profile. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12688) Support JSON profile imports in webUI
[ https://issues.apache.org/jira/browse/IMPALA-12688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Surya Hebbar updated IMPALA-12688: -- Attachment: imported_query_statement-1.png > Support JSON profile imports in webUI > - > > Key: IMPALA-12688 > URL: https://issues.apache.org/jira/browse/IMPALA-12688 > Project: IMPALA > Issue Type: New Feature >Reporter: Surya Hebbar >Assignee: Surya Hebbar >Priority: Major > Attachments: imported_queries_button.png, imported_queries_list.png, > imported_queries_page.png, imported_query_statement-1.png, > imported_query_statement.png, imported_query_text_plan.png, > imported_query_timeline-1.png, imported_query_timeline.png > > > It would be helpful for users to visualize the query timeline by selecting a > local JSON query profile. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12688) Support JSON profile imports in webUI
[ https://issues.apache.org/jira/browse/IMPALA-12688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Surya Hebbar updated IMPALA-12688: -- Attachment: (was: imported_query_timeline-1.png) > Support JSON profile imports in webUI > - > > Key: IMPALA-12688 > URL: https://issues.apache.org/jira/browse/IMPALA-12688 > Project: IMPALA > Issue Type: New Feature >Reporter: Surya Hebbar >Assignee: Surya Hebbar >Priority: Major > Attachments: imported_queries_button.png, imported_queries_list.png, > imported_queries_page.png, imported_query_statement.png, > imported_query_text_plan.png, imported_query_timeline.png > > > It would be helpful for users to visualize the query timeline by selecting a > local JSON query profile. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-12688) Support JSON profile imports in webUI
[ https://issues.apache.org/jira/browse/IMPALA-12688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17810440#comment-17810440 ] Surya Hebbar commented on IMPALA-12688: --- On opening any of the imported queries, its query timeline will be displayed. Currently the following tabs are supported for imported queries. - Query Statement - Query Timeline - Query Text Plan !imported_query_text_plan.png|width=970,height=606! !imported_query_timeline.png|width=973,height=608! For imported queries, syntax highlighting on query statement tab has also been supported. !imported_query_statement.png|width=1000,height=625! > Support JSON profile imports in webUI > - > > Key: IMPALA-12688 > URL: https://issues.apache.org/jira/browse/IMPALA-12688 > Project: IMPALA > Issue Type: New Feature >Reporter: Surya Hebbar >Assignee: Surya Hebbar >Priority: Major > Attachments: imported_queries_button.png, imported_queries_list.png, > imported_queries_page.png, imported_query_statement.png, > imported_query_text_plan.png, imported_query_timeline.png > > > It would be helpful for users to visualize the query timeline by selecting a > local JSON query profile. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12688) Support JSON profile imports in webUI
[ https://issues.apache.org/jira/browse/IMPALA-12688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Surya Hebbar updated IMPALA-12688: -- Attachment: (was: imported_query_statement-1.png) > Support JSON profile imports in webUI > - > > Key: IMPALA-12688 > URL: https://issues.apache.org/jira/browse/IMPALA-12688 > Project: IMPALA > Issue Type: New Feature >Reporter: Surya Hebbar >Assignee: Surya Hebbar >Priority: Major > Attachments: imported_queries_button.png, imported_queries_list.png, > imported_queries_page.png, imported_query_statement.png, > imported_query_text_plan.png, imported_query_timeline.png > > > It would be helpful for users to visualize the query timeline by selecting a > local JSON query profile. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Comment Edited] (IMPALA-12688) Support JSON profile imports in webUI
[ https://issues.apache.org/jira/browse/IMPALA-12688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17810434#comment-17810434 ] Surya Hebbar edited comment on IMPALA-12688 at 1/24/24 3:03 PM: The imported queries section is located below completed queries and above query locations. The section also displays number of imported queries. !imported_queries_page.png|width=941,height=460! was (Author: JIRAUSER299620): The imported queries section is located below completed queries and above query locations. The section also displays number of imported queries. !imported_queries_page.png|width=933,height=456! > Support JSON profile imports in webUI > - > > Key: IMPALA-12688 > URL: https://issues.apache.org/jira/browse/IMPALA-12688 > Project: IMPALA > Issue Type: New Feature >Reporter: Surya Hebbar >Assignee: Surya Hebbar >Priority: Major > Attachments: imported_queries_button.png, imported_queries_list.png, > imported_queries_page.png, imported_query_statement.png, > imported_query_text_plan.png, imported_query_timeline.png > > > It would be helpful for users to visualize the query timeline by selecting a > local JSON query profile. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Comment Edited] (IMPALA-12688) Support JSON profile imports in webUI
[ https://issues.apache.org/jira/browse/IMPALA-12688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17810435#comment-17810435 ] Surya Hebbar edited comment on IMPALA-12688 at 1/24/24 3:04 PM: The following columns are displayed for each imported query. Along with a delete button for each imported query. - Query ID - User - Default Db - Query Type - Start Time - End Time - Duration - Bytes Read - Bytes Sent - Query State - No. of rows fetched - Resource Pool - Query Statement !imported_queries_list.png|width=942,height=242! was (Author: JIRAUSER299620): The following columns are displayed for each imported query. Along with a delete button for each imported query. - Query ID - User - Default Db - Query Type - Start Time - End Time - Duration - Bytes Read - Bytes Sent - Query State - No. of rows fetched - Resource Pool - Query Statement !imported_queries_list.png|width=973,height=250! > Support JSON profile imports in webUI > - > > Key: IMPALA-12688 > URL: https://issues.apache.org/jira/browse/IMPALA-12688 > Project: IMPALA > Issue Type: New Feature >Reporter: Surya Hebbar >Assignee: Surya Hebbar >Priority: Major > Attachments: imported_queries_button.png, imported_queries_list.png, > imported_queries_page.png, imported_query_statement.png, > imported_query_text_plan.png, imported_query_timeline.png > > > It would be helpful for users to visualize the query timeline by selecting a > local JSON query profile. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Comment Edited] (IMPALA-12688) Support JSON profile imports in webUI
[ https://issues.apache.org/jira/browse/IMPALA-12688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17810440#comment-17810440 ] Surya Hebbar edited comment on IMPALA-12688 at 1/24/24 3:05 PM: On opening any of the imported queries, its query timeline will be displayed. Currently the following tabs are supported for imported queries. - Query Statement - Query Timeline - Query Text Plan !imported_query_text_plan.png|width=940,height=587! !imported_query_timeline.png|width=941,height=588! For imported queries, syntax highlighting on query statement tab has also been supported. !imported_query_statement.png|width=940,height=587! was (Author: JIRAUSER299620): On opening any of the imported queries, its query timeline will be displayed. Currently the following tabs are supported for imported queries. - Query Statement - Query Timeline - Query Text Plan !imported_query_text_plan.png|width=970,height=606! !imported_query_timeline.png|width=973,height=608! For imported queries, syntax highlighting on query statement tab has also been supported. !imported_query_statement.png|width=1000,height=625! > Support JSON profile imports in webUI > - > > Key: IMPALA-12688 > URL: https://issues.apache.org/jira/browse/IMPALA-12688 > Project: IMPALA > Issue Type: New Feature >Reporter: Surya Hebbar >Assignee: Surya Hebbar >Priority: Major > Attachments: imported_queries_button.png, imported_queries_list.png, > imported_queries_page.png, imported_query_statement.png, > imported_query_text_plan.png, imported_query_timeline.png > > > It would be helpful for users to visualize the query timeline by selecting a > local JSON query profile. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Assigned] (IMPALA-12531) Add support for Merge statements for Iceberg tables
[ https://issues.apache.org/jira/browse/IMPALA-12531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Rozsa reassigned IMPALA-12531: Assignee: Peter Rozsa > Add support for Merge statements for Iceberg tables > --- > > Key: IMPALA-12531 > URL: https://issues.apache.org/jira/browse/IMPALA-12531 > Project: IMPALA > Issue Type: Improvement > Components: Backend, Frontend >Reporter: Manish Maheshwari >Assignee: Peter Rozsa >Priority: Major > Labels: impala-iceberg > > Support Merge command for Iceberg table to support customer use cases. > Impala does support upset for > [Kudu|https://impala.apache.org/docs/build/html/topics/impala_upsert.html] > tables. We can take a look at that or look at what merge syntax is supported > by Hive/Spark etc -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-12752) Enable JS tests for webUI in jenkins
[
https://issues.apache.org/jira/browse/IMPALA-12752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17810501#comment-17810501
]
Riza Suminto commented on IMPALA-12752:
---
Underneath, it will run a quick test using nodejs,
{code:java}
$ ./tests/run-js-tests.sh
NodeJS v16.20.2 installation foundup to date, audited 347 packages in 806ms34
packages are looking for funding
run `npm fund` for detailsfound 0 vulnerabilities> Impala WebUI JS
Tests@1.0.0 test
> JEST_JUNIT_OUTPUT_DIR="${IMPALA_JS_TEST_LOGS_DIR}" "${IMPALA_NODEJS}"
> --experimental-vm-modules node_modules/jest/bin/jest.js(node:756693)
> ExperimentalWarning: VM Modules is an experimental feature. This feature
> could change at any time
(Use `node --trace-warnings ...` to show where the warning was created)
PASS query_timeline/host_utilization_diagram.test.js
PASS query_timeline/chart_commons.test.js
PASS query_timeline/fragment_diagram.test.js
PASS query_timeline/fragment_metrics_diagram.test.jsTest Suites: 4 passed, 4
total
Tests: 21 passed, 21 total
Snapshots: 0 total
Time: 0.52 s, estimated 1 s
Ran all test suites. {code}
[~laszlo.g...@gmail.com] [~joemcdonnell] Can we enable this in
jenkins.impala.io? Maybe enable in ubuntu-20.04-from-scratch that runs BE test.
> Enable JS tests for webUI in jenkins
>
>
> Key: IMPALA-12752
> URL: https://issues.apache.org/jira/browse/IMPALA-12752
> Project: IMPALA
> Issue Type: Improvement
>Reporter: Surya Hebbar
>Assignee: Surya Hebbar
>Priority: Major
>
> The JS tests have been integrated into Impala. But, it has not been
> integrated into jenkins builds. So, it is necessary to integrate with jenkins.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Comment Edited] (IMPALA-12752) Enable JS tests for webUI in jenkins
[
https://issues.apache.org/jira/browse/IMPALA-12752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17810501#comment-17810501
]
Riza Suminto edited comment on IMPALA-12752 at 1/24/24 5:06 PM:
Underneath, it will run a quick test using nodejs,
{code:java}
$ ./tests/run-js-tests.sh
NodeJS v16.20.2 installation foundup to date, audited 347 packages in 806ms34
packages are looking for funding
run `npm fund` for detailsfound 0 vulnerabilities> Impala WebUI JS
Tests@1.0.0 test
> JEST_JUNIT_OUTPUT_DIR="${IMPALA_JS_TEST_LOGS_DIR}" "${IMPALA_NODEJS}"
> --experimental-vm-modules node_modules/jest/bin/jest.js(node:756693)
> ExperimentalWarning: VM Modules is an experimental feature. This feature
> could change at any time
(Use `node --trace-warnings ...` to show where the warning was created)
PASS query_timeline/host_utilization_diagram.test.js
PASS query_timeline/chart_commons.test.js
PASS query_timeline/fragment_diagram.test.js
PASS query_timeline/fragment_metrics_diagram.test.jsTest Suites: 4 passed, 4
total
Tests: 21 passed, 21 total
Snapshots: 0 total
Time: 0.52 s, estimated 1 s
Ran all test suites. {code}
[~laszlog] [~joemcdonnell] Can we enable this in jenkins.impala.io? Maybe
enable in ubuntu-20.04-from-scratch that runs BE test. @Gaal
was (Author: rizaon):
Underneath, it will run a quick test using nodejs,
{code:java}
$ ./tests/run-js-tests.sh
NodeJS v16.20.2 installation foundup to date, audited 347 packages in 806ms34
packages are looking for funding
run `npm fund` for detailsfound 0 vulnerabilities> Impala WebUI JS
Tests@1.0.0 test
> JEST_JUNIT_OUTPUT_DIR="${IMPALA_JS_TEST_LOGS_DIR}" "${IMPALA_NODEJS}"
> --experimental-vm-modules node_modules/jest/bin/jest.js(node:756693)
> ExperimentalWarning: VM Modules is an experimental feature. This feature
> could change at any time
(Use `node --trace-warnings ...` to show where the warning was created)
PASS query_timeline/host_utilization_diagram.test.js
PASS query_timeline/chart_commons.test.js
PASS query_timeline/fragment_diagram.test.js
PASS query_timeline/fragment_metrics_diagram.test.jsTest Suites: 4 passed, 4
total
Tests: 21 passed, 21 total
Snapshots: 0 total
Time: 0.52 s, estimated 1 s
Ran all test suites. {code}
[~laszlo.g...@gmail.com] [~joemcdonnell] Can we enable this in
jenkins.impala.io? Maybe enable in ubuntu-20.04-from-scratch that runs BE test.
> Enable JS tests for webUI in jenkins
>
>
> Key: IMPALA-12752
> URL: https://issues.apache.org/jira/browse/IMPALA-12752
> Project: IMPALA
> Issue Type: Improvement
>Reporter: Surya Hebbar
>Assignee: Surya Hebbar
>Priority: Major
>
> The JS tests have been integrated into Impala. But, it has not been
> integrated into jenkins builds. So, it is necessary to integrate with jenkins.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Comment Edited] (IMPALA-12752) Enable JS tests for webUI in jenkins
[
https://issues.apache.org/jira/browse/IMPALA-12752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17810501#comment-17810501
]
Riza Suminto edited comment on IMPALA-12752 at 1/24/24 5:13 PM:
Underneath, it will run a quick test using nodejs,
{code:java}
$ ./tests/run-js-tests.sh
NodeJS v16.20.2 installation foundup to date, audited 347 packages in 806ms34
packages are looking for funding
run `npm fund` for detailsfound 0 vulnerabilities> Impala WebUI JS
Tests@1.0.0 test
> JEST_JUNIT_OUTPUT_DIR="${IMPALA_JS_TEST_LOGS_DIR}" "${IMPALA_NODEJS}"
> --experimental-vm-modules node_modules/jest/bin/jest.js(node:756693)
> ExperimentalWarning: VM Modules is an experimental feature. This feature
> could change at any time
(Use `node --trace-warnings ...` to show where the warning was created)
PASS query_timeline/host_utilization_diagram.test.js
PASS query_timeline/chart_commons.test.js
PASS query_timeline/fragment_diagram.test.js
PASS query_timeline/fragment_metrics_diagram.test.jsTest Suites: 4 passed, 4
total
Tests: 21 passed, 21 total
Snapshots: 0 total
Time: 0.52 s, estimated 1 s
Ran all test suites. {code}
[~laszlog] [~joemcdonnell] Can we enable this in jenkins.impala.io? Maybe
enable in ubuntu-20.04-from-scratch that runs BE test.
was (Author: rizaon):
Underneath, it will run a quick test using nodejs,
{code:java}
$ ./tests/run-js-tests.sh
NodeJS v16.20.2 installation foundup to date, audited 347 packages in 806ms34
packages are looking for funding
run `npm fund` for detailsfound 0 vulnerabilities> Impala WebUI JS
Tests@1.0.0 test
> JEST_JUNIT_OUTPUT_DIR="${IMPALA_JS_TEST_LOGS_DIR}" "${IMPALA_NODEJS}"
> --experimental-vm-modules node_modules/jest/bin/jest.js(node:756693)
> ExperimentalWarning: VM Modules is an experimental feature. This feature
> could change at any time
(Use `node --trace-warnings ...` to show where the warning was created)
PASS query_timeline/host_utilization_diagram.test.js
PASS query_timeline/chart_commons.test.js
PASS query_timeline/fragment_diagram.test.js
PASS query_timeline/fragment_metrics_diagram.test.jsTest Suites: 4 passed, 4
total
Tests: 21 passed, 21 total
Snapshots: 0 total
Time: 0.52 s, estimated 1 s
Ran all test suites. {code}
[~laszlog] [~joemcdonnell] Can we enable this in jenkins.impala.io? Maybe
enable in ubuntu-20.04-from-scratch that runs BE test. @Gaal
> Enable JS tests for webUI in jenkins
>
>
> Key: IMPALA-12752
> URL: https://issues.apache.org/jira/browse/IMPALA-12752
> Project: IMPALA
> Issue Type: Improvement
>Reporter: Surya Hebbar
>Assignee: Surya Hebbar
>Priority: Major
>
> The JS tests have been integrated into Impala. But, it has not been
> integrated into jenkins builds. So, it is necessary to integrate with jenkins.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12378) Auto Ship Data Source library in Impala
[ https://issues.apache.org/jira/browse/IMPALA-12378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manish Maheshwari updated IMPALA-12378: --- Summary: Auto Ship Data Source library in Impala (was: Auto Ship JDBC external data source) > Auto Ship Data Source library in Impala > --- > > Key: IMPALA-12378 > URL: https://issues.apache.org/jira/browse/IMPALA-12378 > Project: IMPALA > Issue Type: Sub-task > Components: Frontend, Infrastructure >Reporter: Wenzhe Zhou >Assignee: Wenzhe Zhou >Priority: Major > > The library of JDBC external data source should be auto shipped in Impala > binaries so that user don’t need to add the jar file manually. However jdbc > driver jars are provided by user. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-12699) Coordinator should retry GetPartialCatalogObject request and apply a recv timeout
[ https://issues.apache.org/jira/browse/IMPALA-12699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17810533#comment-17810533 ] Wenzhe Zhou commented on IMPALA-12699: -- There is only one client cache object for catalog service on each impalad, which is created with FLAGS_catalog_client_rpc_timeout_ms as recv and send timeout. https://github.com/apache/impala/blob/cdac777c51febc99500b8426c2b3aabc7e9addd7/be/src/runtime/exec-env.cc#L224-L226 The catalog client cache object is initialized by calling catalogd_client_cache_->InitMetrics(metrics_.get(), "catalog.server"); https://github.com/apache/impala/blob/cdac777c51febc99500b8426c2b3aabc7e9addd7/be/src/runtime/exec-env.cc#L430 If we want to apply a recv_timeout for GetPartialCatalogObject RPC only, then we have to create a separate catalogd client cache object. How do we set metrics for this additional client cache object? Currently the metrics names are fixed. https://github.com/apache/impala/blob/cdac777c51febc99500b8426c2b3aabc7e9addd7/be/src/runtime/client-cache.cc#L236-L249 Use a different key_prefix? > Coordinator should retry GetPartialCatalogObject request and apply a recv > timeout > - > > Key: IMPALA-12699 > URL: https://issues.apache.org/jira/browse/IMPALA-12699 > Project: IMPALA > Issue Type: Bug > Components: Catalog >Reporter: Quanlong Huang >Assignee: Quanlong Huang >Priority: Critical > > We have seen trivial GetPartialCatalogObject RPCs hanging in coordinator > side, e.g. IMPALA-11409. Due to the piggyback mechanism of fetching metadata > in local-catalog mode (see IMPALA-7534 or comments in > CatalogdMetaProvider#loadWithCaching()), a hanging RPC on shared metadata > (e.g. db list or table list of a db) could block other queries. > We have also seen thrift RPCs hanging in IMPALA-3575. In fact, > GetPartialCatalogObject RPCs are read-only requests. They can be cleanly > retried. We should consider using a dedicated catalogd client cache for > GetPartialCatalogObject requests and set an appropriate timeout for the > socket. > The current catalogd client cache: > https://github.com/apache/impala/blob/cdac777c51febc99500b8426c2b3aabc7e9addd7/be/src/runtime/exec-env.cc#L224-L226 > The related flags: > https://github.com/apache/impala/blob/cdac777c51febc99500b8426c2b3aabc7e9addd7/be/src/runtime/exec-env.cc#L161-L167 > CC [~wzhou] -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Comment Edited] (IMPALA-12699) Coordinator should retry GetPartialCatalogObject request and apply a recv timeout
[ https://issues.apache.org/jira/browse/IMPALA-12699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17810533#comment-17810533 ] Wenzhe Zhou edited comment on IMPALA-12699 at 1/24/24 7:08 PM: --- There is only one client cache object for catalog service on each impalad, which is created with FLAGS_catalog_client_rpc_timeout_ms as recv and send timeout. https://github.com/apache/impala/blob/cdac777c51febc99500b8426c2b3aabc7e9addd7/be/src/runtime/exec-env.cc#L224-L226 The catalog client cache object is initialized by calling catalogd_client_cache_->InitMetrics(metrics_.get(), "catalog.server"); https://github.com/apache/impala/blob/cdac777c51febc99500b8426c2b3aabc7e9addd7/be/src/runtime/exec-env.cc#L430 If we want to apply a recv_timeout for GetPartialCatalogObject RPC only, then we have to create a separate catalogd client cache object, e.g. open a new TSocket for this Thrift RPC. How do we set metrics for this additional client cache object? Currently the metrics names are fixed. https://github.com/apache/impala/blob/cdac777c51febc99500b8426c2b3aabc7e9addd7/be/src/runtime/client-cache.cc#L236-L249 Use a different key_prefix? was (Author: wzhou): There is only one client cache object for catalog service on each impalad, which is created with FLAGS_catalog_client_rpc_timeout_ms as recv and send timeout. https://github.com/apache/impala/blob/cdac777c51febc99500b8426c2b3aabc7e9addd7/be/src/runtime/exec-env.cc#L224-L226 The catalog client cache object is initialized by calling catalogd_client_cache_->InitMetrics(metrics_.get(), "catalog.server"); https://github.com/apache/impala/blob/cdac777c51febc99500b8426c2b3aabc7e9addd7/be/src/runtime/exec-env.cc#L430 If we want to apply a recv_timeout for GetPartialCatalogObject RPC only, then we have to create a separate catalogd client cache object. How do we set metrics for this additional client cache object? Currently the metrics names are fixed. https://github.com/apache/impala/blob/cdac777c51febc99500b8426c2b3aabc7e9addd7/be/src/runtime/client-cache.cc#L236-L249 Use a different key_prefix? > Coordinator should retry GetPartialCatalogObject request and apply a recv > timeout > - > > Key: IMPALA-12699 > URL: https://issues.apache.org/jira/browse/IMPALA-12699 > Project: IMPALA > Issue Type: Bug > Components: Catalog >Reporter: Quanlong Huang >Assignee: Quanlong Huang >Priority: Critical > > We have seen trivial GetPartialCatalogObject RPCs hanging in coordinator > side, e.g. IMPALA-11409. Due to the piggyback mechanism of fetching metadata > in local-catalog mode (see IMPALA-7534 or comments in > CatalogdMetaProvider#loadWithCaching()), a hanging RPC on shared metadata > (e.g. db list or table list of a db) could block other queries. > We have also seen thrift RPCs hanging in IMPALA-3575. In fact, > GetPartialCatalogObject RPCs are read-only requests. They can be cleanly > retried. We should consider using a dedicated catalogd client cache for > GetPartialCatalogObject requests and set an appropriate timeout for the > socket. > The current catalogd client cache: > https://github.com/apache/impala/blob/cdac777c51febc99500b8426c2b3aabc7e9addd7/be/src/runtime/exec-env.cc#L224-L226 > The related flags: > https://github.com/apache/impala/blob/cdac777c51febc99500b8426c2b3aabc7e9addd7/be/src/runtime/exec-env.cc#L161-L167 > CC [~wzhou] -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-12513) The metadata should be reset when the CatalogD become active
[ https://issues.apache.org/jira/browse/IMPALA-12513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenzhe Zhou resolved IMPALA-12513. -- Fix Version/s: Impala 4.4.0 Resolution: Fixed > The metadata should be reset when the CatalogD become active > > > Key: IMPALA-12513 > URL: https://issues.apache.org/jira/browse/IMPALA-12513 > Project: IMPALA > Issue Type: Bug >Reporter: Zhi Tang >Priority: Major > Fix For: Impala 4.4.0 > > > When performing an automatic failover of CatalogD, we should ensure that the > metadata for active Catalogd is up to date. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Assigned] (IMPALA-12513) The metadata should be reset when the CatalogD become active
[ https://issues.apache.org/jira/browse/IMPALA-12513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenzhe Zhou reassigned IMPALA-12513: Assignee: Zhi Tang > The metadata should be reset when the CatalogD become active > > > Key: IMPALA-12513 > URL: https://issues.apache.org/jira/browse/IMPALA-12513 > Project: IMPALA > Issue Type: Bug >Reporter: Zhi Tang >Assignee: Zhi Tang >Priority: Major > Fix For: Impala 4.4.0 > > > When performing an automatic failover of CatalogD, we should ensure that the > metadata for active Catalogd is up to date. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-12754) Update Impala document to cover external data source
Wenzhe Zhou created IMPALA-12754: Summary: Update Impala document to cover external data source Key: IMPALA-12754 URL: https://issues.apache.org/jira/browse/IMPALA-12754 Project: IMPALA Issue Type: Sub-task Components: Docs Reporter: Wenzhe Zhou Impala external data source is undocumented in upstream. We need to document the external data source APIs, SQL syntax to create jdbc data source, show data sources, create table for external jdbc data source, including the properties to be set for jdbc and DBCP (Database connection pool). -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12754) Update Impala document to cover external jdbc data source
[ https://issues.apache.org/jira/browse/IMPALA-12754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenzhe Zhou updated IMPALA-12754: - Summary: Update Impala document to cover external jdbc data source (was: Update Impala document to cover external data source) > Update Impala document to cover external jdbc data source > - > > Key: IMPALA-12754 > URL: https://issues.apache.org/jira/browse/IMPALA-12754 > Project: IMPALA > Issue Type: Sub-task > Components: Docs >Reporter: Wenzhe Zhou >Priority: Major > > Impala external data source is undocumented in upstream. We need to document > the external data source APIs, SQL syntax to create jdbc data source, show > data sources, create table for external jdbc data source, including the > properties to be set for jdbc and DBCP (Database connection pool). > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-12755) could not resolve artifact 'cdh-root:6.x-20220131.115617-21547795'
guojingfeng created IMPALA-12755:
Summary: could not resolve artifact
'cdh-root:6.x-20220131.115617-21547795'
Key: IMPALA-12755
URL: https://issues.apache.org/jira/browse/IMPALA-12755
Project: IMPALA
Issue Type: Bug
Affects Versions: Impala 3.4.1, Impala 3.4.0
Reporter: guojingfeng
Attachments: image-2024-01-25-10-53-04-706.png
We could not resolve 'cdh-root:6.x-20220131.115617-21547795' when building
impala-3.4.x versions.
{code:java}
[INFO] BUILD FAILURE
[ERROR] Failed to execute goal on project impala-minimal-hive-exec: Could not
resolve dependencies for project
org.apache.impala:impala-minimal-hive-exec:jar:0.1-SNAPSHOT: Failed to collect
dependencies at org.apache.hive:hive-exec:jar:2.1.1-cdh6.x-SNAPSHOT: Failed to
read artifact descriptor for
org.apache.hive:hive-exec:jar:2.1.1-cdh6.x-SNAPSHOT: Could not find artifact
com.cloudera.cdh:cdh-root:pom:6.x-20220131.115617-21547795 in
cdh.rcs.releases.repo
(https://repository.cloudera.com/artifactory/cdh-releases-rcs) -> [Help 1]{code}
There are only metadata in 'cdh.rcs.releases.repo' while the artifacts is
missing.
!image-2024-01-25-10-53-04-706.png!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-12755) could not resolve artifact 'cdh-root:6.x-20220131.115617-21547795'
[
https://issues.apache.org/jira/browse/IMPALA-12755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17810630#comment-17810630
]
guojingfeng commented on IMPALA-12755:
--
In offline practice, we disable 'cdh.rcs.releases.repo' and instead use
'impala.cdh.repo' to resolve the missing artifacts successfully.
> could not resolve artifact 'cdh-root:6.x-20220131.115617-21547795'
> --
>
> Key: IMPALA-12755
> URL: https://issues.apache.org/jira/browse/IMPALA-12755
> Project: IMPALA
> Issue Type: Bug
>Affects Versions: Impala 3.4.0, Impala 3.4.1
>Reporter: guojingfeng
>Priority: Major
> Attachments: image-2024-01-25-10-53-04-706.png
>
>
> We could not resolve 'cdh-root:6.x-20220131.115617-21547795' when building
> impala-3.4.x versions.
> {code:java}
> [INFO] BUILD FAILURE
> [ERROR] Failed to execute goal on project impala-minimal-hive-exec: Could not
> resolve dependencies for project
> org.apache.impala:impala-minimal-hive-exec:jar:0.1-SNAPSHOT: Failed to
> collect dependencies at org.apache.hive:hive-exec:jar:2.1.1-cdh6.x-SNAPSHOT:
> Failed to read artifact descriptor for
> org.apache.hive:hive-exec:jar:2.1.1-cdh6.x-SNAPSHOT: Could not find artifact
> com.cloudera.cdh:cdh-root:pom:6.x-20220131.115617-21547795 in
> cdh.rcs.releases.repo
> (https://repository.cloudera.com/artifactory/cdh-releases-rcs) -> [Help
> 1]{code}
> There are only metadata in 'cdh.rcs.releases.repo' while the artifacts is
> missing.
> !image-2024-01-25-10-53-04-706.png!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-12755) could not resolve artifact 'cdh-root:6.x-20220131.115617-21547795'
[
https://issues.apache.org/jira/browse/IMPALA-12755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17810647#comment-17810647
]
Quanlong Huang commented on IMPALA-12755:
-
Thanks for reporting this! I can reproduce the error. It seems disabing
snapshots from 'cdh.rcs.releases.repo' also works.
{noformat}
diff --git a/impala-parent/pom.xml b/impala-parent/pom.xml
index 45a8218a7..4b18d4c41 100644
--- a/impala-parent/pom.xml
+++ b/impala-parent/pom.xml
@@ -146,7 +146,7 @@ under the License.
https://repository.cloudera.com/artifactory/cdh-releases-rcs
CDH Releases Repository
-true
+false
{noformat}
> could not resolve artifact 'cdh-root:6.x-20220131.115617-21547795'
> --
>
> Key: IMPALA-12755
> URL: https://issues.apache.org/jira/browse/IMPALA-12755
> Project: IMPALA
> Issue Type: Bug
>Affects Versions: Impala 3.4.0, Impala 3.4.1
>Reporter: guojingfeng
>Priority: Major
> Attachments: image-2024-01-25-10-53-04-706.png
>
>
> We could not resolve 'cdh-root:6.x-20220131.115617-21547795' when building
> impala-3.4.x versions.
> {code:java}
> [INFO] BUILD FAILURE
> [ERROR] Failed to execute goal on project impala-minimal-hive-exec: Could not
> resolve dependencies for project
> org.apache.impala:impala-minimal-hive-exec:jar:0.1-SNAPSHOT: Failed to
> collect dependencies at org.apache.hive:hive-exec:jar:2.1.1-cdh6.x-SNAPSHOT:
> Failed to read artifact descriptor for
> org.apache.hive:hive-exec:jar:2.1.1-cdh6.x-SNAPSHOT: Could not find artifact
> com.cloudera.cdh:cdh-root:pom:6.x-20220131.115617-21547795 in
> cdh.rcs.releases.repo
> (https://repository.cloudera.com/artifactory/cdh-releases-rcs) -> [Help
> 1]{code}
> There are only metadata in 'cdh.rcs.releases.repo' while the artifacts is
> missing.
> !image-2024-01-25-10-53-04-706.png!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Work started] (IMPALA-12378) Auto Ship Data Source library in Impala
[ https://issues.apache.org/jira/browse/IMPALA-12378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on IMPALA-12378 started by Wenzhe Zhou. > Auto Ship Data Source library in Impala > --- > > Key: IMPALA-12378 > URL: https://issues.apache.org/jira/browse/IMPALA-12378 > Project: IMPALA > Issue Type: Sub-task > Components: Frontend, Infrastructure >Reporter: Wenzhe Zhou >Assignee: Wenzhe Zhou >Priority: Major > > The library of JDBC external data source should be auto shipped in Impala > binaries so that user don’t need to add the jar file manually. However jdbc > driver jars are provided by user. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-12756) [DOCS] Unicode column name support documentation
Pranav Yogi Lodha created IMPALA-12756: -- Summary: [DOCS] Unicode column name support documentation Key: IMPALA-12756 URL: https://issues.apache.org/jira/browse/IMPALA-12756 Project: IMPALA Issue Type: Documentation Reporter: Pranav Yogi Lodha -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Assigned] (IMPALA-12756) [DOCS] Unicode column name support documentation
[ https://issues.apache.org/jira/browse/IMPALA-12756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pranav Yogi Lodha reassigned IMPALA-12756: -- Assignee: Pranav Yogi Lodha > [DOCS] Unicode column name support documentation > > > Key: IMPALA-12756 > URL: https://issues.apache.org/jira/browse/IMPALA-12756 > Project: IMPALA > Issue Type: Documentation >Reporter: Pranav Yogi Lodha >Assignee: Pranav Yogi Lodha >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12756) [DOCS] Unicode column name support documentation
[ https://issues.apache.org/jira/browse/IMPALA-12756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pranav Yogi Lodha updated IMPALA-12756: --- Description: To document Impala currently support for unicode column names, consistent with Hive's current support (as we use Hive MetaStore to store table metadata). > [DOCS] Unicode column name support documentation > > > Key: IMPALA-12756 > URL: https://issues.apache.org/jira/browse/IMPALA-12756 > Project: IMPALA > Issue Type: Documentation >Reporter: Pranav Yogi Lodha >Assignee: Pranav Yogi Lodha >Priority: Major > > To document Impala currently support for unicode column names, consistent > with Hive's current support (as we use Hive MetaStore to store table > metadata). -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12755) could not resolve artifact 'cdh-root:6.x-20220131.115617-21547795'
[
https://issues.apache.org/jira/browse/IMPALA-12755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Quanlong Huang updated IMPALA-12755:
Target Version: Impala 3.4.2
> could not resolve artifact 'cdh-root:6.x-20220131.115617-21547795'
> --
>
> Key: IMPALA-12755
> URL: https://issues.apache.org/jira/browse/IMPALA-12755
> Project: IMPALA
> Issue Type: Bug
>Affects Versions: Impala 3.4.0, Impala 3.4.1
>Reporter: guojingfeng
>Priority: Major
> Attachments: image-2024-01-25-10-53-04-706.png
>
>
> We could not resolve 'cdh-root:6.x-20220131.115617-21547795' when building
> impala-3.4.x versions.
> {code:java}
> [INFO] BUILD FAILURE
> [ERROR] Failed to execute goal on project impala-minimal-hive-exec: Could not
> resolve dependencies for project
> org.apache.impala:impala-minimal-hive-exec:jar:0.1-SNAPSHOT: Failed to
> collect dependencies at org.apache.hive:hive-exec:jar:2.1.1-cdh6.x-SNAPSHOT:
> Failed to read artifact descriptor for
> org.apache.hive:hive-exec:jar:2.1.1-cdh6.x-SNAPSHOT: Could not find artifact
> com.cloudera.cdh:cdh-root:pom:6.x-20220131.115617-21547795 in
> cdh.rcs.releases.repo
> (https://repository.cloudera.com/artifactory/cdh-releases-rcs) -> [Help
> 1]{code}
> There are only metadata in 'cdh.rcs.releases.repo' while the artifacts is
> missing.
> !image-2024-01-25-10-53-04-706.png!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12756) [DOCS] Unicode column name support documentation
[ https://issues.apache.org/jira/browse/IMPALA-12756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pranav Yogi Lodha updated IMPALA-12756: --- Description: The patch focuses on documenting that Impala supports unicode column names, consistent with Hive's current support (as we use Hive MetaStore to store table metadata). (was: To document Impala currently support for unicode column names, consistent with Hive's current support (as we use Hive MetaStore to store table metadata).) > [DOCS] Unicode column name support documentation > > > Key: IMPALA-12756 > URL: https://issues.apache.org/jira/browse/IMPALA-12756 > Project: IMPALA > Issue Type: Documentation >Reporter: Pranav Yogi Lodha >Assignee: Pranav Yogi Lodha >Priority: Major > > The patch focuses on documenting that Impala supports unicode column names, > consistent with Hive's current support (as we use Hive MetaStore to store > table metadata). -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-10130) Catalog restart doesn't invalidate authPolicy cache in local catalog
[
https://issues.apache.org/jira/browse/IMPALA-10130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Quanlong Huang updated IMPALA-10130:
Target Version: Impala 3.4.3 (was: Impala 3.4.2)
> Catalog restart doesn't invalidate authPolicy cache in local catalog
>
>
> Key: IMPALA-10130
> URL: https://issues.apache.org/jira/browse/IMPALA-10130
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
>Affects Versions: Impala 2.0, Impala 3.0, Impala 4.0.0
>Reporter: Abhishek Rawat
>Priority: Major
>
> When the catalog service is restarted, LocalCatalog detects it in the topic
> update due to the change in its service id and invalidates its cache
> contents. However, it looks like it doesn't invalidate its existing
> authPolicy cache contents.
> {code:java}
> private void witnessCatalogServiceId(TUniqueId serviceId) {
> synchronized (catalogServiceIdLock_) {
> if (!catalogServiceId_.equals(serviceId)) {
> if (!catalogServiceId_.equals(Catalog.INITIAL_CATALOG_SERVICE_ID)) {
> LOG.warn("Detected catalog service restart: service ID changed from
> " +
> "{} to {}. Invalidating all cached metadata on this
> coordinator.",
> catalogServiceId_, serviceId);
> }
> catalogServiceId_ = serviceId;
> cache_.invalidateAll();
> // Clear cached items from the previous catalogd instance. Otherwise,
> we'll
> // ignore new updates from the new catalogd instance since they have
> lower
> // versions.
> hdfsCachePools_.clear();
> // TODO(todd): we probably need to invalidate the auth policy too.
> // we are probably better off detecting this at a higher level and
> // reinstantiating the metaprovider entirely, similar to how
> ImpaladCatalog
> // handles this.
> // TODO(todd): slight race here: a concurrent request from the old
> catalog
> // could theoretically be just about to write something back into the
> cache
> // after we do the above invalidate. Maybe we would be better off
> replacing
> // the whole cache object, or doing a soft barrier here to wait for
> any
> // concurrent cache accessors to cycle out. Another option is to
> associate
> // the catalog service ID as part of all of the cache keys.
> //
> // This is quite unlikely to be an issue in practice, so deferring it
> to later
> // clean-up.
> }
> }
> }
> {code}
> If the older authpolicy is not cleared above, it is possible that when the
> principle was added into the cache it was ignored since its catalog version
> was higher as seen below:
> {code:java}
> public synchronized void addPrincipal(Principal principal) {
> Principal existingPrincipal = getPrincipal(principal.getName(),
> principal.getPrincipalType());
> // There is already a newer version of this principal in the catalog,
> ignore
> // just return.
> if (existingPrincipal != null &&
> existingPrincipal.getCatalogVersion() >=
> principal.getCatalogVersion()) return;
> {code}
> When the update tries to add the privilege associated with the principle
> above, it looks up the principal using id instead of name and if there is a
> id mismatch it throws error.
> {code:java}
> /**
>* Adds a new privilege to the policy mapping to the principal specified by
> the
>* principal ID in the privilege. Throws a CatalogException no principal
> with a
>* corresponding ID existing in the catalog.
>*/
> public synchronized void addPrivilege(PrincipalPrivilege privilege)
> throws CatalogException {
> if (LOG.isTraceEnabled()) {
> LOG.trace("Adding privilege: " + privilege.getName() + " " +
> Principal.toString(privilege.getPrincipalType()).toLowerCase() +
> " ID: " + privilege.getPrincipalId());
> }
> Principal principal = getPrincipal(privilege.getPrincipalId(),
> privilege.getPrincipalType());
> if (principal == null) {
> throw new CatalogException(String.format("Error adding privilege: %s.
> %s ID " +
> "'%d' does not exist.", privilege.getName(),
> Principal.toString(privilege.getPrincipalType()),
> privilege.getPrincipalId()));
> }{code}
>
> The legacy catalog mode doesn't have this issue because the whole
> ImpaladCatalog instance is re-created when detecting catalogd restarts:
> {code:java}
> @Override
> TUpdateCatalogCacheResponse updateCatalogCache(TUpdateCatalogCacheRequest
> req)
> throws CatalogException, TException {
> ImpaladCatalog catalog = catalog_.get();
> if (req.is_delta) return catalog.updateCatalog(req);
> // If this is not a delta, this update shoul
[jira] [Updated] (IMPALA-10326) PrincipalPrivilegeTree doesn't handle empty string and wildcards correctly
[ https://issues.apache.org/jira/browse/IMPALA-10326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Quanlong Huang updated IMPALA-10326: Target Version: Impala 3.4.3 (was: Impala 3.4.2) > PrincipalPrivilegeTree doesn't handle empty string and wildcards correctly > -- > > Key: IMPALA-10326 > URL: https://issues.apache.org/jira/browse/IMPALA-10326 > Project: IMPALA > Issue Type: Bug > Components: Catalog >Affects Versions: Impala 3.4.0 >Reporter: Csaba Ringhofer >Assignee: Quanlong Huang >Priority: Major > Labels: sentry > > Two bugs slipped through the tests in IMPALA-9242: > 1. It assumes that unfilled elements in a privilege (e.g. table in a database > privilege) are null, which is true in FE tests, but it is an empty string > when the privilege comes from SentryPrivilege. > 2. wildcards (e.g. * in server=server1->db=tpcds->table=*) are not handled > well in the tree - it will be added as a table level privilege, while it > should be added as a database-wide privilege -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
issues-all@impala.apache.org
[
https://issues.apache.org/jira/browse/IMPALA-12166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17810719#comment-17810719
]
Quanlong Huang commented on IMPALA-12166:
-
[~ssimmen], sorry to be late on this. Could you upload the minidump file
(compressed it) and the error report file (hs_err_pid23611.log) ? More details
on the startup flags and specific configuration changes help us to reproduce
this.
Is the issue resolved now?
> Impala crashes with SIGSEGV in
> llvm::SCEVAddRecExpr::getNumIterationsInRange(llvm::ConstantRange const&,
> llvm::ScalarEvolution&)
>
>
> Key: IMPALA-12166
> URL: https://issues.apache.org/jira/browse/IMPALA-12166
> Project: IMPALA
> Issue Type: Bug
>Affects Versions: Impala 3.4.1
>Reporter: Silka Simmen
>Priority: Major
>
> We built impala in version 3.4.1 (on Centos 7). {{impalad}} and {{catalogd}}
> are crashing during their startup with the following log entries:
> {code:java}
> Wrote minidump to
> /var/log/impala/minidumps/impalad/5029e102-5879-425b-b921f69c-32ebd214.dmp
> #
> # A fatal error has been detected by the Java Runtime Environment:
> #
> # SIGSEGV (0xb) at pc=0x02bae34c, pid=23611, tid=0x7f80a36f69c0
> #
> # JRE version: OpenJDK Runtime Environment (8.0_372-b07) (build 1.8.0_372-b07)
> # Java VM: OpenJDK 64-Bit Server VM (25.372-b07 mixed mode linux-amd64
> compressed oops)
> # Problematic frame:
> # C [impalad+0x27ae34c]
> llvm::SCEVAddRecExpr::getNumIterationsInRange(llvm::ConstantRange const&,
> llvm::ScalarEvolution&) const+0x73c
> #
> # Core dump written. Default location: /var/lib/impala/core or core.23611
> #
> # An error report file with more information is saved as:
> # /var/lib/impala/hs_err_pid23611.log
> #
> # If you would like to submit a bug report, please visit:
> #
> https://bugzilla.redhat.com/enter_bug.cgi?product=Red%20Hat%20Enterprise%20Linux%207&component=java-1.8.0-openjdk
> # The crash happened outside the Java Virtual Machine in native code.
> # See problematic frame for where to report the bug.
> # {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-12709) Hierarchical metastore event processing
[ https://issues.apache.org/jira/browse/IMPALA-12709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17810730#comment-17810730 ] Maxwell Guo commented on IMPALA-12709: -- Hi [~VenuReddy], are there any plan on this patch ? such as the release timeline . If this patch is going to split into some small task , and I think I can do some help with some of the tasks. > Hierarchical metastore event processing > --- > > Key: IMPALA-12709 > URL: https://issues.apache.org/jira/browse/IMPALA-12709 > Project: IMPALA > Issue Type: Improvement > Components: Catalog >Reporter: Venugopal Reddy K >Assignee: Venugopal Reddy K >Priority: Major > Attachments: Hierarchical metastore event processing.docx > > > *Current Issue:* > At present, metastore event processor is single threaded. Notification events > are processed sequentially with a maximum limit of 1000 events fetched and > processed in a single batch. Multiple locks are used to address the > concurrency issues that may arise when catalog DDL operation processing and > metastore event processing tries to access/update the catalog objects > concurrently. Waiting for a lock or file metadata loading of a table can slow > the event processing and can affect the processing of other events following > it. Those events may not be dependent on the previous event. Altogether it > takes a very long time to synchronize all the HMS events. > *Proposal:* > Existing metastore event processing can be turned into multi-level event > processing. Idea is to segregate the events based on their dependency, > maintain the order of events as they occur within the dependency and process > them independently as much as possible: > # All the events of a table are processed in the same order they have > actually occurred. > # Events of different tables are processed in parallel. > # When a database is altered, all the events relating to the database(i.e., > for all its tables) occurring after the alter db event are processed only > after the alter database event is processed ensuring the order. > Have attached an initial proposal design document > https://docs.google.com/document/d/1KZ-ANko-qn5CYmY13m4OVJXAYjLaS1VP-c64Pumipq8/edit?pli=1#heading=h.qyk8qz8ez37b -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-12755) could not resolve artifact 'cdh-root:6.x-20220131.115617-21547795'
[
https://issues.apache.org/jira/browse/IMPALA-12755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17810735#comment-17810735
]
guojingfeng commented on IMPALA-12755:
--
Thanks for support, i verify the patch, i works.(y)
> could not resolve artifact 'cdh-root:6.x-20220131.115617-21547795'
> --
>
> Key: IMPALA-12755
> URL: https://issues.apache.org/jira/browse/IMPALA-12755
> Project: IMPALA
> Issue Type: Bug
>Affects Versions: Impala 3.4.0, Impala 3.4.1
>Reporter: guojingfeng
>Priority: Major
> Attachments: image-2024-01-25-10-53-04-706.png
>
>
> We could not resolve 'cdh-root:6.x-20220131.115617-21547795' when building
> impala-3.4.x versions.
> {code:java}
> [INFO] BUILD FAILURE
> [ERROR] Failed to execute goal on project impala-minimal-hive-exec: Could not
> resolve dependencies for project
> org.apache.impala:impala-minimal-hive-exec:jar:0.1-SNAPSHOT: Failed to
> collect dependencies at org.apache.hive:hive-exec:jar:2.1.1-cdh6.x-SNAPSHOT:
> Failed to read artifact descriptor for
> org.apache.hive:hive-exec:jar:2.1.1-cdh6.x-SNAPSHOT: Could not find artifact
> com.cloudera.cdh:cdh-root:pom:6.x-20220131.115617-21547795 in
> cdh.rcs.releases.repo
> (https://repository.cloudera.com/artifactory/cdh-releases-rcs) -> [Help
> 1]{code}
> There are only metadata in 'cdh.rcs.releases.repo' while the artifacts is
> missing.
> !image-2024-01-25-10-53-04-706.png!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
