[jira] [Work stopped] (IMPALA-12752) Enable JS tests for webUI in jenkins
[ https://issues.apache.org/jira/browse/IMPALA-12752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on IMPALA-12752 stopped by Surya Hebbar. - > Enable JS tests for webUI in jenkins > > > Key: IMPALA-12752 > URL: https://issues.apache.org/jira/browse/IMPALA-12752 > Project: IMPALA > Issue Type: Improvement >Reporter: Surya Hebbar >Assignee: Surya Hebbar >Priority: Major > > The JS tests have been integrated into Impala. But, it has not been > integrated into jenkins builds. So, it is necessary to integrate with jenkins. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-12803) Fix missing exchange lines in query timeline
Surya Hebbar created IMPALA-12803: - Summary: Fix missing exchange lines in query timeline Key: IMPALA-12803 URL: https://issues.apache.org/jira/browse/IMPALA-12803 Project: IMPALA Issue Type: Bug Reporter: Surya Hebbar Attachments: missing_exchange_lines.png In the fragment diagram of the query timeline, the exchange lines between nodes is missing when plan order is used. !missing_exchange_lines.png! -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Work started] (IMPALA-12803) Fix missing exchange lines in query timeline
[ https://issues.apache.org/jira/browse/IMPALA-12803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on IMPALA-12803 started by Surya Hebbar. - > Fix missing exchange lines in query timeline > > > Key: IMPALA-12803 > URL: https://issues.apache.org/jira/browse/IMPALA-12803 > Project: IMPALA > Issue Type: Bug >Reporter: Surya Hebbar >Assignee: Surya Hebbar >Priority: Major > Attachments: missing_exchange_lines.png > > > In the fragment diagram of the query timeline, the exchange lines between > nodes is missing when plan order is used. !missing_exchange_lines.png! -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Assigned] (IMPALA-12803) Fix missing exchange lines in query timeline
[ https://issues.apache.org/jira/browse/IMPALA-12803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Surya Hebbar reassigned IMPALA-12803: - Assignee: Surya Hebbar > Fix missing exchange lines in query timeline > > > Key: IMPALA-12803 > URL: https://issues.apache.org/jira/browse/IMPALA-12803 > Project: IMPALA > Issue Type: Bug >Reporter: Surya Hebbar >Assignee: Surya Hebbar >Priority: Major > Attachments: missing_exchange_lines.png > > > In the fragment diagram of the query timeline, the exchange lines between > nodes is missing when plan order is used. !missing_exchange_lines.png! -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12688) Support JSON profile imports in webUI
[ https://issues.apache.org/jira/browse/IMPALA-12688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Surya Hebbar updated IMPALA-12688: -- Attachment: imported_profiles_section.png > Support JSON profile imports in webUI > - > > Key: IMPALA-12688 > URL: https://issues.apache.org/jira/browse/IMPALA-12688 > Project: IMPALA > Issue Type: New Feature >Reporter: Surya Hebbar >Assignee: Surya Hebbar >Priority: Major > Attachments: imported_profiles_section.png, > imported_queries_button.png, imported_queries_list.png, > imported_queries_page.png, imported_query_statement.png, > imported_query_text_plan.png, imported_query_timeline.png > > > It would be helpful for users to visualize the query timeline by selecting a > local JSON query profile. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-12688) Support JSON profile imports in webUI
[ https://issues.apache.org/jira/browse/IMPALA-12688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17816062#comment-17816062 ] Surya Hebbar commented on IMPALA-12688: --- The "Imported Queries" section has been renamed to "Imported Query Profiles". Also, the "Duration" section has been removed, as it was a recent addition to the query profile. !imported_profiles_section.png|width=936,height=116! > Support JSON profile imports in webUI > - > > Key: IMPALA-12688 > URL: https://issues.apache.org/jira/browse/IMPALA-12688 > Project: IMPALA > Issue Type: New Feature >Reporter: Surya Hebbar >Assignee: Surya Hebbar >Priority: Major > Attachments: imported_profiles_section.png, > imported_queries_button.png, imported_queries_list.png, > imported_queries_page.png, imported_query_statement.png, > imported_query_text_plan.png, imported_query_timeline.png > > > It would be helpful for users to visualize the query timeline by selecting a > local JSON query profile. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12688) Support JSON profile imports in webUI
[ https://issues.apache.org/jira/browse/IMPALA-12688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Surya Hebbar updated IMPALA-12688: -- Attachment: descending_order_start_time.png > Support JSON profile imports in webUI > - > > Key: IMPALA-12688 > URL: https://issues.apache.org/jira/browse/IMPALA-12688 > Project: IMPALA > Issue Type: New Feature >Reporter: Surya Hebbar >Assignee: Surya Hebbar >Priority: Major > Attachments: descending_order_start_time.png, > imported_profiles_section.png, imported_queries_button.png, > imported_queries_list.png, imported_queries_page.png, > imported_query_statement.png, imported_query_text_plan.png, > imported_query_timeline.png > > > It would be helpful for users to visualize the query timeline by selecting a > local JSON query profile. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12688) Support JSON profile imports in webUI
[ https://issues.apache.org/jira/browse/IMPALA-12688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Surya Hebbar updated IMPALA-12688: -- Attachment: descending_order_start_time-1.png > Support JSON profile imports in webUI > - > > Key: IMPALA-12688 > URL: https://issues.apache.org/jira/browse/IMPALA-12688 > Project: IMPALA > Issue Type: New Feature >Reporter: Surya Hebbar >Assignee: Surya Hebbar >Priority: Major > Attachments: descending_order_start_time.png, > imported_profiles_section.png, imported_queries_button.png, > imported_queries_list.png, imported_queries_page.png, > imported_query_statement.png, imported_query_text_plan.png, > imported_query_timeline.png > > > It would be helpful for users to visualize the query timeline by selecting a > local JSON query profile. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12688) Support JSON profile imports in webUI
[ https://issues.apache.org/jira/browse/IMPALA-12688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Surya Hebbar updated IMPALA-12688: -- Attachment: (was: descending_order_start_time.png) > Support JSON profile imports in webUI > - > > Key: IMPALA-12688 > URL: https://issues.apache.org/jira/browse/IMPALA-12688 > Project: IMPALA > Issue Type: New Feature >Reporter: Surya Hebbar >Assignee: Surya Hebbar >Priority: Major > Attachments: descending_order_start_time.png, > imported_profiles_section.png, imported_queries_button.png, > imported_queries_list.png, imported_queries_page.png, > imported_query_statement.png, imported_query_text_plan.png, > imported_query_timeline.png > > > It would be helpful for users to visualize the query timeline by selecting a > local JSON query profile. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12688) Support JSON profile imports in webUI
[ https://issues.apache.org/jira/browse/IMPALA-12688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Surya Hebbar updated IMPALA-12688: -- Attachment: descending_order_start_time.png > Support JSON profile imports in webUI > - > > Key: IMPALA-12688 > URL: https://issues.apache.org/jira/browse/IMPALA-12688 > Project: IMPALA > Issue Type: New Feature >Reporter: Surya Hebbar >Assignee: Surya Hebbar >Priority: Major > Attachments: descending_order_start_time.png, > imported_profiles_section.png, imported_queries_button.png, > imported_queries_list.png, imported_queries_page.png, > imported_query_statement.png, imported_query_text_plan.png, > imported_query_timeline.png > > > It would be helpful for users to visualize the query timeline by selecting a > local JSON query profile. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12688) Support JSON profile imports in webUI
[ https://issues.apache.org/jira/browse/IMPALA-12688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Surya Hebbar updated IMPALA-12688: -- Attachment: (was: descending_order_start_time-1.png) > Support JSON profile imports in webUI > - > > Key: IMPALA-12688 > URL: https://issues.apache.org/jira/browse/IMPALA-12688 > Project: IMPALA > Issue Type: New Feature >Reporter: Surya Hebbar >Assignee: Surya Hebbar >Priority: Major > Attachments: descending_order_start_time.png, > imported_profiles_section.png, imported_queries_button.png, > imported_queries_list.png, imported_queries_page.png, > imported_query_statement.png, imported_query_text_plan.png, > imported_query_timeline.png > > > It would be helpful for users to visualize the query timeline by selecting a > local JSON query profile. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-12688) Support JSON profile imports in webUI
[ https://issues.apache.org/jira/browse/IMPALA-12688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17816064#comment-17816064 ] Surya Hebbar commented on IMPALA-12688: --- Imported query profiles are now sorted in the descending order of "Start Time". !descending_order_start_time.png|width=941,height=312! > Support JSON profile imports in webUI > - > > Key: IMPALA-12688 > URL: https://issues.apache.org/jira/browse/IMPALA-12688 > Project: IMPALA > Issue Type: New Feature >Reporter: Surya Hebbar >Assignee: Surya Hebbar >Priority: Major > Attachments: descending_order_start_time.png, > imported_profiles_section.png, imported_queries_button.png, > imported_queries_list.png, imported_queries_page.png, > imported_query_statement.png, imported_query_text_plan.png, > imported_query_timeline.png > > > It would be helpful for users to visualize the query timeline by selecting a > local JSON query profile. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-12688) Support JSON profile imports in webUI
[ https://issues.apache.org/jira/browse/IMPALA-12688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17816065#comment-17816065 ] Surya Hebbar commented on IMPALA-12688: --- Multiple query profile imports are now supported. > Support JSON profile imports in webUI > - > > Key: IMPALA-12688 > URL: https://issues.apache.org/jira/browse/IMPALA-12688 > Project: IMPALA > Issue Type: New Feature >Reporter: Surya Hebbar >Assignee: Surya Hebbar >Priority: Major > Attachments: descending_order_start_time.png, > imported_profiles_section.png, imported_queries_button.png, > imported_queries_list.png, imported_queries_page.png, > imported_query_statement.png, imported_query_text_plan.png, > imported_query_timeline.png > > > It would be helpful for users to visualize the query timeline by selecting a > local JSON query profile. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12688) Support JSON profile imports in webUI
[ https://issues.apache.org/jira/browse/IMPALA-12688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Surya Hebbar updated IMPALA-12688: -- Attachment: multiple_query_profile_import.png > Support JSON profile imports in webUI > - > > Key: IMPALA-12688 > URL: https://issues.apache.org/jira/browse/IMPALA-12688 > Project: IMPALA > Issue Type: New Feature >Reporter: Surya Hebbar >Assignee: Surya Hebbar >Priority: Major > Attachments: descending_order_start_time.png, > imported_profiles_section.png, imported_queries_button.png, > imported_queries_list.png, imported_queries_page.png, > imported_query_statement.png, imported_query_text_plan.png, > imported_query_timeline.png, multiple_query_profile_import.png > > > It would be helpful for users to visualize the query timeline by selecting a > local JSON query profile. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Comment Edited] (IMPALA-12688) Support JSON profile imports in webUI
[ https://issues.apache.org/jira/browse/IMPALA-12688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17816065#comment-17816065 ] Surya Hebbar edited comment on IMPALA-12688 at 2/9/24 1:38 PM: --- Multiple query profile imports are now supported. !multiple_query_profile_import.png|width=940,height=562! was (Author: JIRAUSER299620): Multiple query profile imports are now supported. > Support JSON profile imports in webUI > - > > Key: IMPALA-12688 > URL: https://issues.apache.org/jira/browse/IMPALA-12688 > Project: IMPALA > Issue Type: New Feature >Reporter: Surya Hebbar >Assignee: Surya Hebbar >Priority: Major > Attachments: descending_order_start_time.png, > imported_profiles_section.png, imported_queries_button.png, > imported_queries_list.png, imported_queries_page.png, > imported_query_statement.png, imported_query_text_plan.png, > imported_query_timeline.png, multiple_query_profile_import.png > > > It would be helpful for users to visualize the query timeline by selecting a > local JSON query profile. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-12688) Support JSON profile imports in webUI
[ https://issues.apache.org/jira/browse/IMPALA-12688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17816066#comment-17816066 ] Surya Hebbar commented on IMPALA-12688: --- A "Clear All" button has been provided to remove all imported query profiles from browser's storage at once. !clear_all_button.png|width=940,height=424! > Support JSON profile imports in webUI > - > > Key: IMPALA-12688 > URL: https://issues.apache.org/jira/browse/IMPALA-12688 > Project: IMPALA > Issue Type: New Feature >Reporter: Surya Hebbar >Assignee: Surya Hebbar >Priority: Major > Attachments: clear_all_button.png, descending_order_start_time.png, > imported_profiles_section.png, imported_queries_button.png, > imported_queries_list.png, imported_queries_page.png, > imported_query_statement.png, imported_query_text_plan.png, > imported_query_timeline.png, multiple_query_profile_import.png > > > It would be helpful for users to visualize the query timeline by selecting a > local JSON query profile. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12688) Support JSON profile imports in webUI
[ https://issues.apache.org/jira/browse/IMPALA-12688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Surya Hebbar updated IMPALA-12688: -- Attachment: clear_all_button.png > Support JSON profile imports in webUI > - > > Key: IMPALA-12688 > URL: https://issues.apache.org/jira/browse/IMPALA-12688 > Project: IMPALA > Issue Type: New Feature >Reporter: Surya Hebbar >Assignee: Surya Hebbar >Priority: Major > Attachments: clear_all_button.png, descending_order_start_time.png, > imported_profiles_section.png, imported_queries_button.png, > imported_queries_list.png, imported_queries_page.png, > imported_query_statement.png, imported_query_text_plan.png, > imported_query_timeline.png, multiple_query_profile_import.png > > > It would be helpful for users to visualize the query timeline by selecting a > local JSON query profile. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Comment Edited] (IMPALA-12688) Support JSON profile imports in webUI
[ https://issues.apache.org/jira/browse/IMPALA-12688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17816066#comment-17816066 ] Surya Hebbar edited comment on IMPALA-12688 at 2/9/24 1:40 PM: --- A "Clear All" button has been provided to remove all imported query profiles from the browser's storage at once. !clear_all_button.png|width=940,height=424! was (Author: JIRAUSER299620): A "Clear All" button has been provided to remove all imported query profiles from browser's storage at once. !clear_all_button.png|width=940,height=424! > Support JSON profile imports in webUI > - > > Key: IMPALA-12688 > URL: https://issues.apache.org/jira/browse/IMPALA-12688 > Project: IMPALA > Issue Type: New Feature >Reporter: Surya Hebbar >Assignee: Surya Hebbar >Priority: Major > Attachments: clear_all_button.png, descending_order_start_time.png, > imported_profiles_section.png, imported_queries_button.png, > imported_queries_list.png, imported_queries_page.png, > imported_query_statement.png, imported_query_text_plan.png, > imported_query_timeline.png, multiple_query_profile_import.png > > > It would be helpful for users to visualize the query timeline by selecting a > local JSON query profile. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-12783) Nested struct with varlen data crashes
[
https://issues.apache.org/jira/browse/IMPALA-12783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Becker resolved IMPALA-12783.
Resolution: Fixed
> Nested struct with varlen data crashes
> --
>
> Key: IMPALA-12783
> URL: https://issues.apache.org/jira/browse/IMPALA-12783
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Reporter: Daniel Becker
>Assignee: Daniel Becker
>Priority: Major
>
> If a struct ("main") is within an array and contains two child structs ("s1"
> ans "s2") which both contain strings (or other varlen data), it crashes when
> re-materialised (for example in a sort with limit) if codegen is enabled.
> To reproduce:
> In Hive:
> {code:java}
> create table nested (arr ARRAY, s2:
> STRUCT>>) stored as parquet;
> insert into nested values (array( named_struct("s1", named_struct("str1", "A
> string that is long"), "s2", named_struct("str2", "Another string that is
> long") )));{code}
> In Impala:
> {code:java}
> select 1, arr from nested order by 1 limit 1;{code}
> This seems to be because in the codegen'd code, when checking if the strings
> ("str1" and "str2" in the example) are NULL, we incorrectly calculate the
> offset of the null indicator byte from the memory adress of their containing
> struct, not from the beginning of the "master tuple", which in this case is
> the item tuple of the array.
> Note that the null indicators of the struct members are at the end of the
> tuple containing the struct (recursively), i.e. the master tuple.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Assigned] (IMPALA-12264) Add configuration to limit concurrent connections
[ https://issues.apache.org/jira/browse/IMPALA-12264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman reassigned IMPALA-12264: --- Assignee: Andrew Sherman > Add configuration to limit concurrent connections > - > > Key: IMPALA-12264 > URL: https://issues.apache.org/jira/browse/IMPALA-12264 > Project: IMPALA > Issue Type: New Feature > Components: Backend >Reporter: Quanlong Huang >Assignee: Andrew Sherman >Priority: Major > > To prevent a rogue application from repeatedly connecting to and monopolizing > Impala, we can add configurations to limit concurrent connections. E.g. limit > the number of concurrent connections per user, IP address, or the user and IP > address combination. > CC [~mylogi...@gmail.com] -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-12804) C files should be compiled with equivalent flags as the C++ files
Joe McDonnell created IMPALA-12804: -- Summary: C files should be compiled with equivalent flags as the C++ files Key: IMPALA-12804 URL: https://issues.apache.org/jira/browse/IMPALA-12804 Project: IMPALA Issue Type: Improvement Components: Backend Affects Versions: Impala 4.3.0 Reporter: Joe McDonnell Impala's be/CMakeLists.txt does calculations to determine CMAKE_CXX_FLAGS, but it does not set CMAKE_C_FLAGS. This means that C files don't get the options that we specify on C++ files. This is pretty minor, because there are very few C files in the codebase. Even so, it would be good to specify similar flags for CMAKE_C_FLAGS when we set CMAKE_CXX_FLAGS. As an example, be/src/thirdparty/squeasel/squeasel.c won't be compiled with '-fno-omit-frame-pointer' or debug information for release builds. If it fails during webserver startup (e.g. when it tries to read the private key to setup SSL), the resulting stack in the minidump is jumbled a bit. This should be relatively easy to fix, but it may involve reorganizing the way we calculate CMAKE_CXX_FLAGS. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-12805) Avoid problems of HIVE-27114 without configuration changes
Andrew Sherman created IMPALA-12805: --- Summary: Avoid problems of HIVE-27114 without configuration changes Key: IMPALA-12805 URL: https://issues.apache.org/jira/browse/IMPALA-12805 Project: IMPALA Issue Type: Bug Components: Catalog Affects Versions: Impala 4.4.0 Reporter: Andrew Sherman Assignee: Andrew Sherman HIVE-27114 adds a new property,"hive.metastore.partitions.parameters.exclude.pattern", which defaults to "impala_intermediate_stats_chunk%". This default excludes the Impala incremental stats from the results returned from the HMS client. I think this was done to help Hive performance. It means however that Impala cannot retrieve its own partition stats, which will negatively impact the performance of many workloads. One fix for this was IMPALA-12743 which sets a non-empty non-default value for the new Hive property in the development environment’s hive-site.xml file. In some customer situations this would require a configuration change that cannot be deployed simultaneously with the possible HMS upgrade which introduces the regression. This change sets a non-empty non-default value for the new Hive property in the configurations in the MetaStoreClientPool. This avoids the regression. The natural thing would be to set an empty string as the value, but th -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12805) Avoid problems of HIVE-27114 without configuration changes
[ https://issues.apache.org/jira/browse/IMPALA-12805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman updated IMPALA-12805: Description: HIVE-27114 adds a new property,"hive.metastore.partitions.parameters.exclude.pattern", which defaults to "impala_intermediate_stats_chunk%". This default excludes the Impala incremental stats from the results returned from the HMS client. I think this was done to help Hive performance. It means however that Impala cannot retrieve its own partition stats, which will negatively impact the performance of many workloads. One fix for this was IMPALA-12743 which sets a non-empty non-default value for the new Hive property in the development environment’s hive-site.xml file. In some customer situations this would require a configuration change that cannot be deployed simultaneously with the possible HMS upgrade which introduces the regression. Fix this by setting a non-empty non-default value for the new Hive property in the configurations in the MetaStoreClientPool. This avoids the regression. The natural thing would be to set an empty string as the value, but this is not enough to override the default. Instead we set the string “do_not_match” which should never be used for as the name of any partition stats. was: HIVE-27114 adds a new property,"hive.metastore.partitions.parameters.exclude.pattern", which defaults to "impala_intermediate_stats_chunk%". This default excludes the Impala incremental stats from the results returned from the HMS client. I think this was done to help Hive performance. It means however that Impala cannot retrieve its own partition stats, which will negatively impact the performance of many workloads. One fix for this was IMPALA-12743 which sets a non-empty non-default value for the new Hive property in the development environment’s hive-site.xml file. In some customer situations this would require a configuration change that cannot be deployed simultaneously with the possible HMS upgrade which introduces the regression. This change sets a non-empty non-default value for the new Hive property in the configurations in the MetaStoreClientPool. This avoids the regression. The natural thing would be to set an empty string as the value, but this is not enough to override the default. Instead we set the string “do_not_match” which should never be used for as the name of any partition stats. > Avoid problems of HIVE-27114 without configuration changes > -- > > Key: IMPALA-12805 > URL: https://issues.apache.org/jira/browse/IMPALA-12805 > Project: IMPALA > Issue Type: Bug > Components: Catalog >Affects Versions: Impala 4.4.0 >Reporter: Andrew Sherman >Assignee: Andrew Sherman >Priority: Critical > > HIVE-27114 adds a new > property,"hive.metastore.partitions.parameters.exclude.pattern", which > defaults to "impala_intermediate_stats_chunk%". This default excludes the > Impala incremental stats from the results returned from the HMS client. I > think this was done to help Hive performance. It means however that Impala > cannot retrieve its own partition stats, which will negatively impact the > performance of many workloads. > One fix for this was IMPALA-12743 which sets a non-empty non-default value > for the new Hive property in the development environment’s hive-site.xml > file. In some customer situations this would require a configuration change > that cannot be deployed simultaneously with the possible HMS upgrade which > introduces the regression. > Fix this by setting a non-empty non-default value for the new Hive property > in the configurations in the MetaStoreClientPool. This avoids the regression. > The natural thing would be to set an empty string as the value, but this is > not enough to override the default. Instead we set the string “do_not_match” > which should never be used for as the name of any partition stats. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12805) Avoid problems of HIVE-27114 without configuration changes
[ https://issues.apache.org/jira/browse/IMPALA-12805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman updated IMPALA-12805: Description: HIVE-27114 adds a new property,"hive.metastore.partitions.parameters.exclude.pattern", which defaults to "impala_intermediate_stats_chunk%". This default excludes the Impala incremental stats from the results returned from the HMS client. I think this was done to help Hive performance. It means however that Impala cannot retrieve its own partition stats, which will negatively impact the performance of many workloads. One fix for this was IMPALA-12743 which sets a non-empty non-default value for the new Hive property in the development environment’s hive-site.xml file. In some customer situations this would require a configuration change that cannot be deployed simultaneously with the possible HMS upgrade which introduces the regression. This change sets a non-empty non-default value for the new Hive property in the configurations in the MetaStoreClientPool. This avoids the regression. The natural thing would be to set an empty string as the value, but this is not enough to override the default. Instead we set the string “do_not_match” which should never be used for as the name of any partition stats. was: HIVE-27114 adds a new property,"hive.metastore.partitions.parameters.exclude.pattern", which defaults to "impala_intermediate_stats_chunk%". This default excludes the Impala incremental stats from the results returned from the HMS client. I think this was done to help Hive performance. It means however that Impala cannot retrieve its own partition stats, which will negatively impact the performance of many workloads. One fix for this was IMPALA-12743 which sets a non-empty non-default value for the new Hive property in the development environment’s hive-site.xml file. In some customer situations this would require a configuration change that cannot be deployed simultaneously with the possible HMS upgrade which introduces the regression. This change sets a non-empty non-default value for the new Hive property in the configurations in the MetaStoreClientPool. This avoids the regression. The natural thing would be to set an empty string as the value, but th > Avoid problems of HIVE-27114 without configuration changes > -- > > Key: IMPALA-12805 > URL: https://issues.apache.org/jira/browse/IMPALA-12805 > Project: IMPALA > Issue Type: Bug > Components: Catalog >Affects Versions: Impala 4.4.0 >Reporter: Andrew Sherman >Assignee: Andrew Sherman >Priority: Critical > > HIVE-27114 adds a new > property,"hive.metastore.partitions.parameters.exclude.pattern", which > defaults to "impala_intermediate_stats_chunk%". This default excludes the > Impala incremental stats from the results returned from the HMS client. I > think this was done to help Hive performance. It means however that Impala > cannot retrieve its own partition stats, which will negatively impact the > performance of many workloads. > One fix for this was IMPALA-12743 which sets a non-empty non-default value > for the new Hive property in the development environment’s hive-site.xml > file. In some customer situations this would require a configuration change > that cannot be deployed simultaneously with the possible HMS upgrade which > introduces the regression. > This change sets a non-empty non-default value for the new Hive property in > the configurations in the MetaStoreClientPool. This avoids the regression. > The natural thing would be to set an empty string as the value, but this is > not enough to override the default. Instead we set the string “do_not_match” > which should never be used for as the name of any partition stats. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-12806) Set HIVE_DEFAULT_NULLS_LAST in HS2 OpenSession Response
Kurt Deschler created IMPALA-12806: -- Summary: Set HIVE_DEFAULT_NULLS_LAST in HS2 OpenSession Response Key: IMPALA-12806 URL: https://issues.apache.org/jira/browse/IMPALA-12806 Project: IMPALA Issue Type: Improvement Reporter: Kurt Deschler JDBC metadata functions nullsAreSortedLow() and nullsAreSortedHigh() depend on responses from the underlying database for correctness. The method used by Hive is to set HIVE_DEFAULT_NULLS_LAST in the HS2 OpenSession Response. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Assigned] (IMPALA-12806) Set HIVE_DEFAULT_NULLS_LAST in HS2 OpenSession Response
[ https://issues.apache.org/jira/browse/IMPALA-12806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kurt Deschler reassigned IMPALA-12806: -- Assignee: Kurt Deschler > Set HIVE_DEFAULT_NULLS_LAST in HS2 OpenSession Response > --- > > Key: IMPALA-12806 > URL: https://issues.apache.org/jira/browse/IMPALA-12806 > Project: IMPALA > Issue Type: Improvement >Reporter: Kurt Deschler >Assignee: Kurt Deschler >Priority: Major > > JDBC metadata functions nullsAreSortedLow() and nullsAreSortedHigh() depend > on responses from the underlying database for correctness. The method used by > Hive is to set HIVE_DEFAULT_NULLS_LAST in the HS2 OpenSession Response. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-12807) Add option to use the 'mold' linker
Joe McDonnell created IMPALA-12807:
--
Summary: Add option to use the 'mold' linker
Key: IMPALA-12807
URL: https://issues.apache.org/jira/browse/IMPALA-12807
Project: IMPALA
Issue Type: Improvement
Components: Backend
Affects Versions: Impala 4.3.0
Reporter: Joe McDonnell
Mold is a new linker that says it is faster than lld and gold. See
[https://github.com/rui314/mold]
Hand testing it on my machine, it makes a big difference in speed of iteration:
{noformat}
# Test case:
# - Start with fully built impalad
# - touch be/src/scheduling/scheduler.cc
# - time make -j12 impalad
With Gold (current default):
real 0m15.843s
user 0m15.478s
sys 0m2.127s
real 0m15.820s
user 0m15.302s
sys 0m2.157s
real 0m16.136s
user 0m15.799s
sys 0m2.098s
With Mold:
real 0m2.479s
user 0m2.169s
sys 0m0.958s
real 0m2.674s
user 0m2.218s
sys 0m1.086s
real 0m2.524s
user 0m2.136s
sys 0m1.042s{noformat}
This seems like something we should investigate further.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-12808) test_iceberg_deletes_and_updates failed by timeout error
Quanlong Huang created IMPALA-12808:
---
Summary: test_iceberg_deletes_and_updates failed by timeout error
Key: IMPALA-12808
URL: https://issues.apache.org/jira/browse/IMPALA-12808
Project: IMPALA
Issue Type: Bug
Reporter: Quanlong Huang
Assignee: Zoltán Borók-Nagy
Saw the test failed in timeout error:
{noformat}
/data/jenkins/workspace/impala-asf-master-exhaustive-data-cache/repos/Impala/tests/stress/test_update_stress.py:247:
in test_iceberg_deletes_and_updates
run_tasks([deleter, updater, checker])
stress/stress_util.py:46: in run_tasks
pool.map_async(Task.run, tasks).get(timeout_seconds)
/data/jenkins/workspace/impala-asf-master-exhaustive-data-cache/Impala-Toolchain/toolchain-packages-gcc10.4.0/python-2.7.16/lib/python2.7/multiprocessing/pool.py:568:
in get
raise TimeoutError
E TimeoutError{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12808) test_iceberg_deletes_and_updates failed by timeout error
[
https://issues.apache.org/jira/browse/IMPALA-12808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Quanlong Huang updated IMPALA-12808:
Attachment: std_err.txt
> test_iceberg_deletes_and_updates failed by timeout error
>
>
> Key: IMPALA-12808
> URL: https://issues.apache.org/jira/browse/IMPALA-12808
> Project: IMPALA
> Issue Type: Bug
>Reporter: Quanlong Huang
>Assignee: Zoltán Borók-Nagy
>Priority: Critical
> Attachments: std_err.txt
>
>
> Saw the test failed in timeout error:
> {noformat}
> /data/jenkins/workspace/impala-asf-master-exhaustive-data-cache/repos/Impala/tests/stress/test_update_stress.py:247:
> in test_iceberg_deletes_and_updates
> run_tasks([deleter, updater, checker])
> stress/stress_util.py:46: in run_tasks
> pool.map_async(Task.run, tasks).get(timeout_seconds)
> /data/jenkins/workspace/impala-asf-master-exhaustive-data-cache/Impala-Toolchain/toolchain-packages-gcc10.4.0/python-2.7.16/lib/python2.7/multiprocessing/pool.py:568:
> in get
> raise TimeoutError
> E TimeoutError{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Assigned] (IMPALA-12543) test_iceberg_self_events failed in JDK11 build
[
https://issues.apache.org/jira/browse/IMPALA-12543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Riza Suminto reassigned IMPALA-12543:
-
Assignee: Riza Suminto (was: Gabor Kaszab)
> test_iceberg_self_events failed in JDK11 build
> --
>
> Key: IMPALA-12543
> URL: https://issues.apache.org/jira/browse/IMPALA-12543
> Project: IMPALA
> Issue Type: Bug
>Reporter: Riza Suminto
>Assignee: Riza Suminto
>Priority: Major
> Labels: broken-build
>
> test_iceberg_self_events failed in JDK11 build with following error.
>
> {code:java}
> Error Message
> assert 0 == 1
> Stacktrace
> custom_cluster/test_events_custom_configs.py:637: in test_iceberg_self_events
> check_self_events("ALTER TABLE {0} ADD COLUMN j INT".format(tbl_name))
> custom_cluster/test_events_custom_configs.py:624: in check_self_events
> assert tbls_refreshed_before == tbls_refreshed_after
> E assert 0 == 1 {code}
> This test still pass before IMPALA-11387 merged.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-12543) test_iceberg_self_events failed in JDK11 build
[
https://issues.apache.org/jira/browse/IMPALA-12543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17816270#comment-17816270
]
Riza Suminto commented on IMPALA-12543:
---
[~hemanth619] raising hms_event_polling_interval_s from 1 to 5 seconds seems to
help.
[https://gerrit.cloudera.org/c/21029/]
> test_iceberg_self_events failed in JDK11 build
> --
>
> Key: IMPALA-12543
> URL: https://issues.apache.org/jira/browse/IMPALA-12543
> Project: IMPALA
> Issue Type: Bug
>Reporter: Riza Suminto
>Assignee: Riza Suminto
>Priority: Major
> Labels: broken-build
>
> test_iceberg_self_events failed in JDK11 build with following error.
>
> {code:java}
> Error Message
> assert 0 == 1
> Stacktrace
> custom_cluster/test_events_custom_configs.py:637: in test_iceberg_self_events
> check_self_events("ALTER TABLE {0} ADD COLUMN j INT".format(tbl_name))
> custom_cluster/test_events_custom_configs.py:624: in check_self_events
> assert tbls_refreshed_before == tbls_refreshed_after
> E assert 0 == 1 {code}
> This test still pass before IMPALA-11387 merged.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-12788) HBaseTable still get loaded even if HBase is down
[
https://issues.apache.org/jira/browse/IMPALA-12788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17816281#comment-17816281
]
ASF subversion and git services commented on IMPALA-12788:
--
Commit 11d2fe4fc00a1e6ef2d3a45825be9845456adc1d in impala's branch
refs/heads/master from stiga-huang
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=11d2fe4fc ]
IMPALA-12788: Fix HBaseTable still get loaded even if HBase is down
When loading a table backed by HBase, it's intended to check whether the
table exists in HBase in HBaseTable.load() and loadFromThrift().
However, the current check just gets the table object and then closes
it. This won't fail even if HBase is down. See JIRA description for the
stacktrace.
This patch fixes the check to fetch the column family names which is a
light-weight request and will fail if HBase is down or the table doesn't
exist in HBase.
Splits the following tests to skip the HBase part when running on S3:
- TestNestedStructsInSelectList.test_struct_in_select_list
- TestDdlStatements.test_alter_set_column_stats
- TestShowCreateTable.test_show_create_table
Tests:
- Run CORE tests on S3
Change-Id: Ib497f11ecc338d0f84d3d7bd8ccfcf8da4def0cb
Reviewed-on: http://gerrit.cloudera.org:8080/21003
Reviewed-by: Quanlong Huang
Tested-by: Impala Public Jenkins
> HBaseTable still get loaded even if HBase is down
> -
>
> Key: IMPALA-12788
> URL: https://issues.apache.org/jira/browse/IMPALA-12788
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
>Reporter: Quanlong Huang
>Assignee: Quanlong Huang
>Priority: Critical
>
> This is identified by an internal S3 build that doesn't launch HBase. There
> are some tests that still run queries on HBase tables, e.g.
> TestDdlStatements::test_alter_set_column_stats. But they don't fail on even
> if the table can't be correctly loaded. Catalogd logs show that the
> connection failure to HBase is ignored:
> {noformat}
> I0203 14:12:33.687620 20673 TableLoadingMgr.java:71] Loading metadata for
> table: functional_hbase.alltypes
> I0203 14:12:33.687674 24282 TableLoader.java:76] Loading metadata for:
> functional_hbase.alltypes (background load)
> I0203 14:12:33.687706 20673 TableLoadingMgr.java:73] Remaining items in
> queue: 0. Loads in progress: 1
> I0203 14:12:33.690941 26564 JniCatalog.java:257] execDdl request:
> DROP_DATABASE test_compute_stats_9c95c5d8 issued by jenkins
> I0203 14:12:33.691668 24282 Table.java:218] createEventId_ for table:
> functional_hbase.alltypes set to: -1
> ..
> W0203 14:13:06.941573 1978 ReadOnlyZKClient.java:193] 0x65bc7c50 to
> localhost:2181 failed for get of /hbase/hbaseid, code = CONNECTIONLOSS,
> retries = 30, give up
> W0203 14:13:06.947460 24282 ConnectionImplementation.java:641] Retrieve
> cluster id failed
> Java exception follows:
> java.util.concurrent.ExecutionException:
> org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode
> = ConnectionLoss for /hbase/hbaseid
> at
> java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
> at
> java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1908)
> at
> org.apache.hadoop.hbase.client.ConnectionImplementation.retrieveClusterId(ConnectionImplementation.java:639)
> at
> org.apache.hadoop.hbase.client.ConnectionImplementation.(ConnectionImplementation.java:325)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at
> org.apache.hadoop.hbase.client.ConnectionFactory.lambda$createConnection$0(ConnectionFactory.java:231)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1899)
> at
> org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:325)
> at
> org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:230)
> at
> org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:130)
> at
> org.apache.impala.catalog.FeHBaseTable$Util$ConnectionHolder.getConnection(FeHBaseTable.java:722)
> at
> org.apache.impala.catalog.FeHBaseTable$Util.getHBaseTable(FeHBaseTable.java:126)
> at org.apache.impala.catalog.HBaseTable.load(HBaseTable.java:112)
> at org.
