[jira] [Commented] (IMPALA-12901) Add table property to inject delay in event processing
[ https://issues.apache.org/jira/browse/IMPALA-12901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17828605#comment-17828605 ] Sai Hemanth Gantasala commented on IMPALA-12901: I think this feature is addressed as a part of this fix [https://gerrit.cloudera.org/c/21107/.|https://gerrit.cloudera.org/c/21107/] Can this be closed? > Add table property to inject delay in event processing > -- > > Key: IMPALA-12901 > URL: https://issues.apache.org/jira/browse/IMPALA-12901 > Project: IMPALA > Issue Type: Test > Components: Catalog >Reporter: Quanlong Huang >Priority: Major > > We have tests that verify the behaviors during event processing. We use > global debug action like > "--debug_actions=catalogd_event_processing_delay:SLEEP@2000" to inject the > delay. > It'd be helpful to add a table property for the same which only affects > processing events on that table. So we can control the delay more > specifically. > This is pointed by [~csringhofer] during the review of > https://gerrit.cloudera.org/c/20986/8/tests/custom_cluster/test_web_pages.py#444 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-12924) Upgrade xmlsec to address CVE in 2.2.3
[ https://issues.apache.org/jira/browse/IMPALA-12924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Smith resolved IMPALA-12924. Fix Version/s: Impala 4.4.0 Resolution: Fixed > Upgrade xmlsec to address CVE in 2.2.3 > -- > > Key: IMPALA-12924 > URL: https://issues.apache.org/jira/browse/IMPALA-12924 > Project: IMPALA > Issue Type: Task > Components: Frontend >Affects Versions: Impala 4.3.0 >Reporter: Michael Smith >Assignee: Michael Smith >Priority: Major > Fix For: Impala 4.4.0 > > > Upgrade xmlsec to 2.2.6 to address > https://nvd.nist.gov/vuln/detail/CVE-2023-44483. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-12426) SQL Interface to Completed Queries/DDLs/DMLs
[ https://issues.apache.org/jira/browse/IMPALA-12426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17828531#comment-17828531 ] ASF subversion and git services commented on IMPALA-12426: -- Commit 711a9f2bad84f92dc4af61d49ae115f0dc4239da in impala's branch refs/heads/master from jasonmfehr [ https://gitbox.apache.org/repos/asf?p=impala.git;h=711a9f2ba ] IMPALA-12426: Query History Table Adds the ability for users to specify that Impala will create and maintain an internal Iceberg table that contains data about all completed queries. This table is automatically created at startup by each coordinator if it does not exist. Then, most completed queries are queued in memory and flushed to the query history table at a set interval (either minutes or number of records). Set, use, and show queries are not written to this table. This commit leverages the InternalServer class to maintain the query history table. Ctest unit tests have been added to assert the various pieces of code. New custom cluster tests have been added to assert the query history table is properly populated with completed queries. Negative testing consists of attempting sql injection attacks and syntactically incorrect queries. Impala built-in string functions benchmarks have been updated to include the new built-in functions. Change-Id: I2d2da9d450fba4e789400cfa62927fc25d34f844 Reviewed-on: http://gerrit.cloudera.org:8080/20770 Reviewed-by: Riza Suminto Reviewed-by: Michael Smith Tested-by: Impala Public Jenkins > SQL Interface to Completed Queries/DDLs/DMLs > > > Key: IMPALA-12426 > URL: https://issues.apache.org/jira/browse/IMPALA-12426 > Project: IMPALA > Issue Type: New Feature > Components: Backend, be >Reporter: Jason Fehr >Assignee: Jason Fehr >Priority: Major > Labels: impala, workload-management > > Implement a way of querying (via SQL) information about completed > queries/ddls/dmls. Adds coordinator startup flags for users to specify that > Impala will track completed queries in an internal table. > Impala will create and maintain an internal Iceberg table named > "impala_query_log" in the "system database" that contains all completed > queries. This table is automatically created at startup by each coordinator > if it does not exist. Then, each completed query is queued in memory and > flushed to the query history table either at a set interval (user specified > number of minutes) or when a user specified number of completed queries are > queued in memory. Partition this table by the hour of the query end time. > Data in this table must match the corresponding data in the query profile. > Develop automated testing that asserts this requirement is true. > Don't write use, show, and set queries to this table. > Add the following metrics to the "impala-server" metrics group: > * Number of completed queries queued in memory waiting to be written to the > table. > * Number of completed queries successfully written to the table. > * Number of attempts that failed to write completed queries to the table. > * Number of times completed queries were written at the regularly scheduled > time. > * Number of times completed queries were written before the scheduled time > because the max number of queued records was reached. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Assigned] (IMPALA-11735) Handle CREATE_TABLE event when the db is invisible to the impala server user
[
https://issues.apache.org/jira/browse/IMPALA-11735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sai Hemanth Gantasala reassigned IMPALA-11735:
--
Assignee: Sai Hemanth Gantasala
> Handle CREATE_TABLE event when the db is invisible to the impala server user
>
>
> Key: IMPALA-11735
> URL: https://issues.apache.org/jira/browse/IMPALA-11735
> Project: IMPALA
> Issue Type: Bug
>Reporter: Quanlong Huang
>Assignee: Sai Hemanth Gantasala
>Priority: Critical
>
> It's possible that some dbs are invisible to Impala cluster due to
> authorization restrictions. However, the CREATE_TABLE events in such dbs will
> lead the event-processor into ERROR state:
> {noformat}
> E1026 03:02:30.650302 116774 MetastoreEventsProcessor.java:684] Unexpected
> exception received while processing event
> Java exception follows:
> org.apache.impala.catalog.events.MetastoreNotificationException: EventId:
> 184240416 EventType: CREATE_TABLE Unable to process event
> at
> org.apache.impala.catalog.events.MetastoreEvents$CreateTableEvent.process(MetastoreEvents.java:735)
> at
> org.apache.impala.catalog.events.MetastoreEvents$MetastoreEvent.processIfEnabled(MetastoreEvents.java:345)
> at
> org.apache.impala.catalog.events.MetastoreEventsProcessor.processEvents(MetastoreEventsProcessor.java:772)
> at
> org.apache.impala.catalog.events.MetastoreEventsProcessor.processEvents(MetastoreEventsProcessor.java:670)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> E1026 03:02:30.650447 116774 MetastoreEventsProcessor.java:795] Notification
> event is null
> {noformat}
> It should be handled (e.g. ignored) and reported to the admin (e.g. in logs).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Assigned] (IMPALA-12893) Upgrade to Iceberg 1.4.3
[ https://issues.apache.org/jira/browse/IMPALA-12893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltán Borók-Nagy reassigned IMPALA-12893: -- Assignee: Zoltán Borók-Nagy > Upgrade to Iceberg 1.4.3 > > > Key: IMPALA-12893 > URL: https://issues.apache.org/jira/browse/IMPALA-12893 > Project: IMPALA > Issue Type: Bug > Components: Frontend >Reporter: Zoltán Borók-Nagy >Assignee: Zoltán Borók-Nagy >Priority: Major > Labels: impala-iceberg > > Upgrade our Iceberg dependency to Iceberg 1.4.3. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12924) Upgrade xmlsec to address CVE in 2.2.3
[ https://issues.apache.org/jira/browse/IMPALA-12924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Smith updated IMPALA-12924: --- Description: Upgrade xmlsec to 2.2.6 to address https://nvd.nist.gov/vuln/detail/CVE-2023-44483. (was: Upgrade xmlsec to address https://nvd.nist.gov/vuln/detail/CVE-2023-44483.) > Upgrade xmlsec to address CVE in 2.2.3 > -- > > Key: IMPALA-12924 > URL: https://issues.apache.org/jira/browse/IMPALA-12924 > Project: IMPALA > Issue Type: Task > Components: Frontend >Affects Versions: Impala 4.3.0 >Reporter: Michael Smith >Assignee: Michael Smith >Priority: Major > > Upgrade xmlsec to 2.2.6 to address > https://nvd.nist.gov/vuln/detail/CVE-2023-44483. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-12924) Upgrade xmlsec to address CVE in 2.2.3
Michael Smith created IMPALA-12924: -- Summary: Upgrade xmlsec to address CVE in 2.2.3 Key: IMPALA-12924 URL: https://issues.apache.org/jira/browse/IMPALA-12924 Project: IMPALA Issue Type: Task Components: Frontend Affects Versions: Impala 4.3.0 Reporter: Michael Smith Upgrade xmlsec to address https://nvd.nist.gov/vuln/detail/CVE-2023-44483. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Work started] (IMPALA-12924) Upgrade xmlsec to address CVE in 2.2.3
[ https://issues.apache.org/jira/browse/IMPALA-12924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on IMPALA-12924 started by Michael Smith. -- > Upgrade xmlsec to address CVE in 2.2.3 > -- > > Key: IMPALA-12924 > URL: https://issues.apache.org/jira/browse/IMPALA-12924 > Project: IMPALA > Issue Type: Task > Components: Frontend >Affects Versions: Impala 4.3.0 >Reporter: Michael Smith >Assignee: Michael Smith >Priority: Major > > Upgrade xmlsec to address https://nvd.nist.gov/vuln/detail/CVE-2023-44483. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-11527) SUPPORT user-defined table functions (UDTFs)
[ https://issues.apache.org/jira/browse/IMPALA-11527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17828455#comment-17828455 ] Manish Maheshwari commented on IMPALA-11527: Hi [~gaoxiaoqing] - Checking in, Are you planning to work on this feature? > SUPPORT user-defined table functions (UDTFs) > > > Key: IMPALA-11527 > URL: https://issues.apache.org/jira/browse/IMPALA-11527 > Project: IMPALA > Issue Type: New Feature >Reporter: gaoxiaoqing >Assignee: gaoxiaoqing >Priority: Major > > Currently, Impala does not support user-defined table functions (UDTFs), some > Data Analysis Scenario need this feature such as funnel analysis, session > analysis and so on. Other database Hive, Max compute support udtf. for > example explode in hive: > https://github.com/apache/hive/blob/master/ql/src/test/results/clientpositive/llap/udtf_explode.q.out > Do we have plan to support impala udtf? The syntax of query is as follows, > select udtf_explode(info) as (name, phone) from table; > Besides, we can also support the following syntax, > select udtf_example(c1, c2, c3) over (partition by c1 order by c2) as (num, > c1, c2) from table; > Users can create udtf, the create grammar as follows: > create transform function udtf_example(BIGINT, STRING, STRING, ...) > location '/tmp/libimpalaExts.so' prepare_fn='' init_fn='' update_fn='' > finalize_fn='' close_fn='' -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-12818) Implement the initial framework for caching tuples
[ https://issues.apache.org/jira/browse/IMPALA-12818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe McDonnell resolved IMPALA-12818. Fix Version/s: Impala 4.4.0 Assignee: Joe McDonnell Resolution: Fixed > Implement the initial framework for caching tuples > -- > > Key: IMPALA-12818 > URL: https://issues.apache.org/jira/browse/IMPALA-12818 > Project: IMPALA > Issue Type: Task > Components: Backend, Frontend >Affects Versions: Impala 4.3.0 >Reporter: Joe McDonnell >Assignee: Joe McDonnell >Priority: Major > Fix For: Impala 4.4.0 > > > This tracks introducing the basic concepts of tuple caching. Specifically, it > should introduce the TupleCacheNode, which is an ExecNode that can be placed > in a plan to cache tuples between existing nodes in the plan. The > TupleCachePlanner should place these nodes in locations that are eligible. > The TupleCachePlanner is responsible for eligibility as well as producing a > cache key. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-12883) Add support for changing the charge for a cache entry
[ https://issues.apache.org/jira/browse/IMPALA-12883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe McDonnell resolved IMPALA-12883. Fix Version/s: Impala 4.4.0 Assignee: Joe McDonnell Resolution: Fixed > Add support for changing the charge for a cache entry > - > > Key: IMPALA-12883 > URL: https://issues.apache.org/jira/browse/IMPALA-12883 > Project: IMPALA > Issue Type: Task > Components: Backend >Affects Versions: Impala 4.4.0 >Reporter: Joe McDonnell >Assignee: Joe McDonnell >Priority: Major > Fix For: Impala 4.4.0 > > > The Cache implementation in be/src/util/cache currently does not support > modifying the charge of a cache entry after it has been created. For cases > where the size is known up front, this is fine. For example, the data cache > knows the number of bytes it will consume before it creates the cache entry. > This is a problem for caches that may not know the size of an entry up front. > For example, the tuple cache may want to create a cache entry immediately to > avoid concurrency issues, but then it would want to update that entry's > charge as the entry is finalized (or reaches certain size increments). > It would also be useful to expose the maximum charge allowed for a cache > entry. This would allow writers to avoid creating a cache entry that is too > large. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-12883) Add support for changing the charge for a cache entry
[ https://issues.apache.org/jira/browse/IMPALA-12883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17828454#comment-17828454 ] ASF subversion and git services commented on IMPALA-12883: -- Commit 2f7116173071082ea9cb28512894dc1fa537e9fe in impala's branch refs/heads/master from Michael Smith [ https://gitbox.apache.org/repos/asf?p=impala.git;h=2f7116173 ] IMPALA-12883: Support updating the charge on an entry in the cache The cache implementation currently assumes that a cache entry's charge will remain constant over time and does not support updating the charge. This works well for existing cache users like the data cache and codegen cache. However, it doesn't work as well for use cases where the final size is not known up front. Being able to update the charge also allows for avoiding concurrency issues where two different threads are trying to insert the same entry. This adds the ability to update an entry's charge after it has been inserted into the cache. This can trigger evictions if the size increases. This also adds a way to retrieve the maximum charge supported by the cache. This allows a cache user to tune its behavior to avoid generating entries that would exceed the maximum charge. Testing: - Added tests cases to the caching backend tests Change-Id: I34f54fb3a91a77821651c25d8d3bc3a2a3945025 Reviewed-on: http://gerrit.cloudera.org:8080/21122 Reviewed-by: Michael Smith Tested-by: Impala Public Jenkins Reviewed-by: Yida Wu > Add support for changing the charge for a cache entry > - > > Key: IMPALA-12883 > URL: https://issues.apache.org/jira/browse/IMPALA-12883 > Project: IMPALA > Issue Type: Task > Components: Backend >Affects Versions: Impala 4.4.0 >Reporter: Joe McDonnell >Priority: Major > > The Cache implementation in be/src/util/cache currently does not support > modifying the charge of a cache entry after it has been created. For cases > where the size is known up front, this is fine. For example, the data cache > knows the number of bytes it will consume before it creates the cache entry. > This is a problem for caches that may not know the size of an entry up front. > For example, the tuple cache may want to create a cache entry immediately to > avoid concurrency issues, but then it would want to update that entry's > charge as the entry is finalized (or reaches certain size increments). > It would also be useful to expose the maximum charge allowed for a cache > entry. This would allow writers to avoid creating a cache entry that is too > large. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-12899) Temporary workaround for BINARY in complex types
[ https://issues.apache.org/jira/browse/IMPALA-12899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17828387#comment-17828387 ] Csaba Ringhofer commented on IMPALA-12899: -- base64 encoding seems a sane and widely used approach to me. I would suggest the following: # implement it first with base64 encoding # if there is demand to handle this differently, a query option like binary_column_encoding_in_json=base64 / skip / hive_style_unquoted_string I would avoid a "lossy" solution as default, so one where the original binary value can't be decoded from the output. > Temporary workaround for BINARY in complex types > > > Key: IMPALA-12899 > URL: https://issues.apache.org/jira/browse/IMPALA-12899 > Project: IMPALA > Issue Type: Sub-task >Reporter: Daniel Becker >Assignee: Daniel Becker >Priority: Major > > The BINARY type is currently not supported inside complex types and a > cross-component decision is probably needed to support it (see IMPALA-11491). > We would like to enable EXPAND_COMPLEX_TYPES for Iceberg metadata tables > (IMPALA-12612), which requires that queries with BINARY inside complex types > don't fail. Enabling EXPAND_COMPLEX_TYPES is a more prioritised issue than > IMPALA-11491, so we should come up with a temporary solution, e.g. NULLing > BINARY values in complex types and logging a warning, or setting these BINARY > values to a warning string. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Work started] (IMPALA-12923) Fix header alignment during horizontal query timeline scrolling
[ https://issues.apache.org/jira/browse/IMPALA-12923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on IMPALA-12923 started by Surya Hebbar. - > Fix header alignment during horizontal query timeline scrolling > > > Key: IMPALA-12923 > URL: https://issues.apache.org/jira/browse/IMPALA-12923 > Project: IMPALA > Issue Type: Improvement >Reporter: Surya Hebbar >Assignee: Surya Hebbar >Priority: Minor > > When query timeline is scrolled horizontally after zooming in, the controls > and navigation bar at the top of the page move away from the visibility area. > This makes it difficult to zoom while scrolling. > > Although, this issue does not arise, if Ctrl+Scroll wheel is used for zooming. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Assigned] (IMPALA-12879) Conjunct not referring to table field causes ERROR for Iceberg table
[
https://issues.apache.org/jira/browse/IMPALA-12879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Noemi Pap-Takacs reassigned IMPALA-12879:
-
Assignee: (was: Noemi Pap-Takacs)
> Conjunct not referring to table field causes ERROR for Iceberg table
>
>
> Key: IMPALA-12879
> URL: https://issues.apache.org/jira/browse/IMPALA-12879
> Project: IMPALA
> Issue Type: Bug
>Reporter: Zoltán Borók-Nagy
>Priority: Major
> Labels: impala-iceberg, ramp-up
>
> I tried to sample a table with the following query:
> {noformat}
> select * from ice_tbl where rand(unix_timestamp()) < 0.001;{noformat}
> And I got the following error:
> {noformat}
> ERROR: IndexOutOfBoundsException: Index: 0, Size: 0{noformat}
> I caught the exception via JDB, stack trace was:
> {noformat}
> Thread-152931[1] where
> [1] java.util.ArrayList.rangeCheck (ArrayList.java:659)
> [2] java.util.ArrayList.get (ArrayList.java:435)
> [3] org.apache.impala.planner.IcebergScanPlanner.hasPartitionTransformType
> (IcebergScanPlanner.java:813)
> [4] org.apache.impala.planner.IcebergScanPlanner.isPartitionColumnIncluded
> (IcebergScanPlanner.java:795)
> [5] org.apache.impala.planner.IcebergScanPlanner.extractIcebergConjuncts
> (IcebergScanPlanner.java:769)
> [6] org.apache.impala.planner.IcebergScanPlanner.
> (IcebergScanPlanner.java:152)
> [7] org.apache.impala.planner.SingleNodePlanner.createScanNode
> (SingleNodePlanner.java:1,881)
> [8] org.apache.impala.planner.SingleNodePlanner.createTableRefNode
> (SingleNodePlanner.java:2,209)
> [9] org.apache.impala.planner.SingleNodePlanner.createTableRefsPlan
> (SingleNodePlanner.java:933)
> [10] org.apache.impala.planner.SingleNodePlanner.createSelectPlan
> (SingleNodePlanner.java:748)
> [11] org.apache.impala.planner.SingleNodePlanner.createQueryPlan
> (SingleNodePlanner.java:280)
> [12] org.apache.impala.planner.SingleNodePlanner.createSingleNodePlan
> (SingleNodePlanner.java:172)
> [13] org.apache.impala.planner.Planner.createPlanFragments
> (Planner.java:128)
> ...
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-12914) test_banned_log_messages fails if Impala is not running
[
https://issues.apache.org/jira/browse/IMPALA-12914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17828196#comment-17828196
]
ASF subversion and git services commented on IMPALA-12914:
--
Commit caf136fae4951a6eb5216f897d18d313f94ddd83 in impala's branch
refs/heads/master from Michael Smith
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=caf136fae ]
IMPALA-12914: TestBannedLogMessages no longer requires running cluster
Updates test_banned_log_messages.py to no longer require a running
cluster, as it only checks files on disk. This fixes running custom
cluster tests that end with shutting down the cluster via
run-all-tests.sh.
Tested locally with
FE_TEST=false BE_TEST=false EE_TEST=false JDBC_TEST=false\
JS_TEST=false CLUSTER_TEST_FILES=custom_cluster/test_coordinators.py\
::TestCoordinators::test_exclusive_coordinator_plan run-all-tests.sh
Change-Id: Icfa7d5925bf724d714eaa7fa606defae0822cf93
Reviewed-on: http://gerrit.cloudera.org:8080/21152
Reviewed-by: Riza Suminto
Tested-by: Michael Smith
> test_banned_log_messages fails if Impala is not running
> ---
>
> Key: IMPALA-12914
> URL: https://issues.apache.org/jira/browse/IMPALA-12914
> Project: IMPALA
> Issue Type: Task
> Components: Infrastructure
>Reporter: Michael Smith
>Assignee: Michael Smith
>Priority: Major
>
> {{run-all-tests.sh}} fails at the end - in test_banned_log_messages.py - if
> used to test a custom cluster test that finishes with Impala stopped, such as
> {{custom_cluster/test_coordinators.py::TestCoordinators::test_exclusive_coordinator_plan}}.
> The error is
> {code}
> verifiers/test_banned_log_messages.py::TestBannedLogMessages::test_no_inaccessible_objects
> ERROR
> verifiers/test_banned_log_messages.py::TestBannedLogMessages::test_no_unsupported_operations
> ERROR
> generated xml file:
> /data/jenkins/workspace/impala-private-basic-parameterized/repos/Impala/logs/verifiers/TEST-impala-verifiers.xml
>
> === short test summary info
>
> ERROR
> verifiers/test_banned_log_messages.py::TestBannedLogMessages::()::test_no_inaccessible_objects
> ERROR
> verifiers/test_banned_log_messages.py::TestBannedLogMessages::()::test_no_unsupported_operations
> ERRORS
>
> _ ERROR at setup of TestBannedLogMessages.test_no_inaccessible_objects
> _
> common/impala_test_suite.py:226: in setup_class
> cls.create_impala_clients()
> common/impala_test_suite.py:327: in create_impala_clients
> cls.client = cls.create_impala_client(protocol='beeswax')
> common/impala_test_suite.py:310: in create_impala_client
> client.connect()
> common/impala_connection.py:196: in connect
> self.__beeswax_client.connect()
> beeswax/impala_beeswax.py:166: in connect
> raise ImpalaBeeswaxException(self.__build_error_message(e), e)
> E ImpalaBeeswaxException: ImpalaBeeswaxException:
> EINNER EXCEPTION: 'thrift.transport.TTransport.TTransportException'>
> EMESSAGE: Could not connect to any of [('::1', 21000, 0, 0),
> ('127.0.0.1', 21000)]
> Captured stderr setup
> -
> SET
> client_identifier=verifiers/test_banned_log_messages.py::TestBannedLogMessages::()::test_no_inaccessible_objects;
> -- connecting to: localhost:21000
> -- 2024-03-15 12:08:43,506 INFO MainThread: Could not connect to ('::1',
> 21000, 0, 0)
> Traceback (most recent call last):
> File
> "/data0/jenkins/workspace/impala-private-basic-parameterized/repos/Impala/infra/python/env-gcc10.4.0/lib/python2.7/site-packages/thrift/transport/TSocket.py",
> line 137, in open
> handle.connect(sockaddr)
> File
> "/data/jenkins/workspace/impala-private-basic-parameterized/Impala-Toolchain/toolchain-packages-gcc10.4.0/python-2.7.16/lib/python2.7/socket.py",
> line 228, in meth
> return getattr(self._sock,name)(*args)
> error: [Errno 111] Connection refused
> -- 2024-03-15 12:08:43,506 INFO MainThread: Could not connect to
> ('127.0.0.1', 21000)
> Traceback (most recent call last):
> File
> "/data0/jenkins/workspace/impala-private-basic-parameterized/repos/Impala/infra/python/env-gcc10.4.0/lib/python2.7/site-packages/thrift/transport/TSocket.py",
> line 137, in open
> handle.connect(sockaddr)
> File
> "/data/jenkins/workspace/impala-private-basic-parameterized/Impala-Toolchain/toolchain-packages-gcc10.4.0/python-2.7.16/lib/python2.7/socket.py",
> line 228, in meth
> return getattr(self._sock,name)(*args)
> error: [Errno 111] Connection refused
> -- 2024-03-15 12:08:43,506 ERRORMainThread: Could not connect to any of
> [('::1', 21000, 0, 0), ('127.0.0.1', 21000)]
> ERROR at setup of
[jira] [Commented] (IMPALA-12904) test_type_conversions_hive3 silently passes because of wrongly defined test dimensions
[
https://issues.apache.org/jira/browse/IMPALA-12904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17828197#comment-17828197
]
ASF subversion and git services commented on IMPALA-12904:
--
Commit 2c4d84b1e11ae1f8857abebf7e7983e7449ff3e1 in impala's branch
refs/heads/master from Zoltan Borok-Nagy
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=2c4d84b1e ]
IMPALA-12904: test_type_conversions_hive3 silently passes because of wrongly
defined test dimensions
test_type_conversions_hive3 silently passes because we are not creating
the test dimenstion for query option orc_shema_resolution correctly. If
we set orc_shema_resolution correctly, i.e. to also exercise the
name-based schema resolution, the test fails. The cause of the failure
is that the ill-typed tables have dummy column names like 'c1', 'c2',
etc. These are completely fine for position-based schema resolution,
but it is not OK for name-based schema resolution.
The test just wants to check error messages related to type errors,
the column names are irrelevant, so we can just use the correct
names.
Change-Id: I786a5eaae9243b4728484f3f3b1427b20a1d2d28
Reviewed-on: http://gerrit.cloudera.org:8080/21151
Reviewed-by: Impala Public Jenkins
Tested-by: Impala Public Jenkins
> test_type_conversions_hive3 silently passes because of wrongly defined test
> dimensions
> --
>
> Key: IMPALA-12904
> URL: https://issues.apache.org/jira/browse/IMPALA-12904
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Reporter: Zoltán Borók-Nagy
>Assignee: Zoltán Borók-Nagy
>Priority: Major
>
> test_type_conversions_hive3 silently passes because we are not creating the
> test dimenstion for query option orc_shema_resolution correctly.
> Instead of:
>
> {noformat}
> cls.ImpalaTestMatrix.add_dimension(ImpalaTestDimension('orc_schema_resolution',
> 0, 1)){noformat}
> We should do the following:
> {noformat}
> add_exec_option_dimension(cls, 'orc_schema_resolution', [0, 1]){noformat}
> We should fix how we add the test dimension, and also fix the underlying bug
> it reveals.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
