[jira] [Created] (HIVE-27185) Iceberg: Cache iceberg table while loading for stats
Ayush Saxena created HIVE-27185: --- Summary: Iceberg: Cache iceberg table while loading for stats Key: HIVE-27185 URL: https://issues.apache.org/jira/browse/HIVE-27185 Project: Hive Issue Type: Improvement Reporter: Ayush Saxena Assignee: Ayush Saxena Presently iceberg for stats loads the iceberg table multiple times for stats via different routes. Cache it to avoid reading/loading the iceberg table multiple times. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HIVE-27184) Add class name profiling option in ProfileServlet
Rajesh Balamohan created HIVE-27184: --- Summary: Add class name profiling option in ProfileServlet Key: HIVE-27184 URL: https://issues.apache.org/jira/browse/HIVE-27184 Project: Hive Issue Type: Improvement Components: HiveServer2 Reporter: Rajesh Balamohan With async-profiler "-e classame.method", it is possible to profile specific events. Currently profileServlet supports events like cpu, alloc, lock etc. It will be good to enhance to support method name profiling as well. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HIVE-27183) Iceberg: Table information is loaded multiple times
Rajesh Balamohan created HIVE-27183:
---
Summary: Iceberg: Table information is loaded multiple times
Key: HIVE-27183
URL: https://issues.apache.org/jira/browse/HIVE-27183
Project: Hive
Issue Type: Improvement
Reporter: Rajesh Balamohan
HMS::getTable invokes "HiveIcebergMetaHook::postGetTable" which internally
loads iceberg table again.
If this isn't needed or needed only for show-create-table, do not load the
table again.
{noformat}
at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
- parking to wait for <0x00066f84eef0> (a
java.util.concurrent.CompletableFuture$Signaller)
at
java.util.concurrent.locks.LockSupport.park(java.base@11.0.18/LockSupport.java:194)
at
java.util.concurrent.CompletableFuture$Signaller.block(java.base@11.0.18/CompletableFuture.java:1796)
at
java.util.concurrent.ForkJoinPool.managedBlock(java.base@11.0.18/ForkJoinPool.java:3128)
at
java.util.concurrent.CompletableFuture.waitingGet(java.base@11.0.18/CompletableFuture.java:1823)
at
java.util.concurrent.CompletableFuture.get(java.base@11.0.18/CompletableFuture.java:1998)
at org.apache.hadoop.util.functional.FutureIO.awaitFuture(FutureIO.java:77)
at
org.apache.iceberg.hadoop.HadoopInputFile.newStream(HadoopInputFile.java:196)
at org.apache.iceberg.TableMetadataParser.read(TableMetadataParser.java:263)
at org.apache.iceberg.TableMetadataParser.read(TableMetadataParser.java:258)
at
org.apache.iceberg.BaseMetastoreTableOperations.lambda$refreshFromMetadataLocation$0(BaseMetastoreTableOperations.java:177)
at
org.apache.iceberg.BaseMetastoreTableOperations$$Lambda$609/0x000840e18040.apply(Unknown
Source)
at
org.apache.iceberg.BaseMetastoreTableOperations.lambda$refreshFromMetadataLocation$1(BaseMetastoreTableOperations.java:191)
at
org.apache.iceberg.BaseMetastoreTableOperations$$Lambda$610/0x000840e18440.run(Unknown
Source)
at org.apache.iceberg.util.Tasks$Builder.runTaskWithRetry(Tasks.java:404)
at org.apache.iceberg.util.Tasks$Builder.runSingleThreaded(Tasks.java:214)
at org.apache.iceberg.util.Tasks$Builder.run(Tasks.java:198)
at org.apache.iceberg.util.Tasks$Builder.run(Tasks.java:190)
at
org.apache.iceberg.BaseMetastoreTableOperations.refreshFromMetadataLocation(BaseMetastoreTableOperations.java:191)
at
org.apache.iceberg.BaseMetastoreTableOperations.refreshFromMetadataLocation(BaseMetastoreTableOperations.java:176)
at
org.apache.iceberg.BaseMetastoreTableOperations.refreshFromMetadataLocation(BaseMetastoreTableOperations.java:171)
at
org.apache.iceberg.hive.HiveTableOperations.doRefresh(HiveTableOperations.java:153)
at
org.apache.iceberg.BaseMetastoreTableOperations.refresh(BaseMetastoreTableOperations.java:96)
at
org.apache.iceberg.BaseMetastoreTableOperations.current(BaseMetastoreTableOperations.java:79)
at
org.apache.iceberg.BaseMetastoreCatalog.loadTable(BaseMetastoreCatalog.java:44)
at org.apache.iceberg.mr.Catalogs.loadTable(Catalogs.java:115)
at org.apache.iceberg.mr.Catalogs.loadTable(Catalogs.java:105)
at
org.apache.iceberg.mr.hive.IcebergTableUtil.lambda$getTable$1(IcebergTableUtil.java:99)
at
org.apache.iceberg.mr.hive.IcebergTableUtil$$Lambda$552/0x000840d59840.apply(Unknown
Source)
at
org.apache.iceberg.mr.hive.IcebergTableUtil.lambda$getTable$4(IcebergTableUtil.java:111)
at
org.apache.iceberg.mr.hive.IcebergTableUtil$$Lambda$557/0x000840d58c40.get(Unknown
Source)
at java.util.Optional.orElseGet(java.base@11.0.18/Optional.java:369)
at
org.apache.iceberg.mr.hive.IcebergTableUtil.getTable(IcebergTableUtil.java:108)
at
org.apache.iceberg.mr.hive.IcebergTableUtil.getTable(IcebergTableUtil.java:69)
at
org.apache.iceberg.mr.hive.IcebergTableUtil.getTable(IcebergTableUtil.java:73)
at
org.apache.iceberg.mr.hive.HiveIcebergMetaHook.postGetTable(HiveIcebergMetaHook.java:931)
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.executePostGetTableHook(HiveMetaStoreClient.java:2638)
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTable(HiveMetaStoreClient.java:2624)
at
org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.getTable(SessionHiveMetaStoreClient.java:267)
at jdk.internal.reflect.GeneratedMethodAccessor137.invoke(Unknown Source)
at
jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@11.0.18/DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(java.base@11.0.18/Method.java:566)
at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:216)
at com.sun.proxy.$Proxy56.getTable(Unknown Source)
at jdk.internal.reflect.GeneratedMethodAccessor137.invoke(Unknown Source)
at
jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@11.0.18/DelegatingMetho
[jira] [Created] (HIVE-27182) tez_union_with_udf.q with TestMiniTezCliDriver is flaky
Ayush Saxena created HIVE-27182:
---
Summary: tez_union_with_udf.q with TestMiniTezCliDriver is flaky
Key: HIVE-27182
URL: https://issues.apache.org/jira/browse/HIVE-27182
Project: Hive
Issue Type: Improvement
Reporter: Ayush Saxena
Looks like memory issue:
{noformat}
< Caused by: org.apache.hive.com.esotericsoftware.kryo.KryoException:
java.lang.OutOfMemoryError: GC overhead limit exceeded
< Serialization trace:
< genericUDF (org.apache.hadoop.hive.ql.plan.ExprNodeGenericFuncDesc)
< colExprMap (org.apache.hadoop.hive.ql.plan.SelectDesc)
< conf (org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator)
< childOperators (org.apache.hadoop.hive.ql.exec.vector.VectorLimitOperator)
< childOperators (org.apache.hadoop.hive.ql.exec.TableScanOperator)
{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (HIVE-27181) Remove RegexSerDe from hive-contrib, Upgrade should update changed FQN for RegexSerDe in HMS DB
Riju Trivedi created HIVE-27181: --- Summary: Remove RegexSerDe from hive-contrib, Upgrade should update changed FQN for RegexSerDe in HMS DB Key: HIVE-27181 URL: https://issues.apache.org/jira/browse/HIVE-27181 Project: Hive Issue Type: Sub-task Components: Hive Reporter: Riju Trivedi -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HIVE-27180) Remove JsonSerde from hcatalog, Upgrade should update changed FQN for JsonSerDe in HMS DB
Riju Trivedi created HIVE-27180: --- Summary: Remove JsonSerde from hcatalog, Upgrade should update changed FQN for JsonSerDe in HMS DB Key: HIVE-27180 URL: https://issues.apache.org/jira/browse/HIVE-27180 Project: Hive Issue Type: Sub-task Components: Hive Reporter: Riju Trivedi Assignee: Riju Trivedi -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HIVE-27179) HS2 WebUI throws NPE when JspFactory loaded from jetty-runner
Zhihua Deng created HIVE-27179:
--
Summary: HS2 WebUI throws NPE when JspFactory loaded from
jetty-runner
Key: HIVE-27179
URL: https://issues.apache.org/jira/browse/HIVE-27179
Project: Hive
Issue Type: Bug
Components: HiveServer2
Reporter: Zhihua Deng
In HIVE-17088{*},{*} we resolved a NPE thrown from HS2 WebUI by introducing
javax.servlet.jsp-api. It works as expected when the javax.servlet.jsp-api jar
prevails jetty-runner jar, but things can be different in some environments, it
still throws NPE when opening the HS2 web:
{noformat}
java.lang.NullPointerException at
org.apache.hive.generated.hiveserver2.hiveserver2_jsp._jspService(hiveserver2_jsp.java:286)
at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:71) at
javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at
org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1443)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:791) at
org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1626)
...{noformat}
The jetty-runner JspFactory.getDefaultFactory() just returns null.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (HIVE-27178) Backport of HIVE-23321 to branch-3
Aman Raj created HIVE-27178: --- Summary: Backport of HIVE-23321 to branch-3 Key: HIVE-27178 URL: https://issues.apache.org/jira/browse/HIVE-27178 Project: Hive Issue Type: Sub-task Reporter: Aman Raj Assignee: Aman Raj Current branch-3 fails with the diff in select count(*) from skewed_string_list and select count(*) from skewed_string_list_values. Jenkins run : [jenkins / hive-precommit / PR-4156 / #1 (apache.org)|http://ci.hive.apache.org/blue/organizations/jenkins/hive-precommit/detail/PR-4156/1/tests/] Diff : Client Execution succeeded but contained differences (error code = 1) after executing sysdb.q 3740d3739 < hdfs://### HDFS PATH ### default public ROLE 4036c4035 < 3 --- > 6 4045c4044 < 3 --- > 6 This ticket tries to fix this diff. Please read the description of this ticket for the exact reason. -- This message was sent by Atlassian Jira (v8.20.10#820010)
