[jira] [Commented] (HIVE-27113) Increasing default for hive.thrift.client.max.message.size to 2 GB
[
https://issues.apache.org/jira/browse/HIVE-27113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17698231#comment-17698231
]
Riju Trivedi commented on HIVE-27113:
-
Thrift message size can go beyond 1g for wide tables with too many partitions
(5k+), Hence increasing the default to max value 2 GB.
> Increasing default for hive.thrift.client.max.message.size to 2 GB
> --
>
> Key: HIVE-27113
> URL: https://issues.apache.org/jira/browse/HIVE-27113
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Riju Trivedi
>Assignee: Riju Trivedi
>Priority: Major
>
> HIVE_THRIFT_CLIENT_MAX_MESSAGE_SIZE("hive.thrift.client.max.message.size",
> "1gb",
> new SizeValidator(-1L, true, (long) Integer.MAX_VALUE, true),
> "Thrift client configuration for max message size. 0 or -1 will use
> the default defined in the Thrift " +
> "library. The upper limit is 2147483648 bytes (or 2gb).")
> Documentation on the help suggests setting 2147483648 while Integer Max is
> 2147483647. So, it actually becomes -1 and gets set to thrift default limit
> (100 MB)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HIVE-27113) Increasing default for hive.thrift.client.max.message.size to 2 GB
[
https://issues.apache.org/jira/browse/HIVE-27113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Riju Trivedi updated HIVE-27113:
Summary: Increasing default for hive.thrift.client.max.message.size to 2 GB
(was: Documentation for hive.thrift.client.max.message.size config needs to be
corrected)
> Increasing default for hive.thrift.client.max.message.size to 2 GB
> --
>
> Key: HIVE-27113
> URL: https://issues.apache.org/jira/browse/HIVE-27113
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Riju Trivedi
>Assignee: Riju Trivedi
>Priority: Major
>
> HIVE_THRIFT_CLIENT_MAX_MESSAGE_SIZE("hive.thrift.client.max.message.size",
> "1gb",
> new SizeValidator(-1L, true, (long) Integer.MAX_VALUE, true),
> "Thrift client configuration for max message size. 0 or -1 will use
> the default defined in the Thrift " +
> "library. The upper limit is 2147483648 bytes (or 2gb).")
> Documentation on the help suggests setting 2147483648 while Integer Max is
> 2147483647. So, it actually becomes -1 and gets set to thrift default limit
> (100 MB)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27129) Enhanced support to Hive Client http support
[
https://issues.apache.org/jira/browse/HIVE-27129?focusedWorklogId=849982&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-849982
]
ASF GitHub Bot logged work on HIVE-27129:
-
Author: ASF GitHub Bot
Created on: 09/Mar/23 05:46
Start Date: 09/Mar/23 05:46
Worklog Time Spent: 10m
Work Description: vihangk1 commented on code in PR #4104:
URL: https://github.com/apache/hive/pull/4104#discussion_r1130492724
##
standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java:
##
@@ -35,25 +35,15 @@
import java.net.UnknownHostException;
import java.nio.ByteBuffer;
import java.security.PrivilegedExceptionAction;
-import java.util.ArrayList;
-import java.util.Arrays;
-import java.util.Collection;
-import java.util.Collections;
-import java.util.HashMap;
-import java.util.Iterator;
-import java.util.LinkedHashMap;
-import java.util.List;
-import java.util.Map;
+import java.util.*;
Review Comment:
We generally don't use wild-card imports. Can you please revert this part
and explicitly add the necessary imports (You may have to change your IDE
settings so that it doesn't do this for automatically).
##
standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java:
##
@@ -621,31 +611,84 @@ private T
configureThriftMaxMessageSize(T transport) {
return transport;
}
+ private Map getAdditionalHeaders() {
+Map headers = new HashMap<>();
+String keyValuePairs = MetastoreConf.getVar(conf,
ConfVars.METASTORE_CLIENT_ADDITIONAL_HEADERS);
+try {
+ List headerKeyValues =
Splitter.on(',').trimResults().splitToList(keyValuePairs);
+ for (String header : headerKeyValues) {
+String[] parts = header.split("=");
+headers.put(parts[0].trim(), parts[1].trim());
+LOG.warn(parts[0].trim() + "=" + parts[1].trim());
Review Comment:
Is this log needed? If not, let's remove it or change it to debug.
##
standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java:
##
@@ -621,31 +611,84 @@ private T
configureThriftMaxMessageSize(T transport) {
return transport;
}
+ private Map getAdditionalHeaders() {
+Map headers = new HashMap<>();
+String keyValuePairs = MetastoreConf.getVar(conf,
ConfVars.METASTORE_CLIENT_ADDITIONAL_HEADERS);
+try {
+ List headerKeyValues =
Splitter.on(',').trimResults().splitToList(keyValuePairs);
+ for (String header : headerKeyValues) {
+String[] parts = header.split("=");
+headers.put(parts[0].trim(), parts[1].trim());
+LOG.warn(parts[0].trim() + "=" + parts[1].trim());
+ }
+} catch (Exception ex) {
+ LOG.warn("Could not parse the headers provided in " +
ConfVars.METASTORE_CLIENT_ADDITIONAL_HEADERS, ex);
+}
+return headers;
+ }
+
/*
Creates a THttpClient if HTTP mode is enabled. If Client auth mode is set to
JWT,
then the method fetches JWT from environment variable: HMS_JWT and sets in
auth
header in http request
*/
- private THttpClient createHttpClient(URI store, boolean useSSL) throws
MetaException,
- TTransportException {
+ private THttpClient createHttpClient(URI store, boolean useSSL) throws
MetaException, TTransportException {
String path = MetaStoreUtils.getHttpPath(MetastoreConf.getVar(conf,
ConfVars.THRIFT_HTTP_PATH));
-String httpUrl = (useSSL ? "https://"; : "http://";) + store.getHost() + ":"
+ store.getPort() + path;
+String urlScheme;
+if (useSSL || Objects.equals(store.getScheme(), "https")) {
+ urlScheme = "https://";;
+} else {
+ urlScheme = "http://";;
+}
+String httpUrl = urlScheme + store.getHost() + ":" + store.getPort() +
path;
+
+HttpClientBuilder httpClientBuilder = createHttpClientBuilder();
+THttpClient tHttpClient;
+try {
+ if (useSSL) {
+String trustStorePath = MetastoreConf.getVar(conf,
ConfVars.SSL_TRUSTSTORE_PATH).trim();
+if (trustStorePath.isEmpty()) {
+ throw new IllegalArgumentException(ConfVars.SSL_TRUSTSTORE_PATH + "
Not configured for SSL connection");
+}
+String trustStorePassword = MetastoreConf.getPassword(conf,
MetastoreConf.ConfVars.SSL_TRUSTSTORE_PASSWORD);
+String trustStoreType = MetastoreConf.getVar(conf,
ConfVars.SSL_TRUSTSTORE_TYPE).trim();
+String trustStoreAlgorithm = MetastoreConf.getVar(conf,
ConfVars.SSL_TRUSTMANAGERFACTORY_ALGORITHM).trim();
+tHttpClient =
+SecurityUtils.getThriftHttpsClient(httpUrl, trustStorePath,
trustStorePassword, trustStoreAlgorithm,
+trustStoreType, httpClientBuilder);
+ } else {
+tHttpClient = new THttpClient(httpUrl, httpClientBuilder.build());
+ }
+} catch (
[jira] [Resolved] (HIVE-27107) Fix improper metrics count for RESUME/RESET workflow
[ https://issues.apache.org/jira/browse/HIVE-27107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shreenidhi resolved HIVE-27107. --- Resolution: Fixed > Fix improper metrics count for RESUME/RESET workflow > > > Key: HIVE-27107 > URL: https://issues.apache.org/jira/browse/HIVE-27107 > Project: Hive > Issue Type: Task >Reporter: Shreenidhi >Assignee: Shreenidhi >Priority: Major > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > *Analysis :* > REPL DUMP during RESET checks all the tables listed in table_diff_complete > and persists the table count in replication metrics. This is incorrect for > RESET, > because table_diff_complete has those tables which needs to be dropped too. > *Fix :* > To fix this, one can take intersection of (tables in table_diff_complete, > actual tables on source) and capture the same into the replication_metrics -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27123) In PRE_OPTIMIZED_BOOTSTRAP operation, Load side stores dump_execution_id as -1
[ https://issues.apache.org/jira/browse/HIVE-27123?focusedWorklogId=849979&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-849979 ] ASF GitHub Bot logged work on HIVE-27123: - Author: ASF GitHub Bot Created on: 09/Mar/23 05:23 Start Date: 09/Mar/23 05:23 Worklog Time Spent: 10m Work Description: pudidic merged PR #4100: URL: https://github.com/apache/hive/pull/4100 Issue Time Tracking --- Worklog Id: (was: 849979) Time Spent: 40m (was: 0.5h) > In PRE_OPTIMIZED_BOOTSTRAP operation, Load side stores dump_execution_id as -1 > -- > > Key: HIVE-27123 > URL: https://issues.apache.org/jira/browse/HIVE-27123 > Project: Hive > Issue Type: Bug >Reporter: Harshal Patel >Assignee: Harshal Patel >Priority: Major > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > For PRE_OPTIMIZED_BOOSTRAP operation, dump side correctly stores execution > ids in replication_metrics but Load side stores dump_execution_id as -1 which > should be = execution_id of DUMP operation -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27107) Fix improper metrics count for RESUME/RESET workflow
[ https://issues.apache.org/jira/browse/HIVE-27107?focusedWorklogId=849977&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-849977 ] ASF GitHub Bot logged work on HIVE-27107: - Author: ASF GitHub Bot Created on: 09/Mar/23 05:22 Start Date: 09/Mar/23 05:22 Worklog Time Spent: 10m Work Description: pudidic commented on PR #4084: URL: https://github.com/apache/hive/pull/4084#issuecomment-1461304559 LGTM +1. Issue Time Tracking --- Worklog Id: (was: 849977) Time Spent: 0.5h (was: 20m) > Fix improper metrics count for RESUME/RESET workflow > > > Key: HIVE-27107 > URL: https://issues.apache.org/jira/browse/HIVE-27107 > Project: Hive > Issue Type: Task >Reporter: Shreenidhi >Assignee: Shreenidhi >Priority: Major > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > > *Analysis :* > REPL DUMP during RESET checks all the tables listed in table_diff_complete > and persists the table count in replication metrics. This is incorrect for > RESET, > because table_diff_complete has those tables which needs to be dropped too. > *Fix :* > To fix this, one can take intersection of (tables in table_diff_complete, > actual tables on source) and capture the same into the replication_metrics -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27107) Fix improper metrics count for RESUME/RESET workflow
[ https://issues.apache.org/jira/browse/HIVE-27107?focusedWorklogId=849978&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-849978 ] ASF GitHub Bot logged work on HIVE-27107: - Author: ASF GitHub Bot Created on: 09/Mar/23 05:22 Start Date: 09/Mar/23 05:22 Worklog Time Spent: 10m Work Description: pudidic merged PR #4084: URL: https://github.com/apache/hive/pull/4084 Issue Time Tracking --- Worklog Id: (was: 849978) Time Spent: 40m (was: 0.5h) > Fix improper metrics count for RESUME/RESET workflow > > > Key: HIVE-27107 > URL: https://issues.apache.org/jira/browse/HIVE-27107 > Project: Hive > Issue Type: Task >Reporter: Shreenidhi >Assignee: Shreenidhi >Priority: Major > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > *Analysis :* > REPL DUMP during RESET checks all the tables listed in table_diff_complete > and persists the table count in replication metrics. This is incorrect for > RESET, > because table_diff_complete has those tables which needs to be dropped too. > *Fix :* > To fix this, one can take intersection of (tables in table_diff_complete, > actual tables on source) and capture the same into the replication_metrics -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27123) In PRE_OPTIMIZED_BOOTSTRAP operation, Load side stores dump_execution_id as -1
[ https://issues.apache.org/jira/browse/HIVE-27123?focusedWorklogId=849976&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-849976 ] ASF GitHub Bot logged work on HIVE-27123: - Author: ASF GitHub Bot Created on: 09/Mar/23 05:21 Start Date: 09/Mar/23 05:21 Worklog Time Spent: 10m Work Description: pudidic commented on PR #4100: URL: https://github.com/apache/hive/pull/4100#issuecomment-1461303691 +1. LGTM. Issue Time Tracking --- Worklog Id: (was: 849976) Time Spent: 0.5h (was: 20m) > In PRE_OPTIMIZED_BOOTSTRAP operation, Load side stores dump_execution_id as -1 > -- > > Key: HIVE-27123 > URL: https://issues.apache.org/jira/browse/HIVE-27123 > Project: Hive > Issue Type: Bug >Reporter: Harshal Patel >Assignee: Harshal Patel >Priority: Major > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > > For PRE_OPTIMIZED_BOOSTRAP operation, dump side correctly stores execution > ids in replication_metrics but Load side stores dump_execution_id as -1 which > should be = execution_id of DUMP operation -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HIVE-26919) Fix-test-case-TestReplicationOptimisedBootstrap.testReverseFailoverBeforeOptimizedBootstrap
[ https://issues.apache.org/jira/browse/HIVE-26919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Saonerkar resolved HIVE-26919. --- Resolution: Fixed > Fix-test-case-TestReplicationOptimisedBootstrap.testReverseFailoverBeforeOptimizedBootstrap > --- > > Key: HIVE-26919 > URL: https://issues.apache.org/jira/browse/HIVE-26919 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: Amit Saonerkar >Assignee: Amit Saonerkar >Priority: Minor > > This Jira is related to test case failure corresponding to Jira CDPD-48053 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HIVE-26919) Fix-test-case-TestReplicationOptimisedBootstrap.testReverseFailoverBeforeOptimizedBootstrap
[ https://issues.apache.org/jira/browse/HIVE-26919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17698179#comment-17698179 ] Amit Saonerkar commented on HIVE-26919: --- https://github.com/apache/hive/commit/0c059c6e9bba35f353cd3212c5c01e86eba51125 > Fix-test-case-TestReplicationOptimisedBootstrap.testReverseFailoverBeforeOptimizedBootstrap > --- > > Key: HIVE-26919 > URL: https://issues.apache.org/jira/browse/HIVE-26919 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: Amit Saonerkar >Assignee: Amit Saonerkar >Priority: Minor > > This Jira is related to test case failure corresponding to Jira CDPD-48053 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HIVE-26920) Add-new-view-in-sys-db-to-capture-failover-and-failback-metrics
[ https://issues.apache.org/jira/browse/HIVE-26920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Saonerkar resolved HIVE-26920. --- Resolution: Fixed > Add-new-view-in-sys-db-to-capture-failover-and-failback-metrics > --- > > Key: HIVE-26920 > URL: https://issues.apache.org/jira/browse/HIVE-26920 > Project: Hive > Issue Type: New Feature > Components: HiveServer2 >Reporter: Amit Saonerkar >Assignee: Amit Saonerkar >Priority: Major > Labels: pull-request-available > Time Spent: 1h > Remaining Estimate: 0h > > Corresponding to CDPD-46702 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HIVE-26921) Add failover_type, failover_endpoint to replication metrics metadata
[ https://issues.apache.org/jira/browse/HIVE-26921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Saonerkar resolved HIVE-26921. --- Resolution: Fixed > Add failover_type, failover_endpoint to replication metrics metadata > > > Key: HIVE-26921 > URL: https://issues.apache.org/jira/browse/HIVE-26921 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Reporter: Amit Saonerkar >Assignee: Amit Saonerkar >Priority: Major > Labels: pull-request-available > Time Spent: 1.5h > Remaining Estimate: 0h > > Corresponding to CDPD-46494 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27129) Enhanced support to Hive Client http support
[ https://issues.apache.org/jira/browse/HIVE-27129?focusedWorklogId=849975&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-849975 ] ASF GitHub Bot logged work on HIVE-27129: - Author: ASF GitHub Bot Created on: 09/Mar/23 05:08 Start Date: 09/Mar/23 05:08 Worklog Time Spent: 10m Work Description: junlinzeng-db opened a new pull request, #4104: URL: https://github.com/apache/hive/pull/4104 ### What changes were proposed in this pull request? ### Why are the changes needed? ### Does this PR introduce _any_ user-facing change? This PR targets to improve the usability of the http hive metastore client, by adding support for custom headers and also, supporting default trust store for https. 1. Add a new hive conf to allow users to inject custom headers 2. Do not force the headers to be http when the useSSL is turned off, so that if the users are able to use https if they want, w/o a custom truststore. ### How was this patch tested? ut for the http headers Issue Time Tracking --- Worklog Id: (was: 849975) Remaining Estimate: 0h Time Spent: 10m > Enhanced support to Hive Client http support > > > Key: HIVE-27129 > URL: https://issues.apache.org/jira/browse/HIVE-27129 > Project: Hive > Issue Type: Improvement >Reporter: Junlin Zeng >Assignee: Junlin Zeng >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > Currently we support using http in the hive metastore connection. However, we > do not support custom headers and also default trust store. This ticket > tracks the work to improve the http journey. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27129) Enhanced support to Hive Client http support
[ https://issues.apache.org/jira/browse/HIVE-27129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-27129: -- Labels: pull-request-available (was: ) > Enhanced support to Hive Client http support > > > Key: HIVE-27129 > URL: https://issues.apache.org/jira/browse/HIVE-27129 > Project: Hive > Issue Type: Improvement >Reporter: Junlin Zeng >Assignee: Junlin Zeng >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > Currently we support using http in the hive metastore connection. However, we > do not support custom headers and also default trust store. This ticket > tracks the work to improve the http journey. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27087) Fix TestMiniSparkOnYarnCliDriver test failures on branch-3
[ https://issues.apache.org/jira/browse/HIVE-27087?focusedWorklogId=849966&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-849966 ] ASF GitHub Bot logged work on HIVE-27087: - Author: ASF GitHub Bot Created on: 09/Mar/23 03:57 Start Date: 09/Mar/23 03:57 Worklog Time Spent: 10m Work Description: vihangk1 commented on PR #4067: URL: https://github.com/apache/hive/pull/4067#issuecomment-1461235075 @zabetak I am running out of ideas here. Any idea why the replication factor seen in the q.out is 1 instead of 3. I cannot reproduce it on my local. All the q.out file diff failures work for me on my local. Issue Time Tracking --- Worklog Id: (was: 849966) Time Spent: 2h 40m (was: 2.5h) > Fix TestMiniSparkOnYarnCliDriver test failures on branch-3 > -- > > Key: HIVE-27087 > URL: https://issues.apache.org/jira/browse/HIVE-27087 > Project: Hive > Issue Type: Sub-task >Reporter: Vihang Karajgaonkar >Assignee: Vihang Karajgaonkar >Priority: Major > Labels: pull-request-available > Time Spent: 2h 40m > Remaining Estimate: 0h > > TestMiniSparkOnYarnCliDriver are failing with the error below > [ERROR] 2023-02-16 14:13:08.991 [Driver] SparkContext - Error initializing > SparkContext. > java.lang.RuntimeException: java.lang.NoSuchFieldException: > DEFAULT_TINY_CACHE_SIZE > at > org.apache.spark.network.util.NettyUtils.getPrivateStaticField(NettyUtils.java:131) > ~[spark-network-common_2.11-2.3.0.jar:2.3.0] > at > org.apache.spark.network.util.NettyUtils.createPooledByteBufAllocator(NettyUtils.java:118) > ~[spark-network-common_2.11-2.3.0.jar:2.3.0] > at > org.apache.spark.network.server.TransportServer.init(TransportServer.java:94) > ~[spark-network-common_2.11-2.3.0.jar:2.3.0] > at > org.apache.spark.network.server.TransportServer.(TransportServer.java:73) > ~[spark-network-common_2.11-2.3.0.jar:2.3.0] > at > org.apache.spark.network.TransportContext.createServer(TransportContext.java:114) > ~[spark-network-common_2.11-2.3.0.jar:2.3.0] > at org.apache.spark.rpc.netty.NettyRpcEnv.startServer(NettyRpcEnv.scala:119) > ~[spark-core_2.11-2.3.0.jar:2.3.0] > at > org.apache.spark.rpc.netty.NettyRpcEnvFactory$$anonfun$4.apply(NettyRpcEnv.scala:465) > ~[spark-core_2.11-2.3.0.jar:2.3.0] > at > org.apache.spark.rpc.netty.NettyRpcEnvFactory$$anonfun$4.apply(NettyRpcEnv.scala:464) > ~[spark-core_2.11-2.3.0.jar:2.3.0] > at > org.apache.spark.util.Utils$$anonfun$startServiceOnPort$1.apply$mcVI$sp(Utils.scala:2271) > ~[spark-core_2.11-2.3.0.jar:2.3.0] > at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:160) > ~[scala-library-2.11.8.jar:?] > at org.apache.spark.util.Utils$.startServiceOnPort(Utils.scala:2263) > ~[spark-core_2.11-2.3.0.jar:2.3.0] > at > org.apache.spark.rpc.netty.NettyRpcEnvFactory.create(NettyRpcEnv.scala:469) > ~[spark-core_2.11-2.3.0.jar:2.3.0] > at org.apache.spark.rpc.RpcEnv$.create(RpcEnv.scala:57) > ~[spark-core_2.11-2.3.0.jar:2.3.0] > at org.apache.spark.SparkEnv$.create(SparkEnv.scala:249) > ~[spark-core_2.11-2.3.0.jar:2.3.0] > at org.apache.spark.SparkEnv$.createDriverEnv(SparkEnv.scala:175) > ~[spark-core_2.11-2.3.0.jar:2.3.0] > at org.apache.spark.SparkContext.createSparkEnv(SparkContext.scala:256) > [spark-core_2.11-2.3.0.jar:2.3.0] > at org.apache.spark.SparkContext.(SparkContext.scala:423) > [spark-core_2.11-2.3.0.jar:2.3.0] > at > org.apache.spark.api.java.JavaSparkContext.(JavaSparkContext.scala:58) > [spark-core_2.11-2.3.0.jar:2.3.0] > at org.apache.hive.spark.client.RemoteDriver.(RemoteDriver.java:161) > [hive-exec-3.2.0-SNAPSHOT.jar:3.2.0-SNAPSHOT] > at org.apache.hive.spark.client.RemoteDriver.main(RemoteDriver.java:536) > [hive-exec-3.2.0-SNAPSHOT.jar:3.2.0-SNAPSHOT] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_322] > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > ~[?:1.8.0_322] > The root cause of the problem is that we upgrade the netty library from > 4.1.17.Final to 4.1.69.Final. The upgraded library does not have > `DEFAULT_TINY_CACHE_SIZE` field > [here|https://github.com/netty/netty/blob/netty-4.1.51.Final/buffer/src/main/java/io/netty/buffer/PooledByteBufAllocator.java#L46] > which was removed in 4.1.52.Final -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27087) Fix TestMiniSparkOnYarnCliDriver test failures on branch-3
[ https://issues.apache.org/jira/browse/HIVE-27087?focusedWorklogId=849965&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-849965 ] ASF GitHub Bot logged work on HIVE-27087: - Author: ASF GitHub Bot Created on: 09/Mar/23 03:51 Start Date: 09/Mar/23 03:51 Worklog Time Spent: 10m Work Description: amanraj2520 commented on PR #4067: URL: https://github.com/apache/hive/pull/4067#issuecomment-1461230107 I cannot repro in my local as well. Checking still. Issue Time Tracking --- Worklog Id: (was: 849965) Time Spent: 2.5h (was: 2h 20m) > Fix TestMiniSparkOnYarnCliDriver test failures on branch-3 > -- > > Key: HIVE-27087 > URL: https://issues.apache.org/jira/browse/HIVE-27087 > Project: Hive > Issue Type: Sub-task >Reporter: Vihang Karajgaonkar >Assignee: Vihang Karajgaonkar >Priority: Major > Labels: pull-request-available > Time Spent: 2.5h > Remaining Estimate: 0h > > TestMiniSparkOnYarnCliDriver are failing with the error below > [ERROR] 2023-02-16 14:13:08.991 [Driver] SparkContext - Error initializing > SparkContext. > java.lang.RuntimeException: java.lang.NoSuchFieldException: > DEFAULT_TINY_CACHE_SIZE > at > org.apache.spark.network.util.NettyUtils.getPrivateStaticField(NettyUtils.java:131) > ~[spark-network-common_2.11-2.3.0.jar:2.3.0] > at > org.apache.spark.network.util.NettyUtils.createPooledByteBufAllocator(NettyUtils.java:118) > ~[spark-network-common_2.11-2.3.0.jar:2.3.0] > at > org.apache.spark.network.server.TransportServer.init(TransportServer.java:94) > ~[spark-network-common_2.11-2.3.0.jar:2.3.0] > at > org.apache.spark.network.server.TransportServer.(TransportServer.java:73) > ~[spark-network-common_2.11-2.3.0.jar:2.3.0] > at > org.apache.spark.network.TransportContext.createServer(TransportContext.java:114) > ~[spark-network-common_2.11-2.3.0.jar:2.3.0] > at org.apache.spark.rpc.netty.NettyRpcEnv.startServer(NettyRpcEnv.scala:119) > ~[spark-core_2.11-2.3.0.jar:2.3.0] > at > org.apache.spark.rpc.netty.NettyRpcEnvFactory$$anonfun$4.apply(NettyRpcEnv.scala:465) > ~[spark-core_2.11-2.3.0.jar:2.3.0] > at > org.apache.spark.rpc.netty.NettyRpcEnvFactory$$anonfun$4.apply(NettyRpcEnv.scala:464) > ~[spark-core_2.11-2.3.0.jar:2.3.0] > at > org.apache.spark.util.Utils$$anonfun$startServiceOnPort$1.apply$mcVI$sp(Utils.scala:2271) > ~[spark-core_2.11-2.3.0.jar:2.3.0] > at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:160) > ~[scala-library-2.11.8.jar:?] > at org.apache.spark.util.Utils$.startServiceOnPort(Utils.scala:2263) > ~[spark-core_2.11-2.3.0.jar:2.3.0] > at > org.apache.spark.rpc.netty.NettyRpcEnvFactory.create(NettyRpcEnv.scala:469) > ~[spark-core_2.11-2.3.0.jar:2.3.0] > at org.apache.spark.rpc.RpcEnv$.create(RpcEnv.scala:57) > ~[spark-core_2.11-2.3.0.jar:2.3.0] > at org.apache.spark.SparkEnv$.create(SparkEnv.scala:249) > ~[spark-core_2.11-2.3.0.jar:2.3.0] > at org.apache.spark.SparkEnv$.createDriverEnv(SparkEnv.scala:175) > ~[spark-core_2.11-2.3.0.jar:2.3.0] > at org.apache.spark.SparkContext.createSparkEnv(SparkContext.scala:256) > [spark-core_2.11-2.3.0.jar:2.3.0] > at org.apache.spark.SparkContext.(SparkContext.scala:423) > [spark-core_2.11-2.3.0.jar:2.3.0] > at > org.apache.spark.api.java.JavaSparkContext.(JavaSparkContext.scala:58) > [spark-core_2.11-2.3.0.jar:2.3.0] > at org.apache.hive.spark.client.RemoteDriver.(RemoteDriver.java:161) > [hive-exec-3.2.0-SNAPSHOT.jar:3.2.0-SNAPSHOT] > at org.apache.hive.spark.client.RemoteDriver.main(RemoteDriver.java:536) > [hive-exec-3.2.0-SNAPSHOT.jar:3.2.0-SNAPSHOT] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_322] > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > ~[?:1.8.0_322] > The root cause of the problem is that we upgrade the netty library from > 4.1.17.Final to 4.1.69.Final. The upgraded library does not have > `DEFAULT_TINY_CACHE_SIZE` field > [here|https://github.com/netty/netty/blob/netty-4.1.51.Final/buffer/src/main/java/io/netty/buffer/PooledByteBufAllocator.java#L46] > which was removed in 4.1.52.Final -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HIVE-26659) TPC-DS query 16, 69, 94 return wrong results.
[ https://issues.apache.org/jira/browse/HIVE-26659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17698134#comment-17698134 ] Steve Carlin commented on HIVE-26659: - Filed HIVE-27124 to temporarily turn off the anti-join feature by default until this is fixed. > TPC-DS query 16, 69, 94 return wrong results. > - > > Key: HIVE-26659 > URL: https://issues.apache.org/jira/browse/HIVE-26659 > Project: Hive > Issue Type: Sub-task >Affects Versions: 4.0.0-alpha-2 >Reporter: Sungwoo Park >Priority: Major > > TPC-DS query 16, 69, 94 return wrong results when hive.auto.convert.anti.join > is set to true. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work started] (HIVE-27129) Enhanced support to Hive Client http support
[ https://issues.apache.org/jira/browse/HIVE-27129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-27129 started by Junlin Zeng. -- > Enhanced support to Hive Client http support > > > Key: HIVE-27129 > URL: https://issues.apache.org/jira/browse/HIVE-27129 > Project: Hive > Issue Type: Improvement >Reporter: Junlin Zeng >Assignee: Junlin Zeng >Priority: Major > > Currently we support using http in the hive metastore connection. However, we > do not support custom headers and also default trust store. This ticket > tracks the work to improve the http journey. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (HIVE-27129) Enhanced support to Hive Client http support
[ https://issues.apache.org/jira/browse/HIVE-27129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junlin Zeng reassigned HIVE-27129: -- > Enhanced support to Hive Client http support > > > Key: HIVE-27129 > URL: https://issues.apache.org/jira/browse/HIVE-27129 > Project: Hive > Issue Type: Improvement >Reporter: Junlin Zeng >Assignee: Junlin Zeng >Priority: Major > > Currently we support using http in the hive metastore connection. However, we > do not support custom headers and also default trust store. This ticket > tracks the work to improve the http journey. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27089) Update permissions in orc_merge*.q.out files
[ https://issues.apache.org/jira/browse/HIVE-27089?focusedWorklogId=849933&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-849933 ] ASF GitHub Bot logged work on HIVE-27089: - Author: ASF GitHub Bot Created on: 09/Mar/23 00:22 Start Date: 09/Mar/23 00:22 Worklog Time Spent: 10m Work Description: yigress commented on PR #4068: URL: https://github.com/apache/hive/pull/4068#issuecomment-1461074375 @zabetak thank you for taking time detailed analysis! Issue Time Tracking --- Worklog Id: (was: 849933) Time Spent: 2h 50m (was: 2h 40m) > Update permissions in orc_merge*.q.out files > > > Key: HIVE-27089 > URL: https://issues.apache.org/jira/browse/HIVE-27089 > Project: Hive > Issue Type: Sub-task >Reporter: Aman Raj >Assignee: Aman Raj >Priority: Major > Labels: pull-request-available > Fix For: 3.2.0 > > Time Spent: 2h 50m > Remaining Estimate: 0h > > The orc_merge1,2,3,4,10 .q tests fails with the following errors : > < -rw-rw-rw- 3 ### USER ### ### GROUP ### 349 ### HDFS DATE ### hdfs://### > HDFS PATH ### > --- > > -rw-r--r-- 3 ### USER ### ### GROUP ### 349 ### HDFS DATE ### hdfs://### > > HDFS PATH ### -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (HIVE-27128) Exception "Can't finish byte read from uncompressed stream DATA position" when querying ORC table
[
https://issues.apache.org/jira/browse/HIVE-27128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dmitriy Fingerman reassigned HIVE-27128:
> Exception "Can't finish byte read from uncompressed stream DATA position"
> when querying ORC table
> -
>
> Key: HIVE-27128
> URL: https://issues.apache.org/jira/browse/HIVE-27128
> Project: Hive
> Issue Type: Bug
>Reporter: Dmitriy Fingerman
>Assignee: Dmitriy Fingerman
>Priority: Critical
>
> Exception happening when querying an ORC table:
> {code:java}
> Caused by: java.io.EOFException: Can't finish byte read from uncompressed
> stream DATA position: 393216 length: 393216 range: 23 offset: 376832
> position: 16384 limit: 16384
> at
> org.apache.orc.impl.TreeReaderFactory$BytesColumnVectorUtil.commonReadByteArrays(TreeReaderFactory.java:1550)
> at
> org.apache.orc.impl.TreeReaderFactory$BytesColumnVectorUtil.readOrcByteArrays(TreeReaderFactory.java:1566)
> at
> org.apache.orc.impl.TreeReaderFactory$StringDirectTreeReader.nextVector(TreeReaderFactory.java:1662)
> at
> org.apache.orc.impl.TreeReaderFactory$StringTreeReader.nextVector(TreeReaderFactory.java:1508)
> at
> org.apache.hadoop.hive.ql.io.orc.encoded.EncodedTreeReaderFactory$StringStreamReader.nextVector(EncodedTreeReaderFactory.java:305)
> at
> org.apache.hadoop.hive.llap.io.decode.OrcEncodedDataConsumer.decodeBatch(OrcEncodedDataConsumer.java:196)
> at
> org.apache.hadoop.hive.llap.io.decode.OrcEncodedDataConsumer.decodeBatch(OrcEncodedDataConsumer.java:66)
> at
> org.apache.hadoop.hive.llap.io.decode.EncodedDataConsumer.consumeData(EncodedDataConsumer.java:122)
> at
> org.apache.hadoop.hive.llap.io.decode.EncodedDataConsumer.consumeData(EncodedDataConsumer.java:42)
> at
> org.apache.hadoop.hive.ql.io.orc.encoded.EncodedReaderImpl.readEncodedColumns(EncodedReaderImpl.java:608)
> at
> org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader.performDataRead(OrcEncodedDataReader.java:434)
> at
> org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader$4.run(OrcEncodedDataReader.java:282)
> at
> org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader$4.run(OrcEncodedDataReader.java:279)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1878)
> at
> org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader.callInternal(OrcEncodedDataReader.java:279)
> at
> org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader.callInternal(OrcEncodedDataReader.java:118)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at
> org.apache.hadoop.hive.llap.io.decode.EncodedDataConsumer$CpuRecordingCallable.call(EncodedDataConsumer.java:88)
> at
> org.apache.hadoop.hive.llap.io.decode.EncodedDataConsumer$CpuRecordingCallable.call(EncodedDataConsumer.java:73)
> {code}
> I created a q-test that reproduces this issue:
> [https://github.com/difin/hive/commits/orc_read_err_qtest]
> This issue happens in Hive starting from the commit that upgraded ORC version
> in Hive to ORC 1.6.7.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27124) Temporarily disable default usage of anti-join
[ https://issues.apache.org/jira/browse/HIVE-27124?focusedWorklogId=849914&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-849914 ] ASF GitHub Bot logged work on HIVE-27124: - Author: ASF GitHub Bot Created on: 08/Mar/23 21:00 Start Date: 08/Mar/23 21:00 Worklog Time Spent: 10m Work Description: sonarcloud[bot] commented on PR #4101: URL: https://github.com/apache/hive/pull/4101#issuecomment-1460869110 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4101) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4101&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4101&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4101&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4101&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4101&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4101&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4101&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4101&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4101&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4101&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4101&resolved=false&types=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4101&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4101&metric=coverage&view=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4101&metric=duplicated_lines_density&view=list) No Duplication information Issue Time Tracking --- Worklog Id: (was: 849914) Time Spent: 0.5h (was: 20m) > Temporarily disable default usage of anti-join > -- > > Key: HIVE-27124 > URL: https://issues.apache.org/jira/browse/HIVE-27124 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Steve Carlin >Priority: Major > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > > Temporarily disabling the default usage for antijoin due to wrong results as > documented by HIVE-26659 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27087) Fix TestMiniSparkOnYarnCliDriver test failures on branch-3
[ https://issues.apache.org/jira/browse/HIVE-27087?focusedWorklogId=849902&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-849902 ] ASF GitHub Bot logged work on HIVE-27087: - Author: ASF GitHub Bot Created on: 08/Mar/23 19:47 Start Date: 08/Mar/23 19:47 Worklog Time Spent: 10m Work Description: vihangk1 commented on PR #4067: URL: https://github.com/apache/hive/pull/4067#issuecomment-1460758753 The diffs that we see are due to the a difference in replication factor of the files on the hdfs. I wonder if the mini HDFS for the tests is slow to replicate the blocks for these files. Added a sleep to confirm if that helps because I cannot reproduce it in my local even after running all the tests in the split. Issue Time Tracking --- Worklog Id: (was: 849902) Time Spent: 2h 20m (was: 2h 10m) > Fix TestMiniSparkOnYarnCliDriver test failures on branch-3 > -- > > Key: HIVE-27087 > URL: https://issues.apache.org/jira/browse/HIVE-27087 > Project: Hive > Issue Type: Sub-task >Reporter: Vihang Karajgaonkar >Assignee: Vihang Karajgaonkar >Priority: Major > Labels: pull-request-available > Time Spent: 2h 20m > Remaining Estimate: 0h > > TestMiniSparkOnYarnCliDriver are failing with the error below > [ERROR] 2023-02-16 14:13:08.991 [Driver] SparkContext - Error initializing > SparkContext. > java.lang.RuntimeException: java.lang.NoSuchFieldException: > DEFAULT_TINY_CACHE_SIZE > at > org.apache.spark.network.util.NettyUtils.getPrivateStaticField(NettyUtils.java:131) > ~[spark-network-common_2.11-2.3.0.jar:2.3.0] > at > org.apache.spark.network.util.NettyUtils.createPooledByteBufAllocator(NettyUtils.java:118) > ~[spark-network-common_2.11-2.3.0.jar:2.3.0] > at > org.apache.spark.network.server.TransportServer.init(TransportServer.java:94) > ~[spark-network-common_2.11-2.3.0.jar:2.3.0] > at > org.apache.spark.network.server.TransportServer.(TransportServer.java:73) > ~[spark-network-common_2.11-2.3.0.jar:2.3.0] > at > org.apache.spark.network.TransportContext.createServer(TransportContext.java:114) > ~[spark-network-common_2.11-2.3.0.jar:2.3.0] > at org.apache.spark.rpc.netty.NettyRpcEnv.startServer(NettyRpcEnv.scala:119) > ~[spark-core_2.11-2.3.0.jar:2.3.0] > at > org.apache.spark.rpc.netty.NettyRpcEnvFactory$$anonfun$4.apply(NettyRpcEnv.scala:465) > ~[spark-core_2.11-2.3.0.jar:2.3.0] > at > org.apache.spark.rpc.netty.NettyRpcEnvFactory$$anonfun$4.apply(NettyRpcEnv.scala:464) > ~[spark-core_2.11-2.3.0.jar:2.3.0] > at > org.apache.spark.util.Utils$$anonfun$startServiceOnPort$1.apply$mcVI$sp(Utils.scala:2271) > ~[spark-core_2.11-2.3.0.jar:2.3.0] > at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:160) > ~[scala-library-2.11.8.jar:?] > at org.apache.spark.util.Utils$.startServiceOnPort(Utils.scala:2263) > ~[spark-core_2.11-2.3.0.jar:2.3.0] > at > org.apache.spark.rpc.netty.NettyRpcEnvFactory.create(NettyRpcEnv.scala:469) > ~[spark-core_2.11-2.3.0.jar:2.3.0] > at org.apache.spark.rpc.RpcEnv$.create(RpcEnv.scala:57) > ~[spark-core_2.11-2.3.0.jar:2.3.0] > at org.apache.spark.SparkEnv$.create(SparkEnv.scala:249) > ~[spark-core_2.11-2.3.0.jar:2.3.0] > at org.apache.spark.SparkEnv$.createDriverEnv(SparkEnv.scala:175) > ~[spark-core_2.11-2.3.0.jar:2.3.0] > at org.apache.spark.SparkContext.createSparkEnv(SparkContext.scala:256) > [spark-core_2.11-2.3.0.jar:2.3.0] > at org.apache.spark.SparkContext.(SparkContext.scala:423) > [spark-core_2.11-2.3.0.jar:2.3.0] > at > org.apache.spark.api.java.JavaSparkContext.(JavaSparkContext.scala:58) > [spark-core_2.11-2.3.0.jar:2.3.0] > at org.apache.hive.spark.client.RemoteDriver.(RemoteDriver.java:161) > [hive-exec-3.2.0-SNAPSHOT.jar:3.2.0-SNAPSHOT] > at org.apache.hive.spark.client.RemoteDriver.main(RemoteDriver.java:536) > [hive-exec-3.2.0-SNAPSHOT.jar:3.2.0-SNAPSHOT] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_322] > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > ~[?:1.8.0_322] > The root cause of the problem is that we upgrade the netty library from > 4.1.17.Final to 4.1.69.Final. The upgraded library does not have > `DEFAULT_TINY_CACHE_SIZE` field > [here|https://github.com/netty/netty/blob/netty-4.1.51.Final/buffer/src/main/java/io/netty/buffer/PooledByteBufAllocator.java#L46] > which was removed in 4.1.52.Final -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HIVE-26735) Ability to sort the data during rebalancing compaction
[ https://issues.apache.org/jira/browse/HIVE-26735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Végh resolved HIVE-26735. Resolution: Fixed Merged to master, [~kkasa] , [~sbadhya] , [~dkuzmenko] thanks for the review! > Ability to sort the data during rebalancing compaction > -- > > Key: HIVE-26735 > URL: https://issues.apache.org/jira/browse/HIVE-26735 > Project: Hive > Issue Type: Sub-task >Reporter: László Végh >Assignee: László Végh >Priority: Major > Labels: pull-request-available > Time Spent: 6h 10m > Remaining Estimate: 0h > > Ability to sort the data during rebalancing compaction -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26735) Ability to sort the data during rebalancing compaction
[ https://issues.apache.org/jira/browse/HIVE-26735?focusedWorklogId=849893&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-849893 ] ASF GitHub Bot logged work on HIVE-26735: - Author: ASF GitHub Bot Created on: 08/Mar/23 18:14 Start Date: 08/Mar/23 18:14 Worklog Time Spent: 10m Work Description: veghlaci05 merged PR #4077: URL: https://github.com/apache/hive/pull/4077 Issue Time Tracking --- Worklog Id: (was: 849893) Time Spent: 6h 10m (was: 6h) > Ability to sort the data during rebalancing compaction > -- > > Key: HIVE-26735 > URL: https://issues.apache.org/jira/browse/HIVE-26735 > Project: Hive > Issue Type: Sub-task >Reporter: László Végh >Assignee: László Végh >Priority: Major > Labels: pull-request-available > Time Spent: 6h 10m > Remaining Estimate: 0h > > Ability to sort the data during rebalancing compaction -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27013) Provide an option to enable iceberg manifest caching for all catalogs
[
https://issues.apache.org/jira/browse/HIVE-27013?focusedWorklogId=849892&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-849892
]
ASF GitHub Bot logged work on HIVE-27013:
-
Author: ASF GitHub Bot
Created on: 08/Mar/23 18:13
Start Date: 08/Mar/23 18:13
Worklog Time Spent: 10m
Work Description: sonarcloud[bot] commented on PR #4088:
URL: https://github.com/apache/hive/pull/4088#issuecomment-1460631926
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4088)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4088&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4088&resolved=false&types=BUG)
[0
Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4088&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4088&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4088&resolved=false&types=VULNERABILITY)
[0
Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4088&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4088&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4088&resolved=false&types=SECURITY_HOTSPOT)
[0 Security
Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4088&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4088&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4088&resolved=false&types=CODE_SMELL)
[0 Code
Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4088&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4088&metric=coverage&view=list)
No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4088&metric=duplicated_lines_density&view=list)
No Duplication information
Issue Time Tracking
---
Worklog Id: (was: 849892)
Time Spent: 3h (was: 2h 50m)
> Provide an option to enable iceberg manifest caching for all catalogs
> -
>
> Key: HIVE-27013
> URL: https://issues.apache.org/jira/browse/HIVE-27013
> Project: Hive
> Issue Type: Improvement
> Components: Iceberg integration
>Reporter: Rajesh Balamohan
>Assignee: Ayush Saxena
>Priority: Major
> Labels: pull-request-available
> Time Spent: 3h
> Remaining Estimate: 0h
>
> {color:#22}I tried the following thinking that it would work with iceberg
> manifest caching; but it didn't.{color}
> {noformat}
> alter table store_sales set
> tblproperties('io.manifest.cache-enabled'='true');{noformat}
> {color:#22}Creating this ticket as a placeholder to fix the same.{color}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HIVE-27126) Display Yarn resources availability in real time along with Tez job progress
[
https://issues.apache.org/jira/browse/HIVE-27126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17698031#comment-17698031
]
László Bodor commented on HIVE-27126:
-
+1 on improving real-time data in beeline progress, let me link HIVE-25666 here
> Display Yarn resources availability in real time along with Tez job progress
> -
>
> Key: HIVE-27126
> URL: https://issues.apache.org/jira/browse/HIVE-27126
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Reporter: Taraka Rama Rao Lethavadla
>Priority: Major
>
> Right we don't know if the queue used in Hive query execution is having
> enough resources or not. We are already displaying tez job details like
> number of task per each vertex and how are they progressing.
> If the resources available are not good enough to execute the query in
> parallel or query that use to take shorter time is taking time, then we can
> use this new detail to understand that queue is busy and resources are not
> enough
>
>
>
> {noformat}
> --
> VERTICES MODE STATUS TOTAL COMPLETED RUNNING PENDING
> FAILED KILLED
> --
> Map 1 container RUNNING 230 0 10 230
> 0 1
> Reducer 2 container INITED 1 0 0 1
> 0 0
> --
> VERTICES: 00/02 [>>--] 0% ELAPSED TIME: 58.33 s
> --
> {noformat}
> So it would be great to display current resource availability in that
> specific queue along with above info in real time would help understand why a
> query can be slow
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27020) Implement a separate handler to handle aborted transaction cleanup
[ https://issues.apache.org/jira/browse/HIVE-27020?focusedWorklogId=849846&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-849846 ] ASF GitHub Bot logged work on HIVE-27020: - Author: ASF GitHub Bot Created on: 08/Mar/23 16:36 Start Date: 08/Mar/23 16:36 Worklog Time Spent: 10m Work Description: sonarcloud[bot] commented on PR #4091: URL: https://github.com/apache/hive/pull/4091#issuecomment-1460470703 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4091) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4091&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4091&resolved=false&types=BUG) [2 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4091&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4091&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4091&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4091&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4091&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4091&resolved=false&types=SECURITY_HOTSPOT) [1 Security Hotspot](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4091&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4091&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4091&resolved=false&types=CODE_SMELL) [14 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4091&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4091&metric=coverage&view=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4091&metric=duplicated_lines_density&view=list) No Duplication information Issue Time Tracking --- Worklog Id: (was: 849846) Time Spent: 50m (was: 40m) > Implement a separate handler to handle aborted transaction cleanup > -- > > Key: HIVE-27020 > URL: https://issues.apache.org/jira/browse/HIVE-27020 > Project: Hive > Issue Type: Sub-task >Reporter: Sourabh Badhya >Assignee: Sourabh Badhya >Priority: Major > Labels: pull-request-available > Time Spent: 50m > Remaining Estimate: 0h > > As described in the parent task, once the cleaner is separated into different > entities, implement a separate handler which can create requests for aborted > transactions cleanup. This would move the aborted transaction cleanup > exclusively to the cleaner. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26735) Ability to sort the data during rebalancing compaction
[ https://issues.apache.org/jira/browse/HIVE-26735?focusedWorklogId=849830&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-849830 ] ASF GitHub Bot logged work on HIVE-26735: - Author: ASF GitHub Bot Created on: 08/Mar/23 15:02 Start Date: 08/Mar/23 15:02 Worklog Time Spent: 10m Work Description: sonarcloud[bot] commented on PR #4077: URL: https://github.com/apache/hive/pull/4077#issuecomment-1460293044 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4077) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4077&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4077&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4077&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4077&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4077&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4077&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4077&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4077&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4077&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4077&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4077&resolved=false&types=CODE_SMELL) [15 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4077&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4077&metric=coverage&view=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4077&metric=duplicated_lines_density&view=list) No Duplication information Issue Time Tracking --- Worklog Id: (was: 849830) Time Spent: 6h (was: 5h 50m) > Ability to sort the data during rebalancing compaction > -- > > Key: HIVE-26735 > URL: https://issues.apache.org/jira/browse/HIVE-26735 > Project: Hive > Issue Type: Sub-task >Reporter: László Végh >Assignee: László Végh >Priority: Major > Labels: pull-request-available > Time Spent: 6h > Remaining Estimate: 0h > > Ability to sort the data during rebalancing compaction -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HIVE-27089) Update permissions in orc_merge*.q.out files
[ https://issues.apache.org/jira/browse/HIVE-27089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stamatis Zampetakis resolved HIVE-27089. Fix Version/s: 3.2.0 Resolution: Fixed Fixed in https://github.com/apache/hive/commit/c0ec46292b18242f5a65df14141910d8197023d6. Thanks [~amanraj2520] for the PR! > Update permissions in orc_merge*.q.out files > > > Key: HIVE-27089 > URL: https://issues.apache.org/jira/browse/HIVE-27089 > Project: Hive > Issue Type: Sub-task >Reporter: Aman Raj >Assignee: Aman Raj >Priority: Major > Labels: pull-request-available > Fix For: 3.2.0 > > Time Spent: 2h 40m > Remaining Estimate: 0h > > The orc_merge1,2,3,4,10 .q tests fails with the following errors : > < -rw-rw-rw- 3 ### USER ### ### GROUP ### 349 ### HDFS DATE ### hdfs://### > HDFS PATH ### > --- > > -rw-r--r-- 3 ### USER ### ### GROUP ### 349 ### HDFS DATE ### hdfs://### > > HDFS PATH ### -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27089) Fix orc_merge*.q test failures
[ https://issues.apache.org/jira/browse/HIVE-27089?focusedWorklogId=849794&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-849794 ] ASF GitHub Bot logged work on HIVE-27089: - Author: ASF GitHub Bot Created on: 08/Mar/23 12:13 Start Date: 08/Mar/23 12:13 Worklog Time Spent: 10m Work Description: zabetak closed pull request #4068: HIVE-27089 : Test fixes for orc_merge*.q URL: https://github.com/apache/hive/pull/4068 Issue Time Tracking --- Worklog Id: (was: 849794) Time Spent: 2h 40m (was: 2.5h) > Fix orc_merge*.q test failures > -- > > Key: HIVE-27089 > URL: https://issues.apache.org/jira/browse/HIVE-27089 > Project: Hive > Issue Type: Sub-task >Reporter: Aman Raj >Assignee: Aman Raj >Priority: Major > Labels: pull-request-available > Time Spent: 2h 40m > Remaining Estimate: 0h > > The orc_merge1,2,3,4,10 .q tests fails with the following errors : > < -rw-rw-rw- 3 ### USER ### ### GROUP ### 349 ### HDFS DATE ### hdfs://### > HDFS PATH ### > --- > > -rw-r--r-- 3 ### USER ### ### GROUP ### 349 ### HDFS DATE ### hdfs://### > > HDFS PATH ### -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27089) Update permissions in orc_merge*.q.out files
[ https://issues.apache.org/jira/browse/HIVE-27089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stamatis Zampetakis updated HIVE-27089: --- Summary: Update permissions in orc_merge*.q.out files (was: Fix orc_merge*.q test failures) > Update permissions in orc_merge*.q.out files > > > Key: HIVE-27089 > URL: https://issues.apache.org/jira/browse/HIVE-27089 > Project: Hive > Issue Type: Sub-task >Reporter: Aman Raj >Assignee: Aman Raj >Priority: Major > Labels: pull-request-available > Time Spent: 2h 40m > Remaining Estimate: 0h > > The orc_merge1,2,3,4,10 .q tests fails with the following errors : > < -rw-rw-rw- 3 ### USER ### ### GROUP ### 349 ### HDFS DATE ### hdfs://### > HDFS PATH ### > --- > > -rw-r--r-- 3 ### USER ### ### GROUP ### 349 ### HDFS DATE ### hdfs://### > > HDFS PATH ### -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27096) Fix test dbtxnmgr_showlocks.q
[ https://issues.apache.org/jira/browse/HIVE-27096?focusedWorklogId=849793&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-849793 ] ASF GitHub Bot logged work on HIVE-27096: - Author: ASF GitHub Bot Created on: 08/Mar/23 12:12 Start Date: 08/Mar/23 12:12 Worklog Time Spent: 10m Work Description: zabetak closed pull request #4075: HIVE-27096 : Fix test dbtxnmgr_showlocks.q URL: https://github.com/apache/hive/pull/4075 Issue Time Tracking --- Worklog Id: (was: 849793) Time Spent: 2h 20m (was: 2h 10m) > Fix test dbtxnmgr_showlocks.q > - > > Key: HIVE-27096 > URL: https://issues.apache.org/jira/browse/HIVE-27096 > Project: Hive > Issue Type: Sub-task >Reporter: Aman Raj >Assignee: Aman Raj >Priority: Major > Labels: pull-request-available > Time Spent: 2h 20m > Remaining Estimate: 0h > > The test fails due to the following Assertion ERROR : > Client Execution succeeded but contained differences (error code = 1) after > executing dbtxnmgr_showlocks.q > 73c73 > < 5 default partitioned_acid_table p=abc MINOR initiated --- --- --- --- > --- > > 1 default partitioned_acid_table p=abc MINOR initiated --- --- --- --- > > It expects a Compaction ID 1 but gets 5. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HIVE-27096) Update dbtxnmgr_showlocks.q.out file
[ https://issues.apache.org/jira/browse/HIVE-27096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stamatis Zampetakis resolved HIVE-27096. Fix Version/s: 3.2.0 Resolution: Fixed Fixed in https://github.com/apache/hive/commit/f1db3f25a7d77f7fb8f4949d94dc2fe43b604bb2. Thanks for the PR [~amanraj2520]! > Update dbtxnmgr_showlocks.q.out file > > > Key: HIVE-27096 > URL: https://issues.apache.org/jira/browse/HIVE-27096 > Project: Hive > Issue Type: Sub-task >Reporter: Aman Raj >Assignee: Aman Raj >Priority: Major > Labels: pull-request-available > Fix For: 3.2.0 > > Time Spent: 2h 20m > Remaining Estimate: 0h > > The test fails due to the following Assertion ERROR : > Client Execution succeeded but contained differences (error code = 1) after > executing dbtxnmgr_showlocks.q > 73c73 > < 5 default partitioned_acid_table p=abc MINOR initiated --- --- --- --- > --- > > 1 default partitioned_acid_table p=abc MINOR initiated --- --- --- --- > > It expects a Compaction ID 1 but gets 5. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27096) Update dbtxnmgr_showlocks.q.out file
[ https://issues.apache.org/jira/browse/HIVE-27096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stamatis Zampetakis updated HIVE-27096: --- Summary: Update dbtxnmgr_showlocks.q.out file (was: Fix test dbtxnmgr_showlocks.q) > Update dbtxnmgr_showlocks.q.out file > > > Key: HIVE-27096 > URL: https://issues.apache.org/jira/browse/HIVE-27096 > Project: Hive > Issue Type: Sub-task >Reporter: Aman Raj >Assignee: Aman Raj >Priority: Major > Labels: pull-request-available > Time Spent: 2h 20m > Remaining Estimate: 0h > > The test fails due to the following Assertion ERROR : > Client Execution succeeded but contained differences (error code = 1) after > executing dbtxnmgr_showlocks.q > 73c73 > < 5 default partitioned_acid_table p=abc MINOR initiated --- --- --- --- > --- > > 1 default partitioned_acid_table p=abc MINOR initiated --- --- --- --- > > It expects a Compaction ID 1 but gets 5. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27100) Remove unused data/files from repo
[
https://issues.apache.org/jira/browse/HIVE-27100?focusedWorklogId=849789&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-849789
]
ASF GitHub Bot logged work on HIVE-27100:
-
Author: ASF GitHub Bot
Created on: 08/Mar/23 12:01
Start Date: 08/Mar/23 12:01
Worklog Time Spent: 10m
Work Description: sonarcloud[bot] commented on PR #4081:
URL: https://github.com/apache/hive/pull/4081#issuecomment-1460052936
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4081)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4081&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4081&resolved=false&types=BUG)
[0
Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4081&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4081&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4081&resolved=false&types=VULNERABILITY)
[0
Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4081&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4081&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4081&resolved=false&types=SECURITY_HOTSPOT)
[0 Security
Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4081&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4081&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4081&resolved=false&types=CODE_SMELL)
[3 Code
Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4081&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4081&metric=coverage&view=list)
No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4081&metric=duplicated_lines_density&view=list)
No Duplication information
Issue Time Tracking
---
Worklog Id: (was: 849789)
Time Spent: 40m (was: 0.5h)
> Remove unused data/files from repo
> --
>
> Key: HIVE-27100
> URL: https://issues.apache.org/jira/browse/HIVE-27100
> Project: Hive
> Issue Type: Task
>Reporter: Stamatis Zampetakis
>Assignee: Stamatis Zampetakis
>Priority: Major
> Labels: pull-request-available
> Time Spent: 40m
> Remaining Estimate: 0h
>
> Some files under [https://github.com/apache/hive/tree/master/data/files] are
> not referenced anywhere else in the repo and can be removed.
> Removing them makes it easier to see what is actually tested. Other minor
> benefits:
> * faster checkout times;
> * smaller source/binary releases.
> The script that was used to find which files are not referenced can be found
> below:
> {code:bash}
> for f in `ls data/files`; do
> echo -n "$f ";
> grep -a -R "$f" --exclude-dir=".git" --exclude-dir=target
> --exclude=\*.q.out --exclude=\*.class --exclude=\*.jar | wc -l | grep " 0$";
> done
> {code}
> +Output+
> {noformat}
> cbo_t4.txt
[jira] [Work logged] (HIVE-27089) Fix orc_merge*.q test failures
[ https://issues.apache.org/jira/browse/HIVE-27089?focusedWorklogId=849784&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-849784 ] ASF GitHub Bot logged work on HIVE-27089: - Author: ASF GitHub Bot Created on: 08/Mar/23 11:49 Start Date: 08/Mar/23 11:49 Worklog Time Spent: 10m Work Description: zabetak commented on PR #4068: URL: https://github.com/apache/hive/pull/4068#issuecomment-1460040365 @yigress I am not sure if it is LLAP related. If you check the `orc_merge1.q` file there are three `dfs -ls` operations always following an `INSERT OVERWRITE TABLE` query. I assume that the `OVERWRITE` query rewrites the directory every time. Before HIVE-27016: ``` INSERT OVERWRITE TABLE dfs -ls -rw-r--r-- INSERT OVERWRITE TABLE dfs -ls -rw-rw-rw- INSERT OVERWRITE TABLE dfs -ls -rw-rw-rw- ``` After HIVE-27016: ``` INSERT OVERWRITE TABLE dfs -ls -rw-rw-rw- INSERT OVERWRITE TABLE dfs -ls -rw-rw-rw- INSERT OVERWRITE TABLE dfs -ls -rw-rw-rw- ``` Worth mentioning that even before HIVE-27016 we have directories created with `-rw-rw-rw-` just not the first time. There must be a code-path that operates with different permissions and this is definitely a bug. If someone can pinpoint the exact reason of why this happens and raise a JIRA that would be great. However, given that permission differences existed even before HIVE-27016, I will not consider HIVE-27016 as a regression so I am OK to simply update the .q.out files for the moment. Issue Time Tracking --- Worklog Id: (was: 849784) Time Spent: 2.5h (was: 2h 20m) > Fix orc_merge*.q test failures > -- > > Key: HIVE-27089 > URL: https://issues.apache.org/jira/browse/HIVE-27089 > Project: Hive > Issue Type: Sub-task >Reporter: Aman Raj >Assignee: Aman Raj >Priority: Major > Labels: pull-request-available > Time Spent: 2.5h > Remaining Estimate: 0h > > The orc_merge1,2,3,4,10 .q tests fails with the following errors : > < -rw-rw-rw- 3 ### USER ### ### GROUP ### 349 ### HDFS DATE ### hdfs://### > HDFS PATH ### > --- > > -rw-r--r-- 3 ### USER ### ### GROUP ### 349 ### HDFS DATE ### hdfs://### > > HDFS PATH ### -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27127) Upgrade jackson version to 2.13.5
[ https://issues.apache.org/jira/browse/HIVE-27127?focusedWorklogId=849779&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-849779 ] ASF GitHub Bot logged work on HIVE-27127: - Author: ASF GitHub Bot Created on: 08/Mar/23 11:11 Start Date: 08/Mar/23 11:11 Worklog Time Spent: 10m Work Description: sonarcloud[bot] commented on PR #4102: URL: https://github.com/apache/hive/pull/4102#issuecomment-1460001416 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4102) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4102&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4102&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4102&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4102&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4102&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4102&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4102&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4102&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4102&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4102&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4102&resolved=false&types=CODE_SMELL) [3 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4102&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4102&metric=coverage&view=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4102&metric=duplicated_lines_density&view=list) No Duplication information Issue Time Tracking --- Worklog Id: (was: 849779) Time Spent: 20m (was: 10m) > Upgrade jackson version to 2.13.5 > - > > Key: HIVE-27127 > URL: https://issues.apache.org/jira/browse/HIVE-27127 > Project: Hive > Issue Type: Improvement >Reporter: Sand Shreeya >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (HIVE-27034) Backport HIVE-22648 HIVE-24408 HIVE-26917: Upgrade parquet to 1.12.3 in branch-3
[ https://issues.apache.org/jira/browse/HIVE-27034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sand Shreeya reassigned HIVE-27034: --- Assignee: (was: Sand Shreeya) > Backport HIVE-22648 HIVE-24408 HIVE-26917: Upgrade parquet to 1.12.3 in > branch-3 > > > Key: HIVE-27034 > URL: https://issues.apache.org/jira/browse/HIVE-27034 > Project: Hive > Issue Type: Task >Reporter: Sand Shreeya >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26900) Error message not representing the correct line number with a syntax error in a HQL File
[ https://issues.apache.org/jira/browse/HIVE-26900?focusedWorklogId=849772&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-849772 ] ASF GitHub Bot logged work on HIVE-26900: - Author: ASF GitHub Bot Created on: 08/Mar/23 10:04 Start Date: 08/Mar/23 10:04 Worklog Time Spent: 10m Work Description: shreeyasand closed pull request #4097: HIVE-26900: Error message not representing the correct line number wi… URL: https://github.com/apache/hive/pull/4097 Issue Time Tracking --- Worklog Id: (was: 849772) Time Spent: 40m (was: 0.5h) > Error message not representing the correct line number with a syntax error in > a HQL File > > > Key: HIVE-26900 > URL: https://issues.apache.org/jira/browse/HIVE-26900 > Project: Hive > Issue Type: Bug >Affects Versions: 3.1.2, 4.0.0-alpha-1, 4.0.0-alpha-2 >Reporter: Vikram Ahuja >Priority: Minor > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > When a wrong syntax is added in a HQL file, the error thrown by beeline while > running the HQL file is having the wrong line number. The line number and > even the position is incorrect. Seems like parser is not considering spaces > and new lines and always throwing the error on line number 1 irrespective of > what line the error is on in the HQL file > > For instance, consider the following test.hql file: > # --comment > # --comment > # SET hive.server2.logging.operation.enabled=true; > # SET hive.server2.logging.operation.level=VERBOSE; > # show tables; > # > # > # CREATE TABLEE DUMMY; > > when we call !run test.hql in beeline or trigger ./beeline -u > jdbc:hive2://localhost:1 -f test.hql, The issue thrown by beeline is > >>> CREATE TABLEE DUMMY; > Error: Error while compiling statement: FAILED: ParseException line 1:7 > cannot recongize input near 'CREATE' 'TABLEE' 'DUMMY' in ddl statement > (state=42000,code=4) > The parser seems to be taking all the lines from 1 and is ignoring spaces in > the line. > The error line in the parse exception is shown as 1:7 but it should have been > 8:13. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27127) Upgrade jackson version to 2.13.5
[ https://issues.apache.org/jira/browse/HIVE-27127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-27127: -- Labels: pull-request-available (was: ) > Upgrade jackson version to 2.13.5 > - > > Key: HIVE-27127 > URL: https://issues.apache.org/jira/browse/HIVE-27127 > Project: Hive > Issue Type: Improvement >Reporter: Sand Shreeya >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27127) Upgrade jackson version to 2.13.5
[ https://issues.apache.org/jira/browse/HIVE-27127?focusedWorklogId=849771&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-849771 ] ASF GitHub Bot logged work on HIVE-27127: - Author: ASF GitHub Bot Created on: 08/Mar/23 10:03 Start Date: 08/Mar/23 10:03 Worklog Time Spent: 10m Work Description: shreeyasand opened a new pull request, #4102: URL: https://github.com/apache/hive/pull/4102 ### What changes were proposed in this pull request? Upgrade jackson version to 2.13.5 ### Why are the changes needed? To resolve CVE ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? Existing tests Issue Time Tracking --- Worklog Id: (was: 849771) Remaining Estimate: 0h Time Spent: 10m > Upgrade jackson version to 2.13.5 > - > > Key: HIVE-27127 > URL: https://issues.apache.org/jira/browse/HIVE-27127 > Project: Hive > Issue Type: Improvement >Reporter: Sand Shreeya >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HIVE-27122) Use Caffeine for caching metadata objects in Compactor threads
[ https://issues.apache.org/jira/browse/HIVE-27122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17697789#comment-17697789 ] Sourabh Badhya commented on HIVE-27122: --- Thanks [~akshatm] , [~kokila19] , [~rkirtir] , [~veghlaci05] for the reviews. > Use Caffeine for caching metadata objects in Compactor threads > -- > > Key: HIVE-27122 > URL: https://issues.apache.org/jira/browse/HIVE-27122 > Project: Hive > Issue Type: Improvement >Reporter: Sourabh Badhya >Assignee: Sourabh Badhya >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Time Spent: 3h 50m > Remaining Estimate: 0h > > Currently, compactor threads make use of Guava package to cache metadata > objects like database/table objects. We should consider using Caffeine > package since it provides more control on the cache. It is also observed that > cache created from Caffeine package is more performant than cache created > from Guava package. > Some benchmarks comparing Caffeine package vs Guava package - > [https://github.com/ben-manes/caffeine/wiki/Benchmarks] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27122) Use Caffeine for caching metadata objects in Compactor threads
[ https://issues.apache.org/jira/browse/HIVE-27122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Végh updated HIVE-27122: --- Fix Version/s: 4.0.0 Resolution: Fixed Status: Resolved (was: Patch Available) Merged to master, thanks [~sbadhya] for the patch! > Use Caffeine for caching metadata objects in Compactor threads > -- > > Key: HIVE-27122 > URL: https://issues.apache.org/jira/browse/HIVE-27122 > Project: Hive > Issue Type: Improvement >Reporter: Sourabh Badhya >Assignee: Sourabh Badhya >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Time Spent: 3h 50m > Remaining Estimate: 0h > > Currently, compactor threads make use of Guava package to cache metadata > objects like database/table objects. We should consider using Caffeine > package since it provides more control on the cache. It is also observed that > cache created from Caffeine package is more performant than cache created > from Guava package. > Some benchmarks comparing Caffeine package vs Guava package - > [https://github.com/ben-manes/caffeine/wiki/Benchmarks] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27122) Use Caffeine for caching metadata objects in Compactor threads
[ https://issues.apache.org/jira/browse/HIVE-27122?focusedWorklogId=849761&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-849761 ] ASF GitHub Bot logged work on HIVE-27122: - Author: ASF GitHub Bot Created on: 08/Mar/23 08:57 Start Date: 08/Mar/23 08:57 Worklog Time Spent: 10m Work Description: veghlaci05 merged PR #4098: URL: https://github.com/apache/hive/pull/4098 Issue Time Tracking --- Worklog Id: (was: 849761) Time Spent: 3h 50m (was: 3h 40m) > Use Caffeine for caching metadata objects in Compactor threads > -- > > Key: HIVE-27122 > URL: https://issues.apache.org/jira/browse/HIVE-27122 > Project: Hive > Issue Type: Improvement >Reporter: Sourabh Badhya >Assignee: Sourabh Badhya >Priority: Major > Labels: pull-request-available > Time Spent: 3h 50m > Remaining Estimate: 0h > > Currently, compactor threads make use of Guava package to cache metadata > objects like database/table objects. We should consider using Caffeine > package since it provides more control on the cache. It is also observed that > cache created from Caffeine package is more performant than cache created > from Guava package. > Some benchmarks comparing Caffeine package vs Guava package - > [https://github.com/ben-manes/caffeine/wiki/Benchmarks] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27101) Support incremental materialized view rebuild when Iceberg source tables have insert operation only.
[ https://issues.apache.org/jira/browse/HIVE-27101?focusedWorklogId=849760&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-849760 ] ASF GitHub Bot logged work on HIVE-27101: - Author: ASF GitHub Bot Created on: 08/Mar/23 08:47 Start Date: 08/Mar/23 08:47 Worklog Time Spent: 10m Work Description: sonarcloud[bot] commented on PR #4079: URL: https://github.com/apache/hive/pull/4079#issuecomment-1459761551 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4079) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4079&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4079&resolved=false&types=BUG) [5 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4079&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4079&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4079&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4079&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4079&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4079&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4079&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4079&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4079&resolved=false&types=CODE_SMELL) [9 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4079&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4079&metric=coverage&view=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4079&metric=duplicated_lines_density&view=list) No Duplication information Issue Time Tracking --- Worklog Id: (was: 849760) Time Spent: 7h (was: 6h 50m) > Support incremental materialized view rebuild when Iceberg source tables have > insert operation only. > > > Key: HIVE-27101 > URL: https://issues.apache.org/jira/browse/HIVE-27101 > Project: Hive > Issue Type: Improvement > Components: Iceberg integration, Materialized views >Reporter: Krisztian Kasa >Assignee: Krisztian Kasa >Priority: Major > Labels: pull-request-available > Time Spent: 7h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27117) Fix compaction related flaky tests
[ https://issues.apache.org/jira/browse/HIVE-27117?focusedWorklogId=849756&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-849756 ] ASF GitHub Bot logged work on HIVE-27117: - Author: ASF GitHub Bot Created on: 08/Mar/23 08:30 Start Date: 08/Mar/23 08:30 Worklog Time Spent: 10m Work Description: veghlaci05 merged PR #4096: URL: https://github.com/apache/hive/pull/4096 Issue Time Tracking --- Worklog Id: (was: 849756) Time Spent: 2h 40m (was: 2.5h) > Fix compaction related flaky tests > -- > > Key: HIVE-27117 > URL: https://issues.apache.org/jira/browse/HIVE-27117 > Project: Hive > Issue Type: Task >Reporter: László Végh >Assignee: László Végh >Priority: Major > Labels: pull-request-available > Time Spent: 2h 40m > Remaining Estimate: 0h > > The following tests turned out to be flaky recently: > * > org.apache.hadoop.hive.ql.TestTxnCommands2WithSplitUpdateAndVectorization.testDropTableAndCompactionConcurrent > * > org.apache.hadoop.hive.ql.txn.compactor.TestCompactionMetrics.testInitiatorFailuresCountedCorrectly > * > org.apache.hadoop.hive.ql.txn.compactor.TestCrudCompactorOnTez.testMajorCompactionNotPartitionedWithoutBuckets > * > org.apache.hadoop.hive.ql.txn.compactor.TestCrudCompactorOnTez.testCompactionWithCreateTableProps -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27117) Fix compaction related flaky tests
[
https://issues.apache.org/jira/browse/HIVE-27117?focusedWorklogId=849754&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-849754
]
ASF GitHub Bot logged work on HIVE-27117:
-
Author: ASF GitHub Bot
Created on: 08/Mar/23 08:14
Start Date: 08/Mar/23 08:14
Worklog Time Spent: 10m
Work Description: abstractdog commented on code in PR #4096:
URL: https://github.com/apache/hive/pull/4096#discussion_r1129093478
##
ql/src/test/org/apache/hadoop/hive/ql/TestTxnCommands2.java:
##
@@ -1434,6 +1434,8 @@ public void
testTruncatePartitionAndCompactionConcurrent() throws Exception {
execDDLOpAndCompactionConcurrently("TRUNCATE_PARTITION", true);
}
private void execDDLOpAndCompactionConcurrently(String opType, boolean
isPartioned) throws Exception {
+hiveConf.setBoolVar(HiveConf.ConfVars.HIVE_COMPACTOR_GATHER_STATS, false);
Review Comment:
please comment on why is this needed and then it can be merged +1
Issue Time Tracking
---
Worklog Id: (was: 849754)
Time Spent: 2.5h (was: 2h 20m)
> Fix compaction related flaky tests
> --
>
> Key: HIVE-27117
> URL: https://issues.apache.org/jira/browse/HIVE-27117
> Project: Hive
> Issue Type: Task
>Reporter: László Végh
>Assignee: László Végh
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> The following tests turned out to be flaky recently:
> *
> org.apache.hadoop.hive.ql.TestTxnCommands2WithSplitUpdateAndVectorization.testDropTableAndCompactionConcurrent
> *
> org.apache.hadoop.hive.ql.txn.compactor.TestCompactionMetrics.testInitiatorFailuresCountedCorrectly
> *
> org.apache.hadoop.hive.ql.txn.compactor.TestCrudCompactorOnTez.testMajorCompactionNotPartitionedWithoutBuckets
> *
> org.apache.hadoop.hive.ql.txn.compactor.TestCrudCompactorOnTez.testCompactionWithCreateTableProps
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27101) Support incremental materialized view rebuild when Iceberg source tables have insert operation only.
[
https://issues.apache.org/jira/browse/HIVE-27101?focusedWorklogId=849753&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-849753
]
ASF GitHub Bot logged work on HIVE-27101:
-
Author: ASF GitHub Bot
Created on: 08/Mar/23 08:00
Start Date: 08/Mar/23 08:00
Worklog Time Spent: 10m
Work Description: kasakrisz commented on code in PR #4079:
URL: https://github.com/apache/hive/pull/4079#discussion_r1129081896
##
iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/mapreduce/IcebergInputFormat.java:
##
@@ -116,6 +116,13 @@ private static TableScan createTableScan(Table table,
Configuration conf) {
scan = scan.useSnapshot(snapshotId);
}
+if (snapshotId == -1) {
+ long snapshotIntervalFrom =
conf.getLong(InputFormatConfig.SNAPSHOT_INTERVAL_FROM, -1);
+ if (snapshotIntervalFrom != -1) {
+scan = scan.appendsBetween(snapshotIntervalFrom,
table.currentSnapshot().snapshotId());
Review Comment:
Changed to use the new api but I also had to customize
org.apache.iceberg.SerializableTable because it does not support it.
Issue Time Tracking
---
Worklog Id: (was: 849753)
Time Spent: 6h 50m (was: 6h 40m)
> Support incremental materialized view rebuild when Iceberg source tables have
> insert operation only.
>
>
> Key: HIVE-27101
> URL: https://issues.apache.org/jira/browse/HIVE-27101
> Project: Hive
> Issue Type: Improvement
> Components: Iceberg integration, Materialized views
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Labels: pull-request-available
> Time Spent: 6h 50m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
