[jira] [Work logged] (HIVE-26947) Hive compactor.Worker can respawn connections to HMS at extremely high frequency
[ https://issues.apache.org/jira/browse/HIVE-26947?focusedWorklogId=839527=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-839527 ] ASF GitHub Bot logged work on HIVE-26947: - Author: ASF GitHub Bot Created on: 17/Jan/23 07:42 Start Date: 17/Jan/23 07:42 Worklog Time Spent: 10m Work Description: sonarcloud[bot] commented on PR #3955: URL: https://github.com/apache/hive/pull/3955#issuecomment-1384954113 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive=3955) [](https://sonarcloud.io/project/issues?id=apache_hive=3955=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=3955=false=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive=3955=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=3955=false=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive=3955=false=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive=3955=false=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3955=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3955=false=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3955=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive=3955=false=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive=3955=false=CODE_SMELL) [7 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive=3955=false=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive=3955=coverage=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive=3955=duplicated_lines_density=list) No Duplication information Issue Time Tracking --- Worklog Id: (was: 839527) Time Spent: 1h (was: 50m) > Hive compactor.Worker can respawn connections to HMS at extremely high > frequency > > > Key: HIVE-26947 > URL: https://issues.apache.org/jira/browse/HIVE-26947 > Project: Hive > Issue Type: Bug >Reporter: Akshat Mathur >Assignee: Akshat Mathur >Priority: Major > Labels: pull-request-available > Time Spent: 1h > Remaining Estimate: 0h > > After catching the exception generated by the findNextCompactionAndExecute() > task, HS2 appears to immediately rerun the task with no delay or backoff. As > a result there are ~3500 connection attempts from HS2 to HMS over just a 5 > second period in the HS2 log > The compactor.Worker should wait between failed attempts and maybe do an > exponential backoff. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (HIVE-26952) set the value of metastore.storage.schema.reader.impl to org.apache.hadoop.hive.metastore.SerDeStorageSchemaReader as default
[
https://issues.apache.org/jira/browse/HIVE-26952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Taraka Rama Rao Lethavadla reassigned HIVE-26952:
-
> set the value of metastore.storage.schema.reader.impl
to
> org.apache.hadoop.hive.metastore.SerDeStorageSchemaReader as default
> --
>
> Key: HIVE-26952
> URL: https://issues.apache.org/jira/browse/HIVE-26952
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore
>Reporter: Taraka Rama Rao Lethavadla
>Assignee: Taraka Rama Rao Lethavadla
>Priority: Minor
>
> With the default value of
>
> {code:java}
> DefaultStorageSchemaReader.class.getName(){code}
>
> in the Metastore Config, *metastore.storage.schema.reader.impl*
> below exception is thrown when trying to read Avro schema
> {noformat}
> Caused by: org.apache.hive.service.cli.HiveSQLException: MetaException
> (message:java.lang.UnsupportedOperationException: Storage schema reading not
> supported)
> at
> org.apache.hive.service.cli.operation.GetColumnsOperation.runInternal(GetColumnsOperation.java:213)
> at org.apache.hive.service.cli.operation.Operation.run(Operation.java:247)
> at
> org.apache.hive.service.cli.session.HiveSessionImpl.getColumns(HiveSessionImpl.java:729)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:78)
> at
> org.apache.hive.service.cli.session.HiveSessionProxy.access-zsh(HiveSessionProxy.java:36)
> at
> org.apache.hive.service.cli.session.HiveSessionProxy.run(HiveSessionProxy.java:63)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
> at
> org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:59)
> at com.sun.proxy..getColumns(Unknown Source)
> at
> org.apache.hive.service.cli.CLIService.getColumns(CLIService.java:390){noformat}
> setting the above config with
> *org.apache.hadoop.hive.metastore.SerDeStorageSchemaReader* resolves issue
> Proposing to make this value as default in code base, so that in upcoming
> versions we don't have to set this value manually
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-26932) Correct stage name value in replication_metrics.progress column in replication_metrics table
[ https://issues.apache.org/jira/browse/HIVE-26932?focusedWorklogId=839517=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-839517 ] ASF GitHub Bot logged work on HIVE-26932: - Author: ASF GitHub Bot Created on: 17/Jan/23 06:33 Start Date: 17/Jan/23 06:33 Worklog Time Spent: 10m Work Description: harshal-16 opened a new pull request, #3958: URL: https://github.com/apache/hive/pull/3958 Problem: - If Incremental Dump operation failes while dumping any event id in the staging directory. Then dump directory for this event id along with file _dumpmetadata still exists in the dump location. which is getting stored in _events_dump file - When user triggers dump operation for this policy again, It again resumes dumping from failed event id, and tries to dump it again but as that event id directory already created in previous cycle, it fails with the exception Solution: - fixed cleanFailedEventDirIfExists to remove folder for failed event id for a selected database Issue Time Tracking --- Worklog Id: (was: 839517) Time Spent: 0.5h (was: 20m) > Correct stage name value in replication_metrics.progress column in > replication_metrics table > > > Key: HIVE-26932 > URL: https://issues.apache.org/jira/browse/HIVE-26932 > Project: Hive > Issue Type: Improvement >Reporter: Harshal Patel >Assignee: Harshal Patel >Priority: Major > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > > To improve diagnostic capability from Source to backup replication, update > replication_metrics table by adding pre_optimized_bootstrap in progress bar > in case of optimized bootstrap first cycle. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26932) Correct stage name value in replication_metrics.progress column in replication_metrics table
[ https://issues.apache.org/jira/browse/HIVE-26932?focusedWorklogId=839516=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-839516 ] ASF GitHub Bot logged work on HIVE-26932: - Author: ASF GitHub Bot Created on: 17/Jan/23 06:32 Start Date: 17/Jan/23 06:32 Worklog Time Spent: 10m Work Description: harshal-16 closed pull request #3957: HIVE-26932: Cleanup dump directory for eventId which was failed in previous dump cycle URL: https://github.com/apache/hive/pull/3957 Issue Time Tracking --- Worklog Id: (was: 839516) Time Spent: 20m (was: 10m) > Correct stage name value in replication_metrics.progress column in > replication_metrics table > > > Key: HIVE-26932 > URL: https://issues.apache.org/jira/browse/HIVE-26932 > Project: Hive > Issue Type: Improvement >Reporter: Harshal Patel >Assignee: Harshal Patel >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > To improve diagnostic capability from Source to backup replication, update > replication_metrics table by adding pre_optimized_bootstrap in progress bar > in case of optimized bootstrap first cycle. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-26932) Correct stage name value in replication_metrics.progress column in replication_metrics table
[ https://issues.apache.org/jira/browse/HIVE-26932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-26932: -- Labels: pull-request-available (was: ) > Correct stage name value in replication_metrics.progress column in > replication_metrics table > > > Key: HIVE-26932 > URL: https://issues.apache.org/jira/browse/HIVE-26932 > Project: Hive > Issue Type: Improvement >Reporter: Harshal Patel >Assignee: Harshal Patel >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > To improve diagnostic capability from Source to backup replication, update > replication_metrics table by adding pre_optimized_bootstrap in progress bar > in case of optimized bootstrap first cycle. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26932) Correct stage name value in replication_metrics.progress column in replication_metrics table
[ https://issues.apache.org/jira/browse/HIVE-26932?focusedWorklogId=839515=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-839515 ] ASF GitHub Bot logged work on HIVE-26932: - Author: ASF GitHub Bot Created on: 17/Jan/23 06:28 Start Date: 17/Jan/23 06:28 Worklog Time Spent: 10m Work Description: harshal-16 opened a new pull request, #3957: URL: https://github.com/apache/hive/pull/3957 Problem: - If Incremental Dump operation failes while dumping any event id in the staging directory. Then dump directory for this event id along with file _dumpmetadata still exists in the dump location. which is getting stored in _events_dump file - When user triggers dump operation for this policy again, It again resumes dumping from failed event id, and tries to dump it again but as that event id directory already created in previous cycle, it fails with the exception Solution: - Fixed cleanFailedEventDirIfExists to remove folder for failed event id for a selected database Issue Time Tracking --- Worklog Id: (was: 839515) Remaining Estimate: 0h Time Spent: 10m > Correct stage name value in replication_metrics.progress column in > replication_metrics table > > > Key: HIVE-26932 > URL: https://issues.apache.org/jira/browse/HIVE-26932 > Project: Hive > Issue Type: Improvement >Reporter: Harshal Patel >Assignee: Harshal Patel >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > To improve diagnostic capability from Source to backup replication, update > replication_metrics table by adding pre_optimized_bootstrap in progress bar > in case of optimized bootstrap first cycle. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26711) The very first REPL Load should make the Target Database read-only
[ https://issues.apache.org/jira/browse/HIVE-26711?focusedWorklogId=839514=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-839514 ] ASF GitHub Bot logged work on HIVE-26711: - Author: ASF GitHub Bot Created on: 17/Jan/23 06:27 Start Date: 17/Jan/23 06:27 Worklog Time Spent: 10m Work Description: sonarcloud[bot] commented on PR #3736: URL: https://github.com/apache/hive/pull/3736#issuecomment-1384892704 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive=3736) [](https://sonarcloud.io/project/issues?id=apache_hive=3736=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=3736=false=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive=3736=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=3736=false=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive=3736=false=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive=3736=false=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3736=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3736=false=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3736=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive=3736=false=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive=3736=false=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive=3736=false=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive=3736=coverage=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive=3736=duplicated_lines_density=list) No Duplication information Issue Time Tracking --- Worklog Id: (was: 839514) Time Spent: 1.5h (was: 1h 20m) > The very first REPL Load should make the Target Database read-only > -- > > Key: HIVE-26711 > URL: https://issues.apache.org/jira/browse/HIVE-26711 > Project: Hive > Issue Type: Task >Reporter: Shreenidhi >Assignee: Shreenidhi >Priority: Major > Labels: pull-request-available > Time Spent: 1.5h > Remaining Estimate: 0h > > Use EnforceReadOnly hook to set TARGET database read only during BootStrap > load. > Also ensure backward compatibility. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-10476) Hive query should fail when it fails to initialize a session in SetSparkReducerParallelism [Spark Branch]
[ https://issues.apache.org/jira/browse/HIVE-10476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenping Zhan updated HIVE-10476: Summary: Hive query should fail when it fails to initialize a session in SetSparkReducerParallelism [Spark Branch] (was: Hive query should fail when it fails to initialize a session in SetSparkReducerParallelism [Spark Branch]) > Hive query should fail when it fails to initialize a session in > SetSparkReducerParallelism [Spark Branch] > - > > Key: HIVE-10476 > URL: https://issues.apache.org/jira/browse/HIVE-10476 > Project: Hive > Issue Type: Sub-task > Components: Spark >Affects Versions: spark-branch >Reporter: Chao Sun >Assignee: Chao Sun >Priority: Minor > Fix For: 1.3.0, 2.0.0 > > Attachments: HIVE-10476.1-spark.patch, HIVE-10476.2-spark.patch > > > Currently, for a Hive query HoS need to get a session > a session twice, once in SparkSetReducerParallelism, and another when > submitting the actual job. > The issue is that sometimes there's problem when launching a Yarn application > (e.g., don't have permission), then user will have to wait for two timeouts, > because both session initializations will fail. This turned out to happen > frequently. > This JIRA proposes to fail the query in SparkSetReducerParallelism, when it > cannot initialize the session. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-10476) Hive query should fail when it fails to initialize a session in SetSparkReducerParallelism [Spark Branch]1111
[ https://issues.apache.org/jira/browse/HIVE-10476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenping Zhan updated HIVE-10476: Summary: Hive query should fail when it fails to initialize a session in SetSparkReducerParallelism [Spark Branch] (was: Hive query should fail when it fails to initialize a session in SetSparkReducerParallelism [Spark Branch]) > Hive query should fail when it fails to initialize a session in > SetSparkReducerParallelism [Spark Branch] > - > > Key: HIVE-10476 > URL: https://issues.apache.org/jira/browse/HIVE-10476 > Project: Hive > Issue Type: Sub-task > Components: Spark >Affects Versions: spark-branch >Reporter: Chao Sun >Assignee: Chao Sun >Priority: Minor > Fix For: 1.3.0, 2.0.0 > > Attachments: HIVE-10476.1-spark.patch, HIVE-10476.2-spark.patch > > > Currently, for a Hive query HoS need to get a session > a session twice, once in SparkSetReducerParallelism, and another when > submitting the actual job. > The issue is that sometimes there's problem when launching a Yarn application > (e.g., don't have permission), then user will have to wait for two timeouts, > because both session initializations will fail. This turned out to happen > frequently. > This JIRA proposes to fail the query in SparkSetReducerParallelism, when it > cannot initialize the session. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HIVE-26606) Expose failover states in replication metrics
[ https://issues.apache.org/jira/browse/HIVE-26606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17677610#comment-17677610 ] Harshal Patel commented on HIVE-26606: -- Hello [~teddy.choi] I have created PR, please review it https://github.com/apache/hive/pull/3956/commits/bd73086ece2a99b4a82fbe9a86c61f88ebb167b3 > Expose failover states in replication metrics > - > > Key: HIVE-26606 > URL: https://issues.apache.org/jira/browse/HIVE-26606 > Project: Hive > Issue Type: Improvement >Reporter: Teddy Choi >Assignee: Harshal Patel >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > Expose the state of failover in replication metrics, -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (HIVE-26581) Test failing on aarch64
[
https://issues.apache.org/jira/browse/HIVE-26581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17677609#comment-17677609
]
odidev edited comment on HIVE-26581 at 1/17/23 5:43 AM:
I ran the test case again on the latest commit and now *ql* module test is
failing on amd64 and arm64 platforms with the below error:
{code:java}
[^[[1;34mINFO^[[m] Running
org.apache.hive.testutils.^[[1mTestHiveTestEnvSetup^[[m
[^[[1;34mINFO^[[m] ^[[1;32mTests run: ^[[0;1;32m1^[[m, Failures: 0, Errors: 0,
Skipped: 0, Time elapsed: 72.23 s - in
org.apache.hive.testutils.^[[1mTestHiveTestEnvSetup^[[m
[^[[1;34mINFO^[[m]
[^[[1;34mINFO^[[m] Results:
[^[[1;34mINFO^[[m]
[^[[1;31mERROR^[[m] ^[[1;31mFailures: ^[[m
[^[[1;31mERROR^[[m] ^[[1;31mTestSessionState.testCreatePath^[[m
[^[[1;31mERROR^[[m] ^[[1;31m Run 1: did not get expected exception creating a
child of a directory with no permissions^[[m
[^[[1;31mERROR^[[m] ^[[1;31m Run 2: did not get expected exception creating a
child of a directory with no permissions^[[m
[^[[1;34mINFO^[[m]
[^[[1;31mERROR^[[m] ^[[1;31m
TestGenericUDFToUnixTimestamp.testStringArg2:167->runAndVerify:70
expected:<-17984790[40]0> but was:<-17984790[39]0>^[[m
[^[[1;31mERROR^[[m] ^[[1;31mErrors: ^[[m
[^[[1;31mERROR^[[m] ^[[1;31mTestSessionState.testReloadAuxJars2^[[m
[^[[1;31mERROR^[[m] ^[[1;31m Run 1: RefreshedJarClass has been compiled by a
more recent version of the Java Runtime (class file version 55.0), this version
of the Java Runtime only recognizes class file versions up to 52.0^[[m
[^[[1;31mERROR^[[m] ^[[1;31m Run 2: RefreshedJarClass has been compiled by a
more recent version of the Java Runtime (class file version 55.0), this version
of the Java Runtime only recognizes class file versions up to 52.0^[[m
[^[[1;34mINFO^[[m]
[^[[1;31mERROR^[[m] ^[[1;31mTestSessionState.testReloadExistingAuxJars2^[[m
[^[[1;31mERROR^[[m] ^[[1;31m Run 1: RefreshedJarClass has been compiled by a
more recent version of the Java Runtime (class file version 55.0), this version
of the Java Runtime only recognizes class file versions up to 52.0^[[m
[^[[1;31mERROR^[[m] ^[[1;31m Run 2: RefreshedJarClass has been compiled by a
more recent version of the Java Runtime (class file version 55.0), this version
of the Java Runtime only recognizes class file versions up to 52.0^[[m
[^[[1;34mINFO^[[m]
[^[[1;34mINFO^[[m]
[^[[1;31mERROR^[[m] ^[[1;31mTests run: 32929, Failures: 2, Errors: 2, Skipped:
87^[[m
[^[[1;34mINFO^[[m]
[^[[1;31mERROR^[[m] There are test failures.
{code}
I have provided proper permissions to all folders but still I am facing above
issue.
Can you please suggest some pointers on the above error as it is giving the
same error on amd64 platform as well
was (Author: odidev):
I ran the test case again on the latest commit and now* ql *module test is
failing on amd64 and arm64 platforms with the below error:
{code:java}
[^[[1;34mINFO^[[m] Running
org.apache.hive.testutils.^[[1mTestHiveTestEnvSetup^[[m
[^[[1;34mINFO^[[m] ^[[1;32mTests run: ^[[0;1;32m1^[[m, Failures: 0, Errors: 0,
Skipped: 0, Time elapsed: 72.23 s - in
org.apache.hive.testutils.^[[1mTestHiveTestEnvSetup^[[m
[^[[1;34mINFO^[[m]
[^[[1;34mINFO^[[m] Results:
[^[[1;34mINFO^[[m]
[^[[1;31mERROR^[[m] ^[[1;31mFailures: ^[[m
[^[[1;31mERROR^[[m] ^[[1;31mTestSessionState.testCreatePath^[[m
[^[[1;31mERROR^[[m] ^[[1;31m Run 1: did not get expected exception creating a
child of a directory with no permissions^[[m
[^[[1;31mERROR^[[m] ^[[1;31m Run 2: did not get expected exception creating a
child of a directory with no permissions^[[m
[^[[1;34mINFO^[[m]
[^[[1;31mERROR^[[m] ^[[1;31m
TestGenericUDFToUnixTimestamp.testStringArg2:167->runAndVerify:70
expected:<-17984790[40]0> but was:<-17984790[39]0>^[[m
[^[[1;31mERROR^[[m] ^[[1;31mErrors: ^[[m
[^[[1;31mERROR^[[m] ^[[1;31mTestSessionState.testReloadAuxJars2^[[m
[^[[1;31mERROR^[[m] ^[[1;31m Run 1: RefreshedJarClass has been compiled by a
more recent version of the Java Runtime (class file version 55.0), this version
of the Java Runtime only recognizes class file versions up to 52.0^[[m
[^[[1;31mERROR^[[m] ^[[1;31m Run 2: RefreshedJarClass has been compiled by a
more recent version of the Java Runtime (class file version 55.0), this version
of the Java Runtime only recognizes class file versions up to 52.0^[[m
[^[[1;34mINFO^[[m]
[^[[1;31mERROR^[[m] ^[[1;31mTestSessionState.testReloadExistingAuxJars2^[[m
[^[[1;31mERROR^[[m] ^[[1;31m Run 1: RefreshedJarClass has been compiled by a
more recent version of the Java Runtime (class file version 55.0), this version
of the Java Runtime only recognizes class file versions up to 52.0^[[m
[^[[1;31mERROR^[[m] ^[[1;31m Run 2: RefreshedJarClass has been compiled by a
more recent version of the Java Runtime (class file version 55.0), this version
of the Java Runtime only
[jira] [Work logged] (HIVE-26606) Expose failover states in replication metrics
[ https://issues.apache.org/jira/browse/HIVE-26606?focusedWorklogId=839512=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-839512 ] ASF GitHub Bot logged work on HIVE-26606: - Author: ASF GitHub Bot Created on: 17/Jan/23 05:40 Start Date: 17/Jan/23 05:40 Worklog Time Spent: 10m Work Description: harshal-16 opened a new pull request, #3956: URL: https://github.com/apache/hive/pull/3956 * Added 2 New replication Types: 1. Pre Optimized BootStrap : 1st cycle of reverse replication 2. Optimized Bootstrap : 2nd cycle of reverse replication * Added both types into replication metric * Added unit test for corresponding changes Issue Time Tracking --- Worklog Id: (was: 839512) Remaining Estimate: 0h Time Spent: 10m > Expose failover states in replication metrics > - > > Key: HIVE-26606 > URL: https://issues.apache.org/jira/browse/HIVE-26606 > Project: Hive > Issue Type: Improvement >Reporter: Teddy Choi >Assignee: Harshal Patel >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > Expose the state of failover in replication metrics, -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-26606) Expose failover states in replication metrics
[ https://issues.apache.org/jira/browse/HIVE-26606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-26606: -- Labels: pull-request-available (was: ) > Expose failover states in replication metrics > - > > Key: HIVE-26606 > URL: https://issues.apache.org/jira/browse/HIVE-26606 > Project: Hive > Issue Type: Improvement >Reporter: Teddy Choi >Assignee: Harshal Patel >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > Expose the state of failover in replication metrics, -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HIVE-26581) Test failing on aarch64
[
https://issues.apache.org/jira/browse/HIVE-26581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17677609#comment-17677609
]
odidev commented on HIVE-26581:
---
I ran the test case again on the latest commit and now* ql *module test is
failing on amd64 and arm64 platforms with the below error:
{code:java}
[^[[1;34mINFO^[[m] Running
org.apache.hive.testutils.^[[1mTestHiveTestEnvSetup^[[m
[^[[1;34mINFO^[[m] ^[[1;32mTests run: ^[[0;1;32m1^[[m, Failures: 0, Errors: 0,
Skipped: 0, Time elapsed: 72.23 s - in
org.apache.hive.testutils.^[[1mTestHiveTestEnvSetup^[[m
[^[[1;34mINFO^[[m]
[^[[1;34mINFO^[[m] Results:
[^[[1;34mINFO^[[m]
[^[[1;31mERROR^[[m] ^[[1;31mFailures: ^[[m
[^[[1;31mERROR^[[m] ^[[1;31mTestSessionState.testCreatePath^[[m
[^[[1;31mERROR^[[m] ^[[1;31m Run 1: did not get expected exception creating a
child of a directory with no permissions^[[m
[^[[1;31mERROR^[[m] ^[[1;31m Run 2: did not get expected exception creating a
child of a directory with no permissions^[[m
[^[[1;34mINFO^[[m]
[^[[1;31mERROR^[[m] ^[[1;31m
TestGenericUDFToUnixTimestamp.testStringArg2:167->runAndVerify:70
expected:<-17984790[40]0> but was:<-17984790[39]0>^[[m
[^[[1;31mERROR^[[m] ^[[1;31mErrors: ^[[m
[^[[1;31mERROR^[[m] ^[[1;31mTestSessionState.testReloadAuxJars2^[[m
[^[[1;31mERROR^[[m] ^[[1;31m Run 1: RefreshedJarClass has been compiled by a
more recent version of the Java Runtime (class file version 55.0), this version
of the Java Runtime only recognizes class file versions up to 52.0^[[m
[^[[1;31mERROR^[[m] ^[[1;31m Run 2: RefreshedJarClass has been compiled by a
more recent version of the Java Runtime (class file version 55.0), this version
of the Java Runtime only recognizes class file versions up to 52.0^[[m
[^[[1;34mINFO^[[m]
[^[[1;31mERROR^[[m] ^[[1;31mTestSessionState.testReloadExistingAuxJars2^[[m
[^[[1;31mERROR^[[m] ^[[1;31m Run 1: RefreshedJarClass has been compiled by a
more recent version of the Java Runtime (class file version 55.0), this version
of the Java Runtime only recognizes class file versions up to 52.0^[[m
[^[[1;31mERROR^[[m] ^[[1;31m Run 2: RefreshedJarClass has been compiled by a
more recent version of the Java Runtime (class file version 55.0), this version
of the Java Runtime only recognizes class file versions up to 52.0^[[m
[^[[1;34mINFO^[[m]
[^[[1;34mINFO^[[m]
[^[[1;31mERROR^[[m] ^[[1;31mTests run: 32929, Failures: 2, Errors: 2, Skipped:
87^[[m
[^[[1;34mINFO^[[m]
[^[[1;31mERROR^[[m] There are test failures.
{code}
I have provided proper permissions to all folders but still I am facing above
issue.
Can you please suggest some pointers on the above error as it is giving the
same error on amd64 platform as well
> Test failing on aarch64
> ---
>
> Key: HIVE-26581
> URL: https://issues.apache.org/jira/browse/HIVE-26581
> Project: Hive
> Issue Type: Bug
>Reporter: odidev
>Priority: Major
>
> Hi Team,
> I tried to build and test the Apache hive repository on an aarch64 machine
> but when I run *mvn clean install* it is giving me the following error:
> {code:java}
> [ERROR] Tests run: 2, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 2.265
> s <<< FAILURE! - in
> org.apache.hadoop.hive.llap.tezplugins.TestLlapTaskCommunicator
> [ERROR]
> org.apache.hadoop.hive.llap.tezplugins.TestLlapTaskCommunicator.testFinishableStateUpdateFailure
> Time elapsed: 2.206 s <<< ERROR!
> java.lang.NullPointerException
> at
> org.apache.hadoop.hive.llap.daemon.rpc.LlapDaemonProtocolProtos$SignableVertexSpec$Builder.setUser(LlapDaemonProtocolProtos.java:5513)
> at
> org.apache.hadoop.hive.llap.tez.Converters.constructSignableVertexSpec(Converters.java:135)
> at
> org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator.constructSubmitWorkRequest(LlapTaskCommunicator.java:912)
> at
> org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator.registerRunningTaskAttempt(LlapTaskCommunicator.java:512)
> at
> org.apache.hadoop.hive.llap.tezplugins.TestLlapTaskCommunicator$LlapTaskCommunicatorWrapperForTest.registerRunningTaskAttemptWithSourceVertex(TestLlapTaskCommunicator.java:335)
> at
> org.apache.hadoop.hive.llap.tezplugins.TestLlapTaskCommunicator.testFinishableStateUpdateFailure(TestLlapTaskCommunicator.java:141)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> at
>
[jira] [Work logged] (HIVE-26035) Explore moving to directsql for ObjectStore::addPartitions
[
https://issues.apache.org/jira/browse/HIVE-26035?focusedWorklogId=839510=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-839510
]
ASF GitHub Bot logged work on HIVE-26035:
-
Author: ASF GitHub Bot
Created on: 17/Jan/23 05:38
Start Date: 17/Jan/23 05:38
Worklog Time Spent: 10m
Work Description: VenuReddy2103 commented on code in PR #3905:
URL: https://github.com/apache/hive/pull/3905#discussion_r1071755609
##
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java:
##
@@ -2607,6 +2614,9 @@ public boolean addPartitions(String catName, String
dbName, String tblName, List
tabColumnGrants = this.listTableAllColumnGrants(catName, dbName,
tblName);
}
List toPersist = new ArrayList<>();
Review Comment:
ok
Issue Time Tracking
---
Worklog Id: (was: 839510)
Time Spent: 2h 10m (was: 2h)
> Explore moving to directsql for ObjectStore::addPartitions
> --
>
> Key: HIVE-26035
> URL: https://issues.apache.org/jira/browse/HIVE-26035
> Project: Hive
> Issue Type: Bug
>Reporter: Rajesh Balamohan
>Assignee: Venugopal Reddy K
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> Currently {{addPartitions}} uses datanuclues and is super slow for large
> number of partitions. It will be good to move to direct sql. Lots of repeated
> SQLs can be avoided as well (e.g SDS, SERDE, TABLE_PARAMS)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-26035) Explore moving to directsql for ObjectStore::addPartitions
[
https://issues.apache.org/jira/browse/HIVE-26035?focusedWorklogId=839511=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-839511
]
ASF GitHub Bot logged work on HIVE-26035:
-
Author: ASF GitHub Bot
Created on: 17/Jan/23 05:38
Start Date: 17/Jan/23 05:38
Worklog Time Spent: 10m
Work Description: VenuReddy2103 commented on code in PR #3905:
URL: https://github.com/apache/hive/pull/3905#discussion_r1071755797
##
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java:
##
@@ -515,6 +529,803 @@ public List
getMaterializedViewsForRewriting(String dbName) throws MetaE
}
}
+ private Long getDataStoreId(Class modelClass) throws MetaException {
+ExecutionContext ec = ((JDOPersistenceManager) pm).getExecutionContext();
+AbstractClassMetaData cmd =
ec.getMetaDataManager().getMetaDataForClass(modelClass,
ec.getClassLoaderResolver());
+if (cmd.getIdentityType() == IdentityType.DATASTORE) {
+ return (Long) ec.getStoreManager().getValueGenerationStrategyValue(ec,
cmd, -1);
+} else {
+ throw new MetaException("Identity type is not datastore.");
+}
+ }
+
+ /**
+ * Interface to execute multiple row insert query in batch for direct SQL
+ */
+ interface BatchExecutionContext {
+void execute(String batchQueryText, int batchRowCount, int
batchParamCount) throws MetaException;
+ }
+
+ private void insertInBatch(String tableName, String columns, int
columnCount, String rowFormat, int rowCount,
+ BatchExecutionContext bec) throws MetaException {
+if (rowCount == 0 || columnCount == 0) {
+ return;
+}
+int maxParamsCount = maxParamsInInsert;
+if (maxParamsCount < columnCount) {
+ LOG.error("Maximum number of parameters in the direct SQL batch insert
query is less than the table: {}"
+ + " columns. Executing single row insert queries.", tableName);
+ maxParamsCount = columnCount;
+}
+int maxRowsInBatch = maxParamsCount / columnCount;
+int maxBatches = rowCount / maxRowsInBatch;
+int last = rowCount % maxRowsInBatch;
+String query = "";
+if (maxBatches > 0) {
+ query = dbType.getBatchInsertQuery(tableName, columns, rowFormat,
maxRowsInBatch);
+}
+int batchParamCount = maxRowsInBatch * columnCount;
+for (int batch = 0; batch < maxBatches; batch++) {
+ bec.execute(query, maxRowsInBatch, batchParamCount);
+}
+if (last != 0) {
+ query = dbType.getBatchInsertQuery(tableName, columns, rowFormat, last);
+ bec.execute(query, last, last * columnCount);
+}
+ }
+
+ private void insertSerdeInBatch(Map serdeIdToSerDeInfo)
throws MetaException {
+int rowCount = serdeIdToSerDeInfo.size();
+String columns =
"(\"SERDE_ID\",\"DESCRIPTION\",\"DESERIALIZER_CLASS\",\"NAME\",\"SERDE_TYPE\",\"SLIB\","
++ "\"SERIALIZER_CLASS\")";
+String row = "(?,?,?,?,?,?,?)";
+int columnCount = 7;
+BatchExecutionContext bec = new BatchExecutionContext() {
+ final Iterator> it =
serdeIdToSerDeInfo.entrySet().iterator();
+ @Override
+ public void execute(String batchQueryText, int batchRowCount, int
batchParamCount) throws MetaException {
+Object[] params = new Object[batchParamCount];
+int paramIndex = 0;
+for (int index = 0; index < batchRowCount; index++) {
+ Map.Entry entry = it.next();
+ MSerDeInfo serdeInfo = entry.getValue();
+ params[paramIndex++] = entry.getKey();
+ params[paramIndex++] = serdeInfo.getDescription();
+ params[paramIndex++] = serdeInfo.getDeserializerClass();
+ params[paramIndex++] = serdeInfo.getName();
+ params[paramIndex++] = serdeInfo.getSerdeType();
+ params[paramIndex++] = serdeInfo.getSerializationLib();
+ params[paramIndex++] = serdeInfo.getSerializerClass();
+}
+try (QueryWrapper query = new
QueryWrapper(pm.newQuery("javax.jdo.query.SQL", batchQueryText))) {
+ executeWithArray(query.getInnerQuery(), params, batchQueryText);
+}
+ }
+};
+insertInBatch(SERDES, columns, columnCount, row, rowCount, bec);
+ }
+
+ private void insertStorageDescriptorInBatch(Map
sdIdToStorageDescriptor,
+ Map sdIdToSerdeId, Map sdIdToCdId) throws
MetaException {
+int rowCount = sdIdToStorageDescriptor.size();
+String columns =
"(\"SD_ID\",\"CD_ID\",\"INPUT_FORMAT\",\"IS_COMPRESSED\",\"IS_STOREDASSUBDIRECTORIES\","
++ "\"LOCATION\",\"NUM_BUCKETS\",\"OUTPUT_FORMAT\",\"SERDE_ID\")";
+String row = "(?,?,?,?,?,?,?,?,?)";
+int columnCount = 9;
+BatchExecutionContext bec = new BatchExecutionContext() {
+ final Iterator> it =
sdIdToStorageDescriptor.entrySet().iterator();
+ @Override
+ public void execute(String batchQueryText, int batchRowCount, int
[jira] [Work logged] (HIVE-26809) Upgrade ORC to 1.8.1
[
https://issues.apache.org/jira/browse/HIVE-26809?focusedWorklogId=839509=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-839509
]
ASF GitHub Bot logged work on HIVE-26809:
-
Author: ASF GitHub Bot
Created on: 17/Jan/23 05:27
Start Date: 17/Jan/23 05:27
Worklog Time Spent: 10m
Work Description: ayushtkn commented on code in PR #3833:
URL: https://github.com/apache/hive/pull/3833#discussion_r1071751361
##
ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/EncodedTreeReaderFactory.java:
##
@@ -224,7 +237,252 @@ private static void skipCompressedIndex(boolean
isCompressed, PositionProvider i
index.getNext();
}
- protected static class StringStreamReader extends StringTreeReader
+ public static class StringDictionaryTreeReaderHive extends TreeReader {
Review Comment:
Ok, that seems to be an improvement or say a bug fix in the ORC project and
we are just implementing our own varient because now the original class is
causing test failures.
This isn't the ideal approach and will backfire in future when we try to
upgrade and the changes in ORC depends on the ones which we ditched.
We should try to adapt to those changes and make sure we don't crash with
those changes in Hive by making hive changes, rather than maintaining a old
version of ORC class at Hive
Issue Time Tracking
---
Worklog Id: (was: 839509)
Time Spent: 5h (was: 4h 50m)
> Upgrade ORC to 1.8.1
>
>
> Key: HIVE-26809
> URL: https://issues.apache.org/jira/browse/HIVE-26809
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 4.0.0
>Reporter: Dmitriy Fingerman
>Assignee: Dmitriy Fingerman
>Priority: Major
> Labels: pull-request-available
> Time Spent: 5h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-26925) MV with iceberg storage format fails when contains 'PARTITIONED ON' clause due to column number/types difference.
[
https://issues.apache.org/jira/browse/HIVE-26925?focusedWorklogId=839508=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-839508
]
ASF GitHub Bot logged work on HIVE-26925:
-
Author: ASF GitHub Bot
Created on: 17/Jan/23 05:22
Start Date: 17/Jan/23 05:22
Worklog Time Spent: 10m
Work Description: sonarcloud[bot] commented on PR #3939:
URL: https://github.com/apache/hive/pull/3939#issuecomment-1384848069
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive=3939)
[](https://sonarcloud.io/project/issues?id=apache_hive=3939=false=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive=3939=false=BUG)
[0
Bugs](https://sonarcloud.io/project/issues?id=apache_hive=3939=false=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive=3939=false=VULNERABILITY)
[](https://sonarcloud.io/project/issues?id=apache_hive=3939=false=VULNERABILITY)
[0
Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive=3939=false=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3939=false=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3939=false=SECURITY_HOTSPOT)
[0 Security
Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3939=false=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive=3939=false=CODE_SMELL)
[](https://sonarcloud.io/project/issues?id=apache_hive=3939=false=CODE_SMELL)
[0 Code
Smells](https://sonarcloud.io/project/issues?id=apache_hive=3939=false=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive=3939=coverage=list)
No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive=3939=duplicated_lines_density=list)
No Duplication information
Issue Time Tracking
---
Worklog Id: (was: 839508)
Time Spent: 1h 10m (was: 1h)
> MV with iceberg storage format fails when contains 'PARTITIONED ON' clause
> due to column number/types difference.
> -
>
> Key: HIVE-26925
> URL: https://issues.apache.org/jira/browse/HIVE-26925
> Project: Hive
> Issue Type: Bug
> Components: Iceberg integration
>Reporter: Dharmik Thakkar
>Assignee: Krisztian Kasa
>Priority: Critical
> Labels: pull-request-available
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> MV with iceberg storage format fails when contains 'PARTITIONED ON' clause
> due to column number/types difference.
> {code:java}
> !!! annotations iceberg
> >>> use iceberg_test_db_hive;
> No rows affected
> >>> set hive.exec.max.dynamic.partitions=2000;
> >>> set hive.exec.max.dynamic.partitions.pernode=2000;
> >>> drop materialized view if exists mv_agg_gby_col_partitioned;
> >>> create materialized view mv_agg_gby_col_partitioned PARTITIONED ON (t)
> >>> stored by iceberg stored as orc tblproperties ('format-version'='1') as
> >>> select b,f,sum(b), sum(f),t from all100k group by b,f,v,c,t;
> >>> analyze table mv_agg_gby_col_partitioned compute statistics for columns;
> >>> set hive.explain.user=false;
> >>> explain select b,f,sum(b) from all100k where t=93 group by c,v,f,b;
> !!! match row_contains
>
[jira] [Work logged] (HIVE-26825) Compactor: Cleaner shouldn't fetch table details again and again for partitioned tables
[
https://issues.apache.org/jira/browse/HIVE-26825?focusedWorklogId=839506=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-839506
]
ASF GitHub Bot logged work on HIVE-26825:
-
Author: ASF GitHub Bot
Created on: 17/Jan/23 05:02
Start Date: 17/Jan/23 05:02
Worklog Time Spent: 10m
Work Description: rkirtir commented on code in PR #3864:
URL: https://github.com/apache/hive/pull/3864#discussion_r1071738311
##
ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/MetaStoreCompactorThread.java:
##
@@ -133,4 +141,27 @@ protected static long updateCycleDurationMetric(String
metric, long startedAt) {
}
return 0;
}
+
+ > T computeIfAbsent(String key, Callable callable)
throws Exception {
+if (metaCache.isPresent()) {
+ try {
+return (T) metaCache.get().get(key, callable);
+ } catch (ExecutionException e) {
+throw (Exception) e.getCause();
+ }
+}
+return callable.call();
+ }
+
+ Optional> initializeCache(boolean tableCacheOn) {
+if (tableCacheOn) {
+ metaCache = Optional.of(CacheBuilder.newBuilder().softValues().build());
+}
+return metaCache;
+ }
+
+ protected void invalidateMetaCache(){
Review Comment:
done
Issue Time Tracking
---
Worklog Id: (was: 839506)
Time Spent: 2h 40m (was: 2.5h)
> Compactor: Cleaner shouldn't fetch table details again and again for
> partitioned tables
> ---
>
> Key: HIVE-26825
> URL: https://issues.apache.org/jira/browse/HIVE-26825
> Project: Hive
> Issue Type: Improvement
> Components: Transactions
>Reporter: KIRTI RUGE
>Assignee: KIRTI RUGE
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2h 40m
> Remaining Estimate: 0h
>
> Cleaner shouldn't be fetch table/partition details for all its partitions.
> When there are large number of databases/tables, it takes lot of time for
> Initiator to complete its initial iteration and load on DB also goes higher.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HIVE-26748) Prepare for Hive 3.2.0 Release
[ https://issues.apache.org/jira/browse/HIVE-26748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17677598#comment-17677598 ] Aman Raj commented on HIVE-26748: - Perfect [~vihangk1] Thanks for contributing. > Prepare for Hive 3.2.0 Release > -- > > Key: HIVE-26748 > URL: https://issues.apache.org/jira/browse/HIVE-26748 > Project: Hive > Issue Type: Task > Components: Hive >Affects Versions: 3.1.3 >Reporter: Aman Raj >Assignee: Aman Raj >Priority: Major > Labels: hive-3.2.0-must > > This is the Umbrella Jira to track all the commits that would go on top of > current branch-3 in this new 3.2.0 Hive release. I will add all the JIRAs > that will be cherry picked as part of this commit by defining subtasks or > linking the JIRAs. > > *Please note that this is an Open forum and I welcome all responses for the > same from the community with regards to any new bug fixes that should be > cherry picked.* -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26943) Fix NPE during Optimised Bootstrap when db is dropped
[ https://issues.apache.org/jira/browse/HIVE-26943?focusedWorklogId=839503=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-839503 ] ASF GitHub Bot logged work on HIVE-26943: - Author: ASF GitHub Bot Created on: 17/Jan/23 03:58 Start Date: 17/Jan/23 03:58 Worklog Time Spent: 10m Work Description: shreenidhiSaigaonkar commented on PR #3953: URL: https://github.com/apache/hive/pull/3953#issuecomment-1384801926 @pudidic Need review on this too Issue Time Tracking --- Worklog Id: (was: 839503) Time Spent: 0.5h (was: 20m) > Fix NPE during Optimised Bootstrap when db is dropped > - > > Key: HIVE-26943 > URL: https://issues.apache.org/jira/browse/HIVE-26943 > Project: Hive > Issue Type: Task >Reporter: Shreenidhi >Assignee: Shreenidhi >Priority: Major > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > > Consider the steps: > 1. Current replication is from A (source) -> B(target) > 2. Failover is complete > so now A (target) <- B(source) > 3. Suppose db at A is dropped before reverse replication. > 4. Now when reverse replication triggers optimised bootstrap it will throw NPE > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26942) Prevent updation of "repl.target.last.id" when RESUME action initiated
[ https://issues.apache.org/jira/browse/HIVE-26942?focusedWorklogId=839502=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-839502 ] ASF GitHub Bot logged work on HIVE-26942: - Author: ASF GitHub Bot Created on: 17/Jan/23 03:58 Start Date: 17/Jan/23 03:58 Worklog Time Spent: 10m Work Description: shreenidhiSaigaonkar commented on PR #3951: URL: https://github.com/apache/hive/pull/3951#issuecomment-1384801756 @pudidic need review on this Issue Time Tracking --- Worklog Id: (was: 839502) Time Spent: 0.5h (was: 20m) > Prevent updation of "repl.target.last.id" when RESUME action initiated > -- > > Key: HIVE-26942 > URL: https://issues.apache.org/jira/browse/HIVE-26942 > Project: Hive > Issue Type: Task >Reporter: Shreenidhi >Assignee: Shreenidhi >Priority: Major > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > > *Problem :* > Currently when "repl.last.id" get's changed/updated this updates the > corresponding "repl.target.last.id" too. This is problematic incase of resume > because we are updating both the properties at source. > > *Solution :* > To prevent the updation of "repl.target.last.id" whenever "repl.last.id" > changed, we can introduce new db param called "repl.resume.started". This new > param will be set to "true" by orchestrator when RESUME is initiated. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HIVE-26911) Renaming a translated external table with a specified location fails with 'location already exists' exception
[
https://issues.apache.org/jira/browse/HIVE-26911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhihua Deng resolved HIVE-26911.
Fix Version/s: 4.0.0
Resolution: Fixed
> Renaming a translated external table with a specified location fails with
> 'location already exists' exception
> -
>
> Key: HIVE-26911
> URL: https://issues.apache.org/jira/browse/HIVE-26911
> Project: Hive
> Issue Type: Bug
> Components: Standalone Metastore
>Affects Versions: 4.0.0
>Reporter: Sai Hemanth Gantasala
>Assignee: Sai Hemanth Gantasala
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Time Spent: 3h
> Remaining Estimate: 0h
>
> Renaming a translated external table with a specified location fails with
> 'location already exists' exception.
> Below are steps for repro
> {code:java}
> create database tmp;
> use tmp;
> create table b(s string) stored as parquet location
> 'hdfs://localhost:20500/test-warehouse/tmp.db/some_location';
> alter table b rename to bb;
> ERROR: InvalidOperationException: New location for this table hive.tmp.bb
> already exists : hdfs://localhost:20500/test-warehouse/tmp.db/some_location
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HIVE-26911) Renaming a translated external table with a specified location fails with 'location already exists' exception
[
https://issues.apache.org/jira/browse/HIVE-26911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17677589#comment-17677589
]
Zhihua Deng commented on HIVE-26911:
Fix has been merged into master. Thank you [~hemanth619] for the PR.
> Renaming a translated external table with a specified location fails with
> 'location already exists' exception
> -
>
> Key: HIVE-26911
> URL: https://issues.apache.org/jira/browse/HIVE-26911
> Project: Hive
> Issue Type: Bug
> Components: Standalone Metastore
>Affects Versions: 4.0.0
>Reporter: Sai Hemanth Gantasala
>Assignee: Sai Hemanth Gantasala
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Time Spent: 3h
> Remaining Estimate: 0h
>
> Renaming a translated external table with a specified location fails with
> 'location already exists' exception.
> Below are steps for repro
> {code:java}
> create database tmp;
> use tmp;
> create table b(s string) stored as parquet location
> 'hdfs://localhost:20500/test-warehouse/tmp.db/some_location';
> alter table b rename to bb;
> ERROR: InvalidOperationException: New location for this table hive.tmp.bb
> already exists : hdfs://localhost:20500/test-warehouse/tmp.db/some_location
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-26911) Renaming a translated external table with a specified location fails with 'location already exists' exception
[
https://issues.apache.org/jira/browse/HIVE-26911?focusedWorklogId=839499=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-839499
]
ASF GitHub Bot logged work on HIVE-26911:
-
Author: ASF GitHub Bot
Created on: 17/Jan/23 03:49
Start Date: 17/Jan/23 03:49
Worklog Time Spent: 10m
Work Description: dengzhhu653 merged PR #3920:
URL: https://github.com/apache/hive/pull/3920
Issue Time Tracking
---
Worklog Id: (was: 839499)
Time Spent: 3h (was: 2h 50m)
> Renaming a translated external table with a specified location fails with
> 'location already exists' exception
> -
>
> Key: HIVE-26911
> URL: https://issues.apache.org/jira/browse/HIVE-26911
> Project: Hive
> Issue Type: Bug
> Components: Standalone Metastore
>Affects Versions: 4.0.0
>Reporter: Sai Hemanth Gantasala
>Assignee: Sai Hemanth Gantasala
>Priority: Major
> Labels: pull-request-available
> Time Spent: 3h
> Remaining Estimate: 0h
>
> Renaming a translated external table with a specified location fails with
> 'location already exists' exception.
> Below are steps for repro
> {code:java}
> create database tmp;
> use tmp;
> create table b(s string) stored as parquet location
> 'hdfs://localhost:20500/test-warehouse/tmp.db/some_location';
> alter table b rename to bb;
> ERROR: InvalidOperationException: New location for this table hive.tmp.bb
> already exists : hdfs://localhost:20500/test-warehouse/tmp.db/some_location
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (HIVE-26944) FileSinkOperator shouldn't check for compactiontable for every row being processed
[ https://issues.apache.org/jira/browse/HIVE-26944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena resolved HIVE-26944. - Fix Version/s: 4.0.0 Resolution: Fixed > FileSinkOperator shouldn't check for compactiontable for every row being > processed > -- > > Key: HIVE-26944 > URL: https://issues.apache.org/jira/browse/HIVE-26944 > Project: Hive > Issue Type: Improvement >Reporter: Rajesh Balamohan >Assignee: Rajesh Balamohan >Priority: Major > Labels: iceberg, pull-request-available > Fix For: 4.0.0 > > Attachments: Screenshot 2023-01-16 at 10.32.24 AM.png > > Time Spent: 0.5h > Remaining Estimate: 0h > > !Screenshot 2023-01-16 at 10.32.24 AM.png! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HIVE-26944) FileSinkOperator shouldn't check for compactiontable for every row being processed
[ https://issues.apache.org/jira/browse/HIVE-26944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17677572#comment-17677572 ] Ayush Saxena commented on HIVE-26944: - Committed to master. Thanx [~rajesh.balamohan] for the contribution!!! > FileSinkOperator shouldn't check for compactiontable for every row being > processed > -- > > Key: HIVE-26944 > URL: https://issues.apache.org/jira/browse/HIVE-26944 > Project: Hive > Issue Type: Improvement >Reporter: Rajesh Balamohan >Assignee: Rajesh Balamohan >Priority: Major > Labels: iceberg, pull-request-available > Attachments: Screenshot 2023-01-16 at 10.32.24 AM.png > > Time Spent: 0.5h > Remaining Estimate: 0h > > !Screenshot 2023-01-16 at 10.32.24 AM.png! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26944) FileSinkOperator shouldn't check for compactiontable for every row being processed
[ https://issues.apache.org/jira/browse/HIVE-26944?focusedWorklogId=839483=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-839483 ] ASF GitHub Bot logged work on HIVE-26944: - Author: ASF GitHub Bot Created on: 17/Jan/23 02:19 Start Date: 17/Jan/23 02:19 Worklog Time Spent: 10m Work Description: ayushtkn merged PR #3952: URL: https://github.com/apache/hive/pull/3952 Issue Time Tracking --- Worklog Id: (was: 839483) Time Spent: 0.5h (was: 20m) > FileSinkOperator shouldn't check for compactiontable for every row being > processed > -- > > Key: HIVE-26944 > URL: https://issues.apache.org/jira/browse/HIVE-26944 > Project: Hive > Issue Type: Improvement >Reporter: Rajesh Balamohan >Assignee: Rajesh Balamohan >Priority: Major > Labels: iceberg, pull-request-available > Attachments: Screenshot 2023-01-16 at 10.32.24 AM.png > > Time Spent: 0.5h > Remaining Estimate: 0h > > !Screenshot 2023-01-16 at 10.32.24 AM.png! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26400) Provide docker images for Hive
[ https://issues.apache.org/jira/browse/HIVE-26400?focusedWorklogId=839481=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-839481 ] ASF GitHub Bot logged work on HIVE-26400: - Author: ASF GitHub Bot Created on: 17/Jan/23 02:12 Start Date: 17/Jan/23 02:12 Worklog Time Spent: 10m Work Description: dengzhhu653 commented on PR #3448: URL: https://github.com/apache/hive/pull/3448#issuecomment-1384751312 @deniskuzZ any thoughts about the PR? Thank you in advance! Issue Time Tracking --- Worklog Id: (was: 839481) Time Spent: 6.5h (was: 6h 20m) > Provide docker images for Hive > -- > > Key: HIVE-26400 > URL: https://issues.apache.org/jira/browse/HIVE-26400 > Project: Hive > Issue Type: Improvement > Components: Build Infrastructure >Reporter: Zhihua Deng >Assignee: Zhihua Deng >Priority: Blocker > Labels: hive-4.0.0-must, pull-request-available > Time Spent: 6.5h > Remaining Estimate: 0h > > Make Apache Hive be able to run inside docker container in pseudo-distributed > mode, with MySQL/Derby as its back database, provide the following: > * Quick-start/Debugging/Prepare a test env for Hive; > * Tools to build target image with specified version of Hive and its > dependencies; > * Images can be used as the basis for the Kubernetes operator. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HIVE-26748) Prepare for Hive 3.2.0 Release
[ https://issues.apache.org/jira/browse/HIVE-26748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17677568#comment-17677568 ] Vihang Karajgaonkar commented on HIVE-26748: Sounds good [~amanraj2520]. I added a couple of tickets for this and linked them under HIVE-26751 similar to other jiras. > Prepare for Hive 3.2.0 Release > -- > > Key: HIVE-26748 > URL: https://issues.apache.org/jira/browse/HIVE-26748 > Project: Hive > Issue Type: Task > Components: Hive >Affects Versions: 3.1.3 >Reporter: Aman Raj >Assignee: Aman Raj >Priority: Major > Labels: hive-3.2.0-must > > This is the Umbrella Jira to track all the commits that would go on top of > current branch-3 in this new 3.2.0 Hive release. I will add all the JIRAs > that will be cherry picked as part of this commit by defining subtasks or > linking the JIRAs. > > *Please note that this is an Open forum and I welcome all responses for the > same from the community with regards to any new bug fixes that should be > cherry picked.* -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-26949) Backport HIVE-26071 to branch-3
[ https://issues.apache.org/jira/browse/HIVE-26949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vihang Karajgaonkar updated HIVE-26949: --- Component/s: Metastore Standalone Metastore > Backport HIVE-26071 to branch-3 > --- > > Key: HIVE-26949 > URL: https://issues.apache.org/jira/browse/HIVE-26949 > Project: Hive > Issue Type: Improvement > Components: Metastore, Standalone Metastore >Reporter: Vihang Karajgaonkar >Assignee: Vihang Karajgaonkar >Priority: Blocker > > Creating this ticket to backport HIVE-26071 to branch-3. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-26949) Backport HIVE-26071 to branch-3
[ https://issues.apache.org/jira/browse/HIVE-26949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vihang Karajgaonkar updated HIVE-26949: --- Priority: Blocker (was: Major) > Backport HIVE-26071 to branch-3 > --- > > Key: HIVE-26949 > URL: https://issues.apache.org/jira/browse/HIVE-26949 > Project: Hive > Issue Type: Improvement >Reporter: Vihang Karajgaonkar >Assignee: Vihang Karajgaonkar >Priority: Blocker > > Creating this ticket to backport HIVE-26071 to branch-3. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-26949) Backport HIVE-26071 to branch-3
[ https://issues.apache.org/jira/browse/HIVE-26949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vihang Karajgaonkar updated HIVE-26949: --- Target Version/s: 3.2.0 > Backport HIVE-26071 to branch-3 > --- > > Key: HIVE-26949 > URL: https://issues.apache.org/jira/browse/HIVE-26949 > Project: Hive > Issue Type: Improvement >Reporter: Vihang Karajgaonkar >Assignee: Vihang Karajgaonkar >Priority: Blocker > > Creating this ticket to backport HIVE-26071 to branch-3. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (HIVE-26949) Backport HIVE-26071 to branch-3
[ https://issues.apache.org/jira/browse/HIVE-26949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vihang Karajgaonkar reassigned HIVE-26949: -- > Backport HIVE-26071 to branch-3 > --- > > Key: HIVE-26949 > URL: https://issues.apache.org/jira/browse/HIVE-26949 > Project: Hive > Issue Type: Improvement >Reporter: Vihang Karajgaonkar >Assignee: Vihang Karajgaonkar >Priority: Major > > Creating this ticket to backport HIVE-26071 to branch-3. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (HIVE-26948) Backport HIVE-21456 to branch-3
[ https://issues.apache.org/jira/browse/HIVE-26948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vihang Karajgaonkar reassigned HIVE-26948: -- > Backport HIVE-21456 to branch-3 > --- > > Key: HIVE-26948 > URL: https://issues.apache.org/jira/browse/HIVE-26948 > Project: Hive > Issue Type: Improvement > Components: Metastore, Standalone Metastore >Reporter: Vihang Karajgaonkar >Assignee: Vihang Karajgaonkar >Priority: Blocker > > HIVE-21456 adds support to connect to Hive metastore over http transport. > This is a very useful feature especially in cloud based environments. > Creating this ticket to backport it to branch-3. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26711) The very first REPL Load should make the Target Database read-only
[ https://issues.apache.org/jira/browse/HIVE-26711?focusedWorklogId=839480=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-839480 ] ASF GitHub Bot logged work on HIVE-26711: - Author: ASF GitHub Bot Created on: 17/Jan/23 00:34 Start Date: 17/Jan/23 00:34 Worklog Time Spent: 10m Work Description: pudidic commented on PR #3736: URL: https://github.com/apache/hive/pull/3736#issuecomment-1384698787 Also please check the CI build. It may need a re-triggering with an empty commit. `git commit --allow-empty -m 'message'`. Issue Time Tracking --- Worklog Id: (was: 839480) Time Spent: 1h 20m (was: 1h 10m) > The very first REPL Load should make the Target Database read-only > -- > > Key: HIVE-26711 > URL: https://issues.apache.org/jira/browse/HIVE-26711 > Project: Hive > Issue Type: Task >Reporter: Shreenidhi >Assignee: Shreenidhi >Priority: Major > Labels: pull-request-available > Time Spent: 1h 20m > Remaining Estimate: 0h > > Use EnforceReadOnly hook to set TARGET database read only during BootStrap > load. > Also ensure backward compatibility. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26809) Upgrade ORC to 1.8.1
[ https://issues.apache.org/jira/browse/HIVE-26809?focusedWorklogId=839479=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-839479 ] ASF GitHub Bot logged work on HIVE-26809: - Author: ASF GitHub Bot Created on: 17/Jan/23 00:31 Start Date: 17/Jan/23 00:31 Worklog Time Spent: 10m Work Description: sonarcloud[bot] commented on PR #3833: URL: https://github.com/apache/hive/pull/3833#issuecomment-1384697767 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive=3833) [](https://sonarcloud.io/project/issues?id=apache_hive=3833=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=3833=false=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive=3833=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=3833=false=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive=3833=false=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive=3833=false=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3833=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3833=false=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3833=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive=3833=false=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive=3833=false=CODE_SMELL) [10 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive=3833=false=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive=3833=coverage=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive=3833=duplicated_lines_density=list) No Duplication information Issue Time Tracking --- Worklog Id: (was: 839479) Time Spent: 4h 50m (was: 4h 40m) > Upgrade ORC to 1.8.1 > > > Key: HIVE-26809 > URL: https://issues.apache.org/jira/browse/HIVE-26809 > Project: Hive > Issue Type: Improvement >Affects Versions: 4.0.0 >Reporter: Dmitriy Fingerman >Assignee: Dmitriy Fingerman >Priority: Major > Labels: pull-request-available > Time Spent: 4h 50m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26711) The very first REPL Load should make the Target Database read-only
[
https://issues.apache.org/jira/browse/HIVE-26711?focusedWorklogId=839478=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-839478
]
ASF GitHub Bot logged work on HIVE-26711:
-
Author: ASF GitHub Bot
Created on: 17/Jan/23 00:28
Start Date: 17/Jan/23 00:28
Worklog Time Spent: 10m
Work Description: pudidic commented on code in PR #3736:
URL: https://github.com/apache/hive/pull/3736#discussion_r1071645397
##
itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/parse/TestReplWithReadOnlyHook.java:
##
@@ -0,0 +1,126 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql.parse;
+
+import static
org.apache.hadoop.hive.ql.hooks.EnforceReadOnlyDatabaseHook.READONLY;
+import static org.apache.hadoop.hive.common.repl.ReplConst.READ_ONLY_HOOK;
+import static org.junit.Assert.assertEquals;
+
+import org.apache.hadoop.hdfs.MiniDFSCluster;
+import org.apache.hadoop.hive.conf.HiveConf;
+import org.apache.hadoop.hive.metastore.conf.MetastoreConf;
+import
org.apache.hadoop.hive.metastore.messaging.json.gzip.GzipJSONMessageEncoder;
+import org.apache.hadoop.hive.shims.Utils;
+import org.junit.After;
+import org.junit.BeforeClass;
+import org.junit.Test;
+
+import java.util.HashMap;
+import java.util.Map;
+
+public class TestReplWithReadOnlyHook extends
BaseReplicationScenariosAcidTables {
+
+ @BeforeClass
+ public static void classLevelSetup() throws Exception {
+Map overrides = new HashMap<>();
+overrides.put(MetastoreConf.ConfVars.EVENT_MESSAGE_FACTORY.getHiveName(),
+ GzipJSONMessageEncoder.class.getCanonicalName());
+
+conf = new HiveConf(TestReplWithReadOnlyHook.class);
+conf.set("hadoop.proxyuser." + Utils.getUGI().getShortUserName() +
".hosts", "*");
+
+MiniDFSCluster miniDFSCluster =
+ new MiniDFSCluster.Builder(conf).numDataNodes(2).format(true).build();
+
+Map acidEnableConf = new HashMap() {{
Review Comment:
It's my first time to see this double brace initialization in my Java
experience. So I may be wrong about this. I did a small research, and some of
references call it an anti-pattern. How about just repeating with HashMap calls
or using ImmutableMap.Builder to chain the methods? Thanks.
1. https://www.baeldung.com/java-double-brace-initialization
2. https://blog.jooq.org/dont-be-clever-the-double-curly-braces-anti-pattern/
Issue Time Tracking
---
Worklog Id: (was: 839478)
Time Spent: 1h 10m (was: 1h)
> The very first REPL Load should make the Target Database read-only
> --
>
> Key: HIVE-26711
> URL: https://issues.apache.org/jira/browse/HIVE-26711
> Project: Hive
> Issue Type: Task
>Reporter: Shreenidhi
>Assignee: Shreenidhi
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> Use EnforceReadOnly hook to set TARGET database read only during BootStrap
> load.
> Also ensure backward compatibility.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-26575) Missing Copy Serde Description when Alter Table's Serde Description
[ https://issues.apache.org/jira/browse/HIVE-26575?focusedWorklogId=839477=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-839477 ] ASF GitHub Bot logged work on HIVE-26575: - Author: ASF GitHub Bot Created on: 17/Jan/23 00:21 Start Date: 17/Jan/23 00:21 Worklog Time Spent: 10m Work Description: github-actions[bot] commented on PR #3633: URL: https://github.com/apache/hive/pull/3633#issuecomment-1384692675 This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Feel free to reach out on the d...@hive.apache.org list if the patch is in need of reviews. Issue Time Tracking --- Worklog Id: (was: 839477) Time Spent: 40m (was: 0.5h) > Missing Copy Serde Description when Alter Table's Serde Description > --- > > Key: HIVE-26575 > URL: https://issues.apache.org/jira/browse/HIVE-26575 > Project: Hive > Issue Type: Bug > Components: Standalone Metastore >Reporter: hyungbin >Priority: Minor > Labels: pull-request-available > Attachments: image-2022-09-29-18-35-44-640.png > > Time Spent: 40m > Remaining Estimate: 0h > > Missing Copy Serde Description when Alter Table's Serde Description > I changed the description to "altered description" but it is not changed. > !image-2022-09-29-18-35-44-640.png! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26947) Hive compactor.Worker can respawn connections to HMS at extremely high frequency
[ https://issues.apache.org/jira/browse/HIVE-26947?focusedWorklogId=839471=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-839471 ] ASF GitHub Bot logged work on HIVE-26947: - Author: ASF GitHub Bot Created on: 16/Jan/23 23:20 Start Date: 16/Jan/23 23:20 Worklog Time Spent: 10m Work Description: sonarcloud[bot] commented on PR #3955: URL: https://github.com/apache/hive/pull/3955#issuecomment-1384666141 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive=3955) [](https://sonarcloud.io/project/issues?id=apache_hive=3955=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=3955=false=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive=3955=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=3955=false=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive=3955=false=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive=3955=false=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3955=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3955=false=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3955=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive=3955=false=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive=3955=false=CODE_SMELL) [7 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive=3955=false=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive=3955=coverage=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive=3955=duplicated_lines_density=list) No Duplication information Issue Time Tracking --- Worklog Id: (was: 839471) Time Spent: 50m (was: 40m) > Hive compactor.Worker can respawn connections to HMS at extremely high > frequency > > > Key: HIVE-26947 > URL: https://issues.apache.org/jira/browse/HIVE-26947 > Project: Hive > Issue Type: Bug >Reporter: Akshat Mathur >Assignee: Akshat Mathur >Priority: Major > Labels: pull-request-available > Time Spent: 50m > Remaining Estimate: 0h > > After catching the exception generated by the findNextCompactionAndExecute() > task, HS2 appears to immediately rerun the task with no delay or backoff. As > a result there are ~3500 connection attempts from HS2 to HMS over just a 5 > second period in the HS2 log > The compactor.Worker should wait between failed attempts and maybe do an > exponential backoff. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26717) Query based Rebalance compaction on insert-only tables
[ https://issues.apache.org/jira/browse/HIVE-26717?focusedWorklogId=839467=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-839467 ] ASF GitHub Bot logged work on HIVE-26717: - Author: ASF GitHub Bot Created on: 16/Jan/23 19:42 Start Date: 16/Jan/23 19:42 Worklog Time Spent: 10m Work Description: sonarcloud[bot] commented on PR #3935: URL: https://github.com/apache/hive/pull/3935#issuecomment-1384479480 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive=3935) [](https://sonarcloud.io/project/issues?id=apache_hive=3935=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=3935=false=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive=3935=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=3935=false=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive=3935=false=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive=3935=false=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3935=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3935=false=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3935=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive=3935=false=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive=3935=false=CODE_SMELL) [5 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive=3935=false=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive=3935=coverage=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive=3935=duplicated_lines_density=list) No Duplication information Issue Time Tracking --- Worklog Id: (was: 839467) Time Spent: 2h (was: 1h 50m) > Query based Rebalance compaction on insert-only tables > -- > > Key: HIVE-26717 > URL: https://issues.apache.org/jira/browse/HIVE-26717 > Project: Hive > Issue Type: Sub-task > Components: Hive >Reporter: László Végh >Assignee: László Végh >Priority: Major > Labels: ACID, compaction, pull-request-available > Time Spent: 2h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26809) Upgrade ORC to 1.8.1
[ https://issues.apache.org/jira/browse/HIVE-26809?focusedWorklogId=839466=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-839466 ] ASF GitHub Bot logged work on HIVE-26809: - Author: ASF GitHub Bot Created on: 16/Jan/23 19:37 Start Date: 16/Jan/23 19:37 Worklog Time Spent: 10m Work Description: sonarcloud[bot] commented on PR #3833: URL: https://github.com/apache/hive/pull/3833#issuecomment-1384475873 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive=3833) [](https://sonarcloud.io/project/issues?id=apache_hive=3833=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=3833=false=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive=3833=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=3833=false=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive=3833=false=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive=3833=false=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3833=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3833=false=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3833=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive=3833=false=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive=3833=false=CODE_SMELL) [10 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive=3833=false=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive=3833=coverage=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive=3833=duplicated_lines_density=list) No Duplication information Issue Time Tracking --- Worklog Id: (was: 839466) Time Spent: 4h 40m (was: 4.5h) > Upgrade ORC to 1.8.1 > > > Key: HIVE-26809 > URL: https://issues.apache.org/jira/browse/HIVE-26809 > Project: Hive > Issue Type: Improvement >Affects Versions: 4.0.0 >Reporter: Dmitriy Fingerman >Assignee: Dmitriy Fingerman >Priority: Major > Labels: pull-request-available > Time Spent: 4h 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26908) Disable Initiator on HMS instance at the same time enable Cleaner thread
[
https://issues.apache.org/jira/browse/HIVE-26908?focusedWorklogId=839465=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-839465
]
ASF GitHub Bot logged work on HIVE-26908:
-
Author: ASF GitHub Bot
Created on: 16/Jan/23 19:32
Start Date: 16/Jan/23 19:32
Worklog Time Spent: 10m
Work Description: veghlaci05 commented on PR #3915:
URL: https://github.com/apache/hive/pull/3915#issuecomment-1384472172
@deniskuzZ No objections, please go on with the merge.
Issue Time Tracking
---
Worklog Id: (was: 839465)
Time Spent: 1h 50m (was: 1h 40m)
> Disable Initiator on HMS instance at the same time enable Cleaner thread
>
>
> Key: HIVE-26908
> URL: https://issues.apache.org/jira/browse/HIVE-26908
> Project: Hive
> Issue Type: New Feature
> Components: Standalone Metastore
>Reporter: Taraka Rama Rao Lethavadla
>Assignee: Taraka Rama Rao Lethavadla
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> In the current implementation, both Initiator and Cleaner are either enabled
> or disabled using the same config
> {noformat}
> hive.compactor.initiator.on{noformat}
> So there is no way to selectively disable initiator and enable cleaner or
> vice versa.
> Introducing another config to handle Cleaner thread alone like
> hive.compactor.cleaner.on
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-26908) Disable Initiator on HMS instance at the same time enable Cleaner thread
[
https://issues.apache.org/jira/browse/HIVE-26908?focusedWorklogId=839463=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-839463
]
ASF GitHub Bot logged work on HIVE-26908:
-
Author: ASF GitHub Bot
Created on: 16/Jan/23 19:21
Start Date: 16/Jan/23 19:21
Worklog Time Spent: 10m
Work Description: deniskuzZ commented on PR #3915:
URL: https://github.com/apache/hive/pull/3915#issuecomment-1384465043
@veghlaci05 if no objections from your side, I'll merge this PR
Issue Time Tracking
---
Worklog Id: (was: 839463)
Time Spent: 1h 40m (was: 1.5h)
> Disable Initiator on HMS instance at the same time enable Cleaner thread
>
>
> Key: HIVE-26908
> URL: https://issues.apache.org/jira/browse/HIVE-26908
> Project: Hive
> Issue Type: New Feature
> Components: Standalone Metastore
>Reporter: Taraka Rama Rao Lethavadla
>Assignee: Taraka Rama Rao Lethavadla
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> In the current implementation, both Initiator and Cleaner are either enabled
> or disabled using the same config
> {noformat}
> hive.compactor.initiator.on{noformat}
> So there is no way to selectively disable initiator and enable cleaner or
> vice versa.
> Introducing another config to handle Cleaner thread alone like
> hive.compactor.cleaner.on
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-26809) Upgrade ORC to 1.8.1
[
https://issues.apache.org/jira/browse/HIVE-26809?focusedWorklogId=839455=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-839455
]
ASF GitHub Bot logged work on HIVE-26809:
-
Author: ASF GitHub Bot
Created on: 16/Jan/23 17:23
Start Date: 16/Jan/23 17:23
Worklog Time Spent: 10m
Work Description: difin commented on code in PR #3833:
URL: https://github.com/apache/hive/pull/3833#discussion_r1071406567
##
ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/EncodedTreeReaderFactory.java:
##
@@ -224,7 +237,252 @@ private static void skipCompressedIndex(boolean
isCompressed, PositionProvider i
index.getNext();
}
- protected static class StringStreamReader extends StringTreeReader
+ public static class StringDictionaryTreeReaderHive extends TreeReader {
Review Comment:
This is added as a fix to many failed CI tests that happened without this
fix.
These new classes are classes from Orc project prior to changes to
StringDictionaryTreeReader that were done as part of ORC-1060.
In more detail: Hive implements its own TreeReaderFactory. In ORC project,
the ticket ORC-1060 - "Reduce memory usage when vectorized reading dictionary
string encoding columns" introduced changes to StringDictionaryTreeReader which
were causing exceptions in Hive EncodedTreeReaderFactory when attempting to
upgrade to ORC 1.8.1. To handle that I added changes to Hive's
EncodedTreeReaderFactory to use StringDictionaryTreeReader version from Orc
project prior to changes from ORC-1060.
Issue Time Tracking
---
Worklog Id: (was: 839455)
Time Spent: 4.5h (was: 4h 20m)
> Upgrade ORC to 1.8.1
>
>
> Key: HIVE-26809
> URL: https://issues.apache.org/jira/browse/HIVE-26809
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 4.0.0
>Reporter: Dmitriy Fingerman
>Assignee: Dmitriy Fingerman
>Priority: Major
> Labels: pull-request-available
> Time Spent: 4.5h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-26809) Upgrade ORC to 1.8.1
[
https://issues.apache.org/jira/browse/HIVE-26809?focusedWorklogId=839454=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-839454
]
ASF GitHub Bot logged work on HIVE-26809:
-
Author: ASF GitHub Bot
Created on: 16/Jan/23 17:20
Start Date: 16/Jan/23 17:20
Worklog Time Spent: 10m
Work Description: difin commented on code in PR #3833:
URL: https://github.com/apache/hive/pull/3833#discussion_r1071406567
##
ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/EncodedTreeReaderFactory.java:
##
@@ -224,7 +237,252 @@ private static void skipCompressedIndex(boolean
isCompressed, PositionProvider i
index.getNext();
}
- protected static class StringStreamReader extends StringTreeReader
+ public static class StringDictionaryTreeReaderHive extends TreeReader {
Review Comment:
This is added as a fix to many failed CI tests that happened without this
fix.
In more detail: Hive implements its own TreeReaderFactory. In ORC project,
the ticket ORC-1060 - "Reduce memory usage when vectorized reading dictionary
string encoding columns" introduced changes to StringDictionaryTreeReader which
were causing exceptions in Hive EncodedTreeReaderFactory when attempting to
upgrade to ORC 1.8.1. To handle that I added changes to Hive's
EncodedTreeReaderFactory to use StringDictionaryTreeReader version from Orc
project prior to changes from ORC-1060.
Issue Time Tracking
---
Worklog Id: (was: 839454)
Time Spent: 4h 20m (was: 4h 10m)
> Upgrade ORC to 1.8.1
>
>
> Key: HIVE-26809
> URL: https://issues.apache.org/jira/browse/HIVE-26809
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 4.0.0
>Reporter: Dmitriy Fingerman
>Assignee: Dmitriy Fingerman
>Priority: Major
> Labels: pull-request-available
> Time Spent: 4h 20m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-26809) Upgrade ORC to 1.8.1
[ https://issues.apache.org/jira/browse/HIVE-26809?focusedWorklogId=839453=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-839453 ] ASF GitHub Bot logged work on HIVE-26809: - Author: ASF GitHub Bot Created on: 16/Jan/23 17:18 Start Date: 16/Jan/23 17:18 Worklog Time Spent: 10m Work Description: difin commented on code in PR #3833: URL: https://github.com/apache/hive/pull/3833#discussion_r1071459244 ## llap-server/src/java/org/apache/hadoop/hive/llap/io/decode/OrcEncodedDataConsumer.java: ## @@ -61,7 +61,7 @@ import org.apache.orc.impl.TreeReaderFactory.TreeReader; import org.apache.orc.impl.WriterImpl; import org.apache.orc.OrcProto; - +import org.apache.orc.impl.reader.tree.TypeReader; Review Comment: fixed Issue Time Tracking --- Worklog Id: (was: 839453) Time Spent: 4h 10m (was: 4h) > Upgrade ORC to 1.8.1 > > > Key: HIVE-26809 > URL: https://issues.apache.org/jira/browse/HIVE-26809 > Project: Hive > Issue Type: Improvement >Affects Versions: 4.0.0 >Reporter: Dmitriy Fingerman >Assignee: Dmitriy Fingerman >Priority: Major > Labels: pull-request-available > Time Spent: 4h 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26809) Upgrade ORC to 1.8.1
[
https://issues.apache.org/jira/browse/HIVE-26809?focusedWorklogId=839452=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-839452
]
ASF GitHub Bot logged work on HIVE-26809:
-
Author: ASF GitHub Bot
Created on: 16/Jan/23 17:18
Start Date: 16/Jan/23 17:18
Worklog Time Spent: 10m
Work Description: difin commented on code in PR #3833:
URL: https://github.com/apache/hive/pull/3833#discussion_r1071458846
##
ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/EncodedTreeReaderFactory.java:
##
@@ -1249,10 +1534,13 @@ public void seek(PositionProvider index) throws
IOException {
}
@Override
-public void nextVector(
- ColumnVector previousVector, boolean[] isNull, int batchSize) throws
IOException {
+public void nextVector(ColumnVector previousVector,
+boolean[] isNull,
+final int batchSize,
+FilterContext filterContext,
+ReadPhase readPhase) throws IOException {
Review Comment:
fixed formatting
Issue Time Tracking
---
Worklog Id: (was: 839452)
Time Spent: 4h (was: 3h 50m)
> Upgrade ORC to 1.8.1
>
>
> Key: HIVE-26809
> URL: https://issues.apache.org/jira/browse/HIVE-26809
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 4.0.0
>Reporter: Dmitriy Fingerman
>Assignee: Dmitriy Fingerman
>Priority: Major
> Labels: pull-request-available
> Time Spent: 4h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-26825) Compactor: Cleaner shouldn't fetch table details again and again for partitioned tables
[ https://issues.apache.org/jira/browse/HIVE-26825?focusedWorklogId=839450=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-839450 ] ASF GitHub Bot logged work on HIVE-26825: - Author: ASF GitHub Bot Created on: 16/Jan/23 16:40 Start Date: 16/Jan/23 16:40 Worklog Time Spent: 10m Work Description: sonarcloud[bot] commented on PR #3864: URL: https://github.com/apache/hive/pull/3864#issuecomment-1384306440 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive=3864) [](https://sonarcloud.io/project/issues?id=apache_hive=3864=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=3864=false=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive=3864=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=3864=false=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive=3864=false=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive=3864=false=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3864=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3864=false=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3864=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive=3864=false=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive=3864=false=CODE_SMELL) [3 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive=3864=false=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive=3864=coverage=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive=3864=duplicated_lines_density=list) No Duplication information Issue Time Tracking --- Worklog Id: (was: 839450) Time Spent: 2.5h (was: 2h 20m) > Compactor: Cleaner shouldn't fetch table details again and again for > partitioned tables > --- > > Key: HIVE-26825 > URL: https://issues.apache.org/jira/browse/HIVE-26825 > Project: Hive > Issue Type: Improvement > Components: Transactions >Reporter: KIRTI RUGE >Assignee: KIRTI RUGE >Priority: Major > Labels: pull-request-available > Time Spent: 2.5h > Remaining Estimate: 0h > > Cleaner shouldn't be fetch table/partition details for all its partitions. > When there are large number of databases/tables, it takes lot of time for > Initiator to complete its initial iteration and load on DB also goes higher. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26925) MV with iceberg storage format fails when contains 'PARTITIONED ON' clause due to column number/types difference.
[
https://issues.apache.org/jira/browse/HIVE-26925?focusedWorklogId=839449=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-839449
]
ASF GitHub Bot logged work on HIVE-26925:
-
Author: ASF GitHub Bot
Created on: 16/Jan/23 16:26
Start Date: 16/Jan/23 16:26
Worklog Time Spent: 10m
Work Description: sonarcloud[bot] commented on PR #3939:
URL: https://github.com/apache/hive/pull/3939#issuecomment-1384290424
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive=3939)
[](https://sonarcloud.io/project/issues?id=apache_hive=3939=false=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive=3939=false=BUG)
[0
Bugs](https://sonarcloud.io/project/issues?id=apache_hive=3939=false=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive=3939=false=VULNERABILITY)
[](https://sonarcloud.io/project/issues?id=apache_hive=3939=false=VULNERABILITY)
[0
Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive=3939=false=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3939=false=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3939=false=SECURITY_HOTSPOT)
[0 Security
Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3939=false=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive=3939=false=CODE_SMELL)
[](https://sonarcloud.io/project/issues?id=apache_hive=3939=false=CODE_SMELL)
[0 Code
Smells](https://sonarcloud.io/project/issues?id=apache_hive=3939=false=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive=3939=coverage=list)
No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive=3939=duplicated_lines_density=list)
No Duplication information
Issue Time Tracking
---
Worklog Id: (was: 839449)
Time Spent: 1h (was: 50m)
> MV with iceberg storage format fails when contains 'PARTITIONED ON' clause
> due to column number/types difference.
> -
>
> Key: HIVE-26925
> URL: https://issues.apache.org/jira/browse/HIVE-26925
> Project: Hive
> Issue Type: Bug
> Components: Iceberg integration
>Reporter: Dharmik Thakkar
>Assignee: Krisztian Kasa
>Priority: Critical
> Labels: pull-request-available
> Time Spent: 1h
> Remaining Estimate: 0h
>
> MV with iceberg storage format fails when contains 'PARTITIONED ON' clause
> due to column number/types difference.
> {code:java}
> !!! annotations iceberg
> >>> use iceberg_test_db_hive;
> No rows affected
> >>> set hive.exec.max.dynamic.partitions=2000;
> >>> set hive.exec.max.dynamic.partitions.pernode=2000;
> >>> drop materialized view if exists mv_agg_gby_col_partitioned;
> >>> create materialized view mv_agg_gby_col_partitioned PARTITIONED ON (t)
> >>> stored by iceberg stored as orc tblproperties ('format-version'='1') as
> >>> select b,f,sum(b), sum(f),t from all100k group by b,f,v,c,t;
> >>> analyze table mv_agg_gby_col_partitioned compute statistics for columns;
> >>> set hive.explain.user=false;
> >>> explain select b,f,sum(b) from all100k where t=93 group by c,v,f,b;
> !!! match row_contains
> alias:
[jira] [Work logged] (HIVE-26809) Upgrade ORC to 1.8.1
[
https://issues.apache.org/jira/browse/HIVE-26809?focusedWorklogId=839446=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-839446
]
ASF GitHub Bot logged work on HIVE-26809:
-
Author: ASF GitHub Bot
Created on: 16/Jan/23 16:15
Start Date: 16/Jan/23 16:15
Worklog Time Spent: 10m
Work Description: difin commented on code in PR #3833:
URL: https://github.com/apache/hive/pull/3833#discussion_r1071406567
##
ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/EncodedTreeReaderFactory.java:
##
@@ -224,7 +237,252 @@ private static void skipCompressedIndex(boolean
isCompressed, PositionProvider i
index.getNext();
}
- protected static class StringStreamReader extends StringTreeReader
+ public static class StringDictionaryTreeReaderHive extends TreeReader {
Review Comment:
This is added as a fix to many failed CI tests that happened without this
fix.
Hive implements its own TreeReaderFactory. In ORC project, ORC-1060 -
"Reduce memory usage when vectorized reading dictionary string encoding
columns" introduced changes to StringDictionaryTreeReader which were causing
exceptions in Hive EncodedTreeReaderFactory when attempting to upgrade to ORC
1.8.1. To handle that I added changes to Hive's EncodedTreeReaderFactory to use
StringDictionaryTreeReader version as without ORC-1060.
Issue Time Tracking
---
Worklog Id: (was: 839446)
Time Spent: 3h 40m (was: 3.5h)
> Upgrade ORC to 1.8.1
>
>
> Key: HIVE-26809
> URL: https://issues.apache.org/jira/browse/HIVE-26809
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 4.0.0
>Reporter: Dmitriy Fingerman
>Assignee: Dmitriy Fingerman
>Priority: Major
> Labels: pull-request-available
> Time Spent: 3h 40m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-26809) Upgrade ORC to 1.8.1
[
https://issues.apache.org/jira/browse/HIVE-26809?focusedWorklogId=839447=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-839447
]
ASF GitHub Bot logged work on HIVE-26809:
-
Author: ASF GitHub Bot
Created on: 16/Jan/23 16:15
Start Date: 16/Jan/23 16:15
Worklog Time Spent: 10m
Work Description: difin commented on code in PR #3833:
URL: https://github.com/apache/hive/pull/3833#discussion_r1071406567
##
ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/EncodedTreeReaderFactory.java:
##
@@ -224,7 +237,252 @@ private static void skipCompressedIndex(boolean
isCompressed, PositionProvider i
index.getNext();
}
- protected static class StringStreamReader extends StringTreeReader
+ public static class StringDictionaryTreeReaderHive extends TreeReader {
Review Comment:
This is added as a fix to many failed CI tests that happened without this
fix.
In more detail: Hive implements its own TreeReaderFactory. In ORC project,
ORC-1060 - "Reduce memory usage when vectorized reading dictionary string
encoding columns" introduced changes to StringDictionaryTreeReader which were
causing exceptions in Hive EncodedTreeReaderFactory when attempting to upgrade
to ORC 1.8.1. To handle that I added changes to Hive's EncodedTreeReaderFactory
to use StringDictionaryTreeReader version as without ORC-1060.
Issue Time Tracking
---
Worklog Id: (was: 839447)
Time Spent: 3h 50m (was: 3h 40m)
> Upgrade ORC to 1.8.1
>
>
> Key: HIVE-26809
> URL: https://issues.apache.org/jira/browse/HIVE-26809
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 4.0.0
>Reporter: Dmitriy Fingerman
>Assignee: Dmitriy Fingerman
>Priority: Major
> Labels: pull-request-available
> Time Spent: 3h 50m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-26717) Query based Rebalance compaction on insert-only tables
[ https://issues.apache.org/jira/browse/HIVE-26717?focusedWorklogId=839444=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-839444 ] ASF GitHub Bot logged work on HIVE-26717: - Author: ASF GitHub Bot Created on: 16/Jan/23 15:54 Start Date: 16/Jan/23 15:54 Worklog Time Spent: 10m Work Description: sonarcloud[bot] commented on PR #3935: URL: https://github.com/apache/hive/pull/3935#issuecomment-1384243680 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive=3935) [](https://sonarcloud.io/project/issues?id=apache_hive=3935=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=3935=false=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive=3935=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=3935=false=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive=3935=false=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive=3935=false=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3935=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3935=false=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3935=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive=3935=false=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive=3935=false=CODE_SMELL) [5 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive=3935=false=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive=3935=coverage=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive=3935=duplicated_lines_density=list) No Duplication information Issue Time Tracking --- Worklog Id: (was: 839444) Time Spent: 1h 50m (was: 1h 40m) > Query based Rebalance compaction on insert-only tables > -- > > Key: HIVE-26717 > URL: https://issues.apache.org/jira/browse/HIVE-26717 > Project: Hive > Issue Type: Sub-task > Components: Hive >Reporter: László Végh >Assignee: László Végh >Priority: Major > Labels: ACID, compaction, pull-request-available > Time Spent: 1h 50m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26896) Backport of Test fixes for lineage3.q and load_static_ptn_into_bucketed_table.q
[ https://issues.apache.org/jira/browse/HIVE-26896?focusedWorklogId=839439=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-839439 ] ASF GitHub Bot logged work on HIVE-26896: - Author: ASF GitHub Bot Created on: 16/Jan/23 15:19 Start Date: 16/Jan/23 15:19 Worklog Time Spent: 10m Work Description: amanraj2520 commented on PR #3901: URL: https://github.com/apache/hive/pull/3901#issuecomment-1384205066 @zabetak Modified it. Kindly review this. Issue Time Tracking --- Worklog Id: (was: 839439) Time Spent: 1h 40m (was: 1.5h) > Backport of Test fixes for lineage3.q and > load_static_ptn_into_bucketed_table.q > --- > > Key: HIVE-26896 > URL: https://issues.apache.org/jira/browse/HIVE-26896 > Project: Hive > Issue Type: Sub-task >Reporter: Aman Raj >Assignee: Aman Raj >Priority: Critical > Labels: pull-request-available > Time Spent: 1h 40m > Remaining Estimate: 0h > > These tests were fixed in branch-3.1 so backporting them to branch-3 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26896) Backport of Test fixes for lineage3.q and load_static_ptn_into_bucketed_table.q
[ https://issues.apache.org/jira/browse/HIVE-26896?focusedWorklogId=839438=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-839438 ] ASF GitHub Bot logged work on HIVE-26896: - Author: ASF GitHub Bot Created on: 16/Jan/23 15:17 Start Date: 16/Jan/23 15:17 Worklog Time Spent: 10m Work Description: amanraj2520 commented on PR #3901: URL: https://github.com/apache/hive/pull/3901#issuecomment-1384202675 @zabetak I cannot find any test case where lineage3 failed. I will revert the lineage3 fix and rename this PR. Issue Time Tracking --- Worklog Id: (was: 839438) Time Spent: 1.5h (was: 1h 20m) > Backport of Test fixes for lineage3.q and > load_static_ptn_into_bucketed_table.q > --- > > Key: HIVE-26896 > URL: https://issues.apache.org/jira/browse/HIVE-26896 > Project: Hive > Issue Type: Sub-task >Reporter: Aman Raj >Assignee: Aman Raj >Priority: Critical > Labels: pull-request-available > Time Spent: 1.5h > Remaining Estimate: 0h > > These tests were fixed in branch-3.1 so backporting them to branch-3 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26947) Hive compactor.Worker can respawn connections to HMS at extremely high frequency
[
https://issues.apache.org/jira/browse/HIVE-26947?focusedWorklogId=839433=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-839433
]
ASF GitHub Bot logged work on HIVE-26947:
-
Author: ASF GitHub Bot
Created on: 16/Jan/23 14:40
Start Date: 16/Jan/23 14:40
Worklog Time Spent: 10m
Work Description: akshat0395 commented on code in PR #3955:
URL: https://github.com/apache/hive/pull/3955#discussion_r1071318885
##
ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/Worker.java:
##
@@ -118,19 +119,23 @@ public void run() {
singleRun.cancel(true);
executor.shutdownNow();
executor = getTimeoutHandlingExecutor();
+ err = true;
} catch (ExecutionException e) {
LOG.info("Exception during executing compaction", e);
+ err = true;
} catch (InterruptedException ie) {
// do not ignore interruption requests
return;
+} catch (Throwable t) {
+ err = true;
}
doPostLoopActions(System.currentTimeMillis() - startedAt);
// If we didn't try to launch a job it either means there was no work
to do or we got
// here as the result of a communication failure with the DB. Either
way we want to wait
// a bit before, otherwise we can start over the loop immediately.
Review Comment:
updated, Thanks for the review @veghlaci05
Issue Time Tracking
---
Worklog Id: (was: 839433)
Time Spent: 40m (was: 0.5h)
> Hive compactor.Worker can respawn connections to HMS at extremely high
> frequency
>
>
> Key: HIVE-26947
> URL: https://issues.apache.org/jira/browse/HIVE-26947
> Project: Hive
> Issue Type: Bug
>Reporter: Akshat Mathur
>Assignee: Akshat Mathur
>Priority: Major
> Labels: pull-request-available
> Time Spent: 40m
> Remaining Estimate: 0h
>
> After catching the exception generated by the findNextCompactionAndExecute()
> task, HS2 appears to immediately rerun the task with no delay or backoff. As
> a result there are ~3500 connection attempts from HS2 to HMS over just a 5
> second period in the HS2 log
> The compactor.Worker should wait between failed attempts and maybe do an
> exponential backoff.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-26947) Hive compactor.Worker can respawn connections to HMS at extremely high frequency
[
https://issues.apache.org/jira/browse/HIVE-26947?focusedWorklogId=839432=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-839432
]
ASF GitHub Bot logged work on HIVE-26947:
-
Author: ASF GitHub Bot
Created on: 16/Jan/23 14:40
Start Date: 16/Jan/23 14:40
Worklog Time Spent: 10m
Work Description: akshat0395 commented on code in PR #3955:
URL: https://github.com/apache/hive/pull/3955#discussion_r1071318885
##
ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/Worker.java:
##
@@ -118,19 +119,23 @@ public void run() {
singleRun.cancel(true);
executor.shutdownNow();
executor = getTimeoutHandlingExecutor();
+ err = true;
} catch (ExecutionException e) {
LOG.info("Exception during executing compaction", e);
+ err = true;
} catch (InterruptedException ie) {
// do not ignore interruption requests
return;
+} catch (Throwable t) {
+ err = true;
}
doPostLoopActions(System.currentTimeMillis() - startedAt);
// If we didn't try to launch a job it either means there was no work
to do or we got
// here as the result of a communication failure with the DB. Either
way we want to wait
// a bit before, otherwise we can start over the loop immediately.
Review Comment:
updated
Issue Time Tracking
---
Worklog Id: (was: 839432)
Time Spent: 0.5h (was: 20m)
> Hive compactor.Worker can respawn connections to HMS at extremely high
> frequency
>
>
> Key: HIVE-26947
> URL: https://issues.apache.org/jira/browse/HIVE-26947
> Project: Hive
> Issue Type: Bug
>Reporter: Akshat Mathur
>Assignee: Akshat Mathur
>Priority: Major
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> After catching the exception generated by the findNextCompactionAndExecute()
> task, HS2 appears to immediately rerun the task with no delay or backoff. As
> a result there are ~3500 connection attempts from HS2 to HMS over just a 5
> second period in the HS2 log
> The compactor.Worker should wait between failed attempts and maybe do an
> exponential backoff.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HIVE-26941) Make SetProcessor configurable to ignore some set variables
[
https://issues.apache.org/jira/browse/HIVE-26941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17677411#comment-17677411
]
Miklos Szurap commented on HIVE-26941:
--
Submitted [PR#3950|https://github.com/apache/hive/pull/3950].
> Make SetProcessor configurable to ignore some set variables
> ---
>
> Key: HIVE-26941
> URL: https://issues.apache.org/jira/browse/HIVE-26941
> Project: Hive
> Issue Type: New Feature
> Components: Configuration, Hive
>Reporter: Miklos Szurap
>Assignee: Miklos Szurap
>Priority: Major
> Labels: config, configuration
>
> In certain environments after upgrades we need to restrict users from
> changing some Hive configurations during runtime (for example
> "mapreduce.job.queuename" or "hive.execution.engine").
> The "hive.security.authorization.sqlstd.confwhitelist" could be used for
> this, however:
> * it is complex to modify that sometimes (for example to exclude a config
> which is otherwise allowed with a wildcard)
> * when a user script tries to set a parameter not in the
> "hive.security.authorization.sqlstd.confwhitelist" then the whole script just
> fails with "Error: Error while processing statement: Cannot modify at
> runtime. It is not in list of params that are allowed to be modified at
> runtime". This would require all the user scripts and jobs to be modified (to
> remove that "set" command), that can be a huge effort.
> With a new configuration item in hive-site.xml cluster operators can
> configure HiveServer2 to ignore the "set" command requests - essentially
> making those settings "final" on HiveServer2 level. Trying to change these
> "final" settings would not fail the scripts - just ignore their request.
> In this jira:
> - add a new config "hive.conf.ignored.variable.list"
> - accepts strings, comma separated list of variables
> - the config is empty by default, it can be set in hive-site.xml only
> - adding the "hive.conf.ignored.variable.list" to the restricted list
> ("hive.conf.restricted.list") internally - so it cannot be modified during
> runtime
> - adding tests for the changes
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-26915) Backport of HIVE-23692 TestCodahaleMetrics.testFileReporting is flaky
[ https://issues.apache.org/jira/browse/HIVE-26915?focusedWorklogId=839431=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-839431 ] ASF GitHub Bot logged work on HIVE-26915: - Author: ASF GitHub Bot Created on: 16/Jan/23 14:18 Start Date: 16/Jan/23 14:18 Worklog Time Spent: 10m Work Description: amanraj2520 commented on PR #3928: URL: https://github.com/apache/hive/pull/3928#issuecomment-1384133593 @zabetak Also here http://ci.hive.apache.org/blue/organizations/jenkins/hive-precommit/detail/PR-3859/11/tests/ Issue Time Tracking --- Worklog Id: (was: 839431) Time Spent: 50m (was: 40m) > Backport of HIVE-23692 TestCodahaleMetrics.testFileReporting is flaky > - > > Key: HIVE-26915 > URL: https://issues.apache.org/jira/browse/HIVE-26915 > Project: Hive > Issue Type: Sub-task >Reporter: Aman Raj >Assignee: Aman Raj >Priority: Major > Labels: pull-request-available > Time Spent: 50m > Remaining Estimate: 0h > > This was committed in master without a HIVE Jira task. This is the commit id > : 130f80445d589cdd82904cea1073c84d1368d079 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26916) Disable TestJdbcGenericUDTFGetSplits.testGenericUDTFOrderBySplitCount1 (Done as part of HIVE-22942)
[ https://issues.apache.org/jira/browse/HIVE-26916?focusedWorklogId=839430=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-839430 ] ASF GitHub Bot logged work on HIVE-26916: - Author: ASF GitHub Bot Created on: 16/Jan/23 14:17 Start Date: 16/Jan/23 14:17 Worklog Time Spent: 10m Work Description: amanraj2520 commented on PR #3929: URL: https://github.com/apache/hive/pull/3929#issuecomment-1384132926 Hi @zabetak Sorry for not attaching those instances. http://ci.hive.apache.org/blue/organizations/jenkins/hive-precommit/detail/PR-3859/11/tests/ Please refer to this build for the same. Issue Time Tracking --- Worklog Id: (was: 839430) Time Spent: 40m (was: 0.5h) > Disable TestJdbcGenericUDTFGetSplits.testGenericUDTFOrderBySplitCount1 (Done > as part of HIVE-22942) > --- > > Key: HIVE-26916 > URL: https://issues.apache.org/jira/browse/HIVE-26916 > Project: Hive > Issue Type: Sub-task >Reporter: Aman Raj >Assignee: Aman Raj >Priority: Major > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26915) Backport of HIVE-23692 TestCodahaleMetrics.testFileReporting is flaky
[ https://issues.apache.org/jira/browse/HIVE-26915?focusedWorklogId=839428=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-839428 ] ASF GitHub Bot logged work on HIVE-26915: - Author: ASF GitHub Bot Created on: 16/Jan/23 14:10 Start Date: 16/Jan/23 14:10 Worklog Time Spent: 10m Work Description: amanraj2520 commented on PR #3928: URL: https://github.com/apache/hive/pull/3928#issuecomment-1384121499 Hi @zabetak Please refer this http://ci.hive.apache.org/blue/organizations/jenkins/hive-precommit/detail/PR-3900/8/tests/. In this run it is failing. Issue Time Tracking --- Worklog Id: (was: 839428) Time Spent: 40m (was: 0.5h) > Backport of HIVE-23692 TestCodahaleMetrics.testFileReporting is flaky > - > > Key: HIVE-26915 > URL: https://issues.apache.org/jira/browse/HIVE-26915 > Project: Hive > Issue Type: Sub-task >Reporter: Aman Raj >Assignee: Aman Raj >Priority: Major > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > This was committed in master without a HIVE Jira task. This is the commit id > : 130f80445d589cdd82904cea1073c84d1368d079 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26896) Backport of Test fixes for lineage3.q and load_static_ptn_into_bucketed_table.q
[ https://issues.apache.org/jira/browse/HIVE-26896?focusedWorklogId=839422=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-839422 ] ASF GitHub Bot logged work on HIVE-26896: - Author: ASF GitHub Bot Created on: 16/Jan/23 13:52 Start Date: 16/Jan/23 13:52 Worklog Time Spent: 10m Work Description: zabetak commented on PR #3901: URL: https://github.com/apache/hive/pull/3901#issuecomment-1384091851 Hey @amanraj2520, I will merge this once I see that the two tests mentioned here are: # failing on the last precommit run in branch-3 # not failing on the last precommit run in this PR. `lineage3` seems to pass here http://ci.hive.apache.org/job/hive-precommit/job/branch-3/15 Issue Time Tracking --- Worklog Id: (was: 839422) Time Spent: 1h 20m (was: 1h 10m) > Backport of Test fixes for lineage3.q and > load_static_ptn_into_bucketed_table.q > --- > > Key: HIVE-26896 > URL: https://issues.apache.org/jira/browse/HIVE-26896 > Project: Hive > Issue Type: Sub-task >Reporter: Aman Raj >Assignee: Aman Raj >Priority: Critical > Labels: pull-request-available > Time Spent: 1h 20m > Remaining Estimate: 0h > > These tests were fixed in branch-3.1 so backporting them to branch-3 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26717) Query based Rebalance compaction on insert-only tables
[
https://issues.apache.org/jira/browse/HIVE-26717?focusedWorklogId=839420=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-839420
]
ASF GitHub Bot logged work on HIVE-26717:
-
Author: ASF GitHub Bot
Created on: 16/Jan/23 13:46
Start Date: 16/Jan/23 13:46
Worklog Time Spent: 10m
Work Description: deniskuzZ commented on code in PR #3935:
URL: https://github.com/apache/hive/pull/3935#discussion_r1071264712
##
ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/Worker.java:
##
@@ -315,10 +315,14 @@ protected Boolean findNextCompactionAndExecute(boolean
collectGenericStats, bool
return false;
}
- if (!ci.type.equals(CompactionType.REBALANCE) && ci.numberOfBuckets > 0)
{
-if (LOG.isWarnEnabled()) {
- LOG.warn("Only the REBALANCE compaction accepts the number of
buckets clause (CLUSTERED INTO {N} BUCKETS). " +
- "Since the compaction request is {}, it will be ignored.",
ci.type);
+ boolean insertOnly = AcidUtils.isInsertOnlyTable(table.getParameters());
+ if (LOG.isWarnEnabled() && ci.type.equals(CompactionType.REBALANCE) &&
insertOnly) {
+LOG.warn("REBALANCE compaction requested on an insert-only table ({}).
Falling back to MAJOR compaction as " +
+"REBALANCE compaction is supported only on full-acid tables",
table.getTableName());
+if (ci.numberOfBuckets > 0) {
+ LOG.warn("Only REBALANCE compaction on a full-acid table accepts the
number of buckets clause " +
+ "(CLUSTERED INTO {N} BUCKETS). Since the compaction request is
{} and the table is {}, it will be ignored.",
Review Comment:
why do we need a table type param here as it's only logged when table is
insert only? that is why i was confused with the above if change
Issue Time Tracking
---
Worklog Id: (was: 839420)
Time Spent: 1h 40m (was: 1.5h)
> Query based Rebalance compaction on insert-only tables
> --
>
> Key: HIVE-26717
> URL: https://issues.apache.org/jira/browse/HIVE-26717
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: László Végh
>Assignee: László Végh
>Priority: Major
> Labels: ACID, compaction, pull-request-available
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-26947) Hive compactor.Worker can respawn connections to HMS at extremely high frequency
[
https://issues.apache.org/jira/browse/HIVE-26947?focusedWorklogId=839418=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-839418
]
ASF GitHub Bot logged work on HIVE-26947:
-
Author: ASF GitHub Bot
Created on: 16/Jan/23 13:45
Start Date: 16/Jan/23 13:45
Worklog Time Spent: 10m
Work Description: veghlaci05 commented on code in PR #3955:
URL: https://github.com/apache/hive/pull/3955#discussion_r1071262323
##
ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/Worker.java:
##
@@ -118,19 +119,23 @@ public void run() {
singleRun.cancel(true);
executor.shutdownNow();
executor = getTimeoutHandlingExecutor();
+ err = true;
} catch (ExecutionException e) {
LOG.info("Exception during executing compaction", e);
+ err = true;
} catch (InterruptedException ie) {
// do not ignore interruption requests
return;
+} catch (Throwable t) {
+ err = true;
}
doPostLoopActions(System.currentTimeMillis() - startedAt);
// If we didn't try to launch a job it either means there was no work
to do or we got
// here as the result of a communication failure with the DB. Either
way we want to wait
// a bit before, otherwise we can start over the loop immediately.
Review Comment:
Please update the comment according to the new behavior
Issue Time Tracking
---
Worklog Id: (was: 839418)
Time Spent: 20m (was: 10m)
> Hive compactor.Worker can respawn connections to HMS at extremely high
> frequency
>
>
> Key: HIVE-26947
> URL: https://issues.apache.org/jira/browse/HIVE-26947
> Project: Hive
> Issue Type: Bug
>Reporter: Akshat Mathur
>Assignee: Akshat Mathur
>Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> After catching the exception generated by the findNextCompactionAndExecute()
> task, HS2 appears to immediately rerun the task with no delay or backoff. As
> a result there are ~3500 connection attempts from HS2 to HMS over just a 5
> second period in the HS2 log
> The compactor.Worker should wait between failed attempts and maybe do an
> exponential backoff.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-26825) Compactor: Cleaner shouldn't fetch table details again and again for partitioned tables
[ https://issues.apache.org/jira/browse/HIVE-26825?focusedWorklogId=839412=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-839412 ] ASF GitHub Bot logged work on HIVE-26825: - Author: ASF GitHub Bot Created on: 16/Jan/23 13:40 Start Date: 16/Jan/23 13:40 Worklog Time Spent: 10m Work Description: sonarcloud[bot] commented on PR #3864: URL: https://github.com/apache/hive/pull/3864#issuecomment-1384079035 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive=3864) [](https://sonarcloud.io/project/issues?id=apache_hive=3864=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=3864=false=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive=3864=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=3864=false=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive=3864=false=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive=3864=false=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3864=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3864=false=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3864=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive=3864=false=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive=3864=false=CODE_SMELL) [3 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive=3864=false=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive=3864=coverage=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive=3864=duplicated_lines_density=list) No Duplication information Issue Time Tracking --- Worklog Id: (was: 839412) Time Spent: 2h 20m (was: 2h 10m) > Compactor: Cleaner shouldn't fetch table details again and again for > partitioned tables > --- > > Key: HIVE-26825 > URL: https://issues.apache.org/jira/browse/HIVE-26825 > Project: Hive > Issue Type: Improvement > Components: Transactions >Reporter: KIRTI RUGE >Assignee: KIRTI RUGE >Priority: Major > Labels: pull-request-available > Time Spent: 2h 20m > Remaining Estimate: 0h > > Cleaner shouldn't be fetch table/partition details for all its partitions. > When there are large number of databases/tables, it takes lot of time for > Initiator to complete its initial iteration and load on DB also goes higher. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HIVE-26882) Allow transactional check of Table parameter before altering the Table
[ https://issues.apache.org/jira/browse/HIVE-26882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17677358#comment-17677358 ] Peter Vary commented on HIVE-26882: --- Thanks [~ayushtkn] for all the help! > Allow transactional check of Table parameter before altering the Table > -- > > Key: HIVE-26882 > URL: https://issues.apache.org/jira/browse/HIVE-26882 > Project: Hive > Issue Type: Improvement > Components: Standalone Metastore >Reporter: Peter Vary >Assignee: Peter Vary >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Time Spent: 4h 40m > Remaining Estimate: 0h > > We should add the possibility to transactionally check if a Table parameter > is changed before altering the table in the HMS. > This would provide an alternative, less error-prone and faster way to commit > an Iceberg table, as the Iceberg table currently needs to: > - Create an exclusive lock > - Get the table metadata to check if the current snapshot is not changed > - Update the table metadata > - Release the lock > After the change these 4 HMS calls could be substituted with a single alter > table call. > Also we could avoid cases where the locks are left hanging by failed processes -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26943) Fix NPE during Optimised Bootstrap when db is dropped
[ https://issues.apache.org/jira/browse/HIVE-26943?focusedWorklogId=839386=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-839386 ] ASF GitHub Bot logged work on HIVE-26943: - Author: ASF GitHub Bot Created on: 16/Jan/23 11:55 Start Date: 16/Jan/23 11:55 Worklog Time Spent: 10m Work Description: sonarcloud[bot] commented on PR #3953: URL: https://github.com/apache/hive/pull/3953#issuecomment-1383940496 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive=3953) [](https://sonarcloud.io/project/issues?id=apache_hive=3953=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=3953=false=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive=3953=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=3953=false=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive=3953=false=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive=3953=false=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3953=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3953=false=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3953=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive=3953=false=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive=3953=false=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive=3953=false=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive=3953=coverage=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive=3953=duplicated_lines_density=list) No Duplication information Issue Time Tracking --- Worklog Id: (was: 839386) Time Spent: 20m (was: 10m) > Fix NPE during Optimised Bootstrap when db is dropped > - > > Key: HIVE-26943 > URL: https://issues.apache.org/jira/browse/HIVE-26943 > Project: Hive > Issue Type: Task >Reporter: Shreenidhi >Assignee: Shreenidhi >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > Consider the steps: > 1. Current replication is from A (source) -> B(target) > 2. Failover is complete > so now A (target) <- B(source) > 3. Suppose db at A is dropped before reverse replication. > 4. Now when reverse replication triggers optimised bootstrap it will throw NPE > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26717) Query based Rebalance compaction on insert-only tables
[
https://issues.apache.org/jira/browse/HIVE-26717?focusedWorklogId=839379=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-839379
]
ASF GitHub Bot logged work on HIVE-26717:
-
Author: ASF GitHub Bot
Created on: 16/Jan/23 11:34
Start Date: 16/Jan/23 11:34
Worklog Time Spent: 10m
Work Description: veghlaci05 commented on code in PR #3935:
URL: https://github.com/apache/hive/pull/3935#discussion_r1071139423
##
ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/Worker.java:
##
@@ -315,10 +315,14 @@ protected Boolean findNextCompactionAndExecute(boolean
collectGenericStats, bool
return false;
}
- if (!ci.type.equals(CompactionType.REBALANCE) && ci.numberOfBuckets > 0)
{
-if (LOG.isWarnEnabled()) {
- LOG.warn("Only the REBALANCE compaction accepts the number of
buckets clause (CLUSTERED INTO {N} BUCKETS). " +
- "Since the compaction request is {}, it will be ignored.",
ci.type);
+ boolean insertOnly = AcidUtils.isInsertOnlyTable(table.getParameters());
+ if (LOG.isWarnEnabled() && ci.type.equals(CompactionType.REBALANCE) &&
insertOnly) {
Review Comment:
No, ci is checked at line 273
```
if (ci == null) {
return false;
}
```
type checked at
org.apache.hadoop.hive.metastore.txn.TxnUtils#dbCompactionType2ThriftType
Issue Time Tracking
---
Worklog Id: (was: 839379)
Time Spent: 1.5h (was: 1h 20m)
> Query based Rebalance compaction on insert-only tables
> --
>
> Key: HIVE-26717
> URL: https://issues.apache.org/jira/browse/HIVE-26717
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: László Végh
>Assignee: László Végh
>Priority: Major
> Labels: ACID, compaction, pull-request-available
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-26717) Query based Rebalance compaction on insert-only tables
[
https://issues.apache.org/jira/browse/HIVE-26717?focusedWorklogId=839376=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-839376
]
ASF GitHub Bot logged work on HIVE-26717:
-
Author: ASF GitHub Bot
Created on: 16/Jan/23 11:30
Start Date: 16/Jan/23 11:30
Worklog Time Spent: 10m
Work Description: veghlaci05 commented on code in PR #3935:
URL: https://github.com/apache/hive/pull/3935#discussion_r1071135779
##
ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/Worker.java:
##
@@ -315,10 +315,14 @@ protected Boolean findNextCompactionAndExecute(boolean
collectGenericStats, bool
return false;
}
- if (!ci.type.equals(CompactionType.REBALANCE) && ci.numberOfBuckets > 0)
{
-if (LOG.isWarnEnabled()) {
- LOG.warn("Only the REBALANCE compaction accepts the number of
buckets clause (CLUSTERED INTO {N} BUCKETS). " +
- "Since the compaction request is {}, it will be ignored.",
ci.type);
+ boolean insertOnly = AcidUtils.isInsertOnlyTable(table.getParameters());
+ if (LOG.isWarnEnabled() && ci.type.equals(CompactionType.REBALANCE) &&
insertOnly) {
+LOG.warn("REBALANCE compaction requested on an insert-only table ({}).
Falling back to MAJOR compaction as " +
+"REBALANCE compaction is supported only on full-acid tables",
table.getTableName());
+if (ci.numberOfBuckets > 0) {
Review Comment:
Yes, during the implementation of the insert-only REBALANCE, two things
turned out:
- only in case of full-acid tables were are accepting the number of buckets
- in case of insert-only tables the number of buckets are ignored like for
non REBALANCE compactions, and we are falling back to MM MAJOR compaction.
This logging block has been adjusted to reflect the changes above.
Issue Time Tracking
---
Worklog Id: (was: 839376)
Time Spent: 1h 20m (was: 1h 10m)
> Query based Rebalance compaction on insert-only tables
> --
>
> Key: HIVE-26717
> URL: https://issues.apache.org/jira/browse/HIVE-26717
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: László Végh
>Assignee: László Végh
>Priority: Major
> Labels: ACID, compaction, pull-request-available
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HIVE-26947) Hive compactor.Worker can respawn connections to HMS at extremely high frequency
[ https://issues.apache.org/jira/browse/HIVE-26947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akshat Mathur updated HIVE-26947: - Status: Patch Available (was: Open) > Hive compactor.Worker can respawn connections to HMS at extremely high > frequency > > > Key: HIVE-26947 > URL: https://issues.apache.org/jira/browse/HIVE-26947 > Project: Hive > Issue Type: Improvement >Reporter: Akshat Mathur >Assignee: Akshat Mathur >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > After catching the exception generated by the findNextCompactionAndExecute() > task, HS2 appears to immediately rerun the task with no delay or backoff. As > a result there are ~3500 connection attempts from HS2 to HMS over just a 5 > second period in the HS2 log > The compactor.Worker should wait between failed attempts and maybe do an > exponential backoff. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-26947) Hive compactor.Worker can respawn connections to HMS at extremely high frequency
[ https://issues.apache.org/jira/browse/HIVE-26947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akshat Mathur updated HIVE-26947: - Issue Type: Bug (was: Improvement) > Hive compactor.Worker can respawn connections to HMS at extremely high > frequency > > > Key: HIVE-26947 > URL: https://issues.apache.org/jira/browse/HIVE-26947 > Project: Hive > Issue Type: Bug >Reporter: Akshat Mathur >Assignee: Akshat Mathur >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > After catching the exception generated by the findNextCompactionAndExecute() > task, HS2 appears to immediately rerun the task with no delay or backoff. As > a result there are ~3500 connection attempts from HS2 to HMS over just a 5 > second period in the HS2 log > The compactor.Worker should wait between failed attempts and maybe do an > exponential backoff. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26947) Hive compactor.Worker can respawn connections to HMS at extremely high frequency
[ https://issues.apache.org/jira/browse/HIVE-26947?focusedWorklogId=839375=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-839375 ] ASF GitHub Bot logged work on HIVE-26947: - Author: ASF GitHub Bot Created on: 16/Jan/23 11:27 Start Date: 16/Jan/23 11:27 Worklog Time Spent: 10m Work Description: akshat0395 opened a new pull request, #3955: URL: https://github.com/apache/hive/pull/3955 ### What changes were proposed in this pull request? Add err flag, which is true when an error is caught which launching a worker job, and use this flag as check for thread to wait/sleep ### Why are the changes needed? [HIVE-26947](https://issues.apache.org/jira/browse/HIVE-26947) Avoid high frequency connection respawning to HMS in case of any exception ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? Unit tests Issue Time Tracking --- Worklog Id: (was: 839375) Remaining Estimate: 0h Time Spent: 10m > Hive compactor.Worker can respawn connections to HMS at extremely high > frequency > > > Key: HIVE-26947 > URL: https://issues.apache.org/jira/browse/HIVE-26947 > Project: Hive > Issue Type: Improvement >Reporter: Akshat Mathur >Assignee: Akshat Mathur >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > After catching the exception generated by the findNextCompactionAndExecute() > task, HS2 appears to immediately rerun the task with no delay or backoff. As > a result there are ~3500 connection attempts from HS2 to HMS over just a 5 > second period in the HS2 log > The compactor.Worker should wait between failed attempts and maybe do an > exponential backoff. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-26947) Hive compactor.Worker can respawn connections to HMS at extremely high frequency
[ https://issues.apache.org/jira/browse/HIVE-26947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-26947: -- Labels: pull-request-available (was: ) > Hive compactor.Worker can respawn connections to HMS at extremely high > frequency > > > Key: HIVE-26947 > URL: https://issues.apache.org/jira/browse/HIVE-26947 > Project: Hive > Issue Type: Improvement >Reporter: Akshat Mathur >Assignee: Akshat Mathur >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > After catching the exception generated by the findNextCompactionAndExecute() > task, HS2 appears to immediately rerun the task with no delay or backoff. As > a result there are ~3500 connection attempts from HS2 to HMS over just a 5 > second period in the HS2 log > The compactor.Worker should wait between failed attempts and maybe do an > exponential backoff. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26717) Query based Rebalance compaction on insert-only tables
[ https://issues.apache.org/jira/browse/HIVE-26717?focusedWorklogId=839374=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-839374 ] ASF GitHub Bot logged work on HIVE-26717: - Author: ASF GitHub Bot Created on: 16/Jan/23 11:23 Start Date: 16/Jan/23 11:23 Worklog Time Spent: 10m Work Description: veghlaci05 commented on code in PR #3935: URL: https://github.com/apache/hive/pull/3935#discussion_r1071130360 ## ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/CompactorFactory.java: ## @@ -99,10 +99,12 @@ public Compactor getCompactor(Table table, HiveConf configuration, CompactionInf case MINOR: return new MmMinorQueryCompactor(); case MAJOR: +case REBALANCE: + // REBALANCE COMPACTION on an insert-only table is simply a MAJOR compaction. Since there is no ACID row data, + // there is no acid row order to keep, and the number of buckets cannot be set at all (it will be calculated + // and created by TEZ dynamically). Initiator won't schedule REBALANCE compactions for insert-only tables, + // however users can request it. In these cases we simply fall back to MAJOR compaction return new MmMajorQueryCompactor(); -default: Review Comment: In that case the execution will jump to the last line of the method and throw an exception. However now I realized that the message needs to be actualized. Issue Time Tracking --- Worklog Id: (was: 839374) Time Spent: 1h 10m (was: 1h) > Query based Rebalance compaction on insert-only tables > -- > > Key: HIVE-26717 > URL: https://issues.apache.org/jira/browse/HIVE-26717 > Project: Hive > Issue Type: Sub-task > Components: Hive >Reporter: László Végh >Assignee: László Végh >Priority: Major > Labels: ACID, compaction, pull-request-available > Time Spent: 1h 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (HIVE-26947) Hive compactor.Worker can respawn connections to HMS at extremely high frequency
[ https://issues.apache.org/jira/browse/HIVE-26947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akshat Mathur reassigned HIVE-26947: Assignee: Akshat Mathur > Hive compactor.Worker can respawn connections to HMS at extremely high > frequency > > > Key: HIVE-26947 > URL: https://issues.apache.org/jira/browse/HIVE-26947 > Project: Hive > Issue Type: Improvement >Reporter: Akshat Mathur >Assignee: Akshat Mathur >Priority: Major > > After catching the exception generated by the findNextCompactionAndExecute() > task, HS2 appears to immediately rerun the task with no delay or backoff. As > a result there are ~3500 connection attempts from HS2 to HMS over just a 5 > second period in the HS2 log > The compactor.Worker should wait between failed attempts and maybe do an > exponential backoff. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26717) Query based Rebalance compaction on insert-only tables
[
https://issues.apache.org/jira/browse/HIVE-26717?focusedWorklogId=839370=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-839370
]
ASF GitHub Bot logged work on HIVE-26717:
-
Author: ASF GitHub Bot
Created on: 16/Jan/23 11:03
Start Date: 16/Jan/23 11:03
Worklog Time Spent: 10m
Work Description: kasakrisz commented on code in PR #3935:
URL: https://github.com/apache/hive/pull/3935#discussion_r1071107854
##
ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/Worker.java:
##
@@ -315,10 +315,14 @@ protected Boolean findNextCompactionAndExecute(boolean
collectGenericStats, bool
return false;
}
- if (!ci.type.equals(CompactionType.REBALANCE) && ci.numberOfBuckets > 0)
{
-if (LOG.isWarnEnabled()) {
- LOG.warn("Only the REBALANCE compaction accepts the number of
buckets clause (CLUSTERED INTO {N} BUCKETS). " +
- "Since the compaction request is {}, it will be ignored.",
ci.type);
+ boolean insertOnly = AcidUtils.isInsertOnlyTable(table.getParameters());
+ if (LOG.isWarnEnabled() && ci.type.equals(CompactionType.REBALANCE) &&
insertOnly) {
Review Comment:
Can `ci` and `ci.type` be null?
Issue Time Tracking
---
Worklog Id: (was: 839370)
Time Spent: 1h (was: 50m)
> Query based Rebalance compaction on insert-only tables
> --
>
> Key: HIVE-26717
> URL: https://issues.apache.org/jira/browse/HIVE-26717
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: László Végh
>Assignee: László Végh
>Priority: Major
> Labels: ACID, compaction, pull-request-available
> Time Spent: 1h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-26717) Query based Rebalance compaction on insert-only tables
[
https://issues.apache.org/jira/browse/HIVE-26717?focusedWorklogId=839368=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-839368
]
ASF GitHub Bot logged work on HIVE-26717:
-
Author: ASF GitHub Bot
Created on: 16/Jan/23 11:02
Start Date: 16/Jan/23 11:02
Worklog Time Spent: 10m
Work Description: deniskuzZ commented on code in PR #3935:
URL: https://github.com/apache/hive/pull/3935#discussion_r1071107521
##
ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/Worker.java:
##
@@ -315,10 +315,14 @@ protected Boolean findNextCompactionAndExecute(boolean
collectGenericStats, bool
return false;
}
- if (!ci.type.equals(CompactionType.REBALANCE) && ci.numberOfBuckets > 0)
{
-if (LOG.isWarnEnabled()) {
- LOG.warn("Only the REBALANCE compaction accepts the number of
buckets clause (CLUSTERED INTO {N} BUCKETS). " +
- "Since the compaction request is {}, it will be ignored.",
ci.type);
+ boolean insertOnly = AcidUtils.isInsertOnlyTable(table.getParameters());
+ if (LOG.isWarnEnabled() && ci.type.equals(CompactionType.REBALANCE) &&
insertOnly) {
+LOG.warn("REBALANCE compaction requested on an insert-only table ({}).
Falling back to MAJOR compaction as " +
+"REBALANCE compaction is supported only on full-acid tables",
table.getTableName());
+if (ci.numberOfBuckets > 0) {
Review Comment:
check was previously for `!ci.type.equals(CompactionType.REBALANCE)`, now
it's inside `ci.type.equals(CompactionType.REBALANCE) && insertOnly`, is it
expected?
Issue Time Tracking
---
Worklog Id: (was: 839368)
Time Spent: 50m (was: 40m)
> Query based Rebalance compaction on insert-only tables
> --
>
> Key: HIVE-26717
> URL: https://issues.apache.org/jira/browse/HIVE-26717
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: László Végh
>Assignee: László Végh
>Priority: Major
> Labels: ACID, compaction, pull-request-available
> Time Spent: 50m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-26717) Query based Rebalance compaction on insert-only tables
[ https://issues.apache.org/jira/browse/HIVE-26717?focusedWorklogId=839367=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-839367 ] ASF GitHub Bot logged work on HIVE-26717: - Author: ASF GitHub Bot Created on: 16/Jan/23 10:58 Start Date: 16/Jan/23 10:58 Worklog Time Spent: 10m Work Description: deniskuzZ commented on code in PR #3935: URL: https://github.com/apache/hive/pull/3935#discussion_r1071104172 ## ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/CompactorFactory.java: ## @@ -99,10 +99,12 @@ public Compactor getCompactor(Table table, HiveConf configuration, CompactionInf case MINOR: return new MmMinorQueryCompactor(); case MAJOR: +case REBALANCE: + // REBALANCE COMPACTION on an insert-only table is simply a MAJOR compaction. Since there is no ACID row data, + // there is no acid row order to keep, and the number of buckets cannot be set at all (it will be calculated + // and created by TEZ dynamically). Initiator won't schedule REBALANCE compactions for insert-only tables, + // however users can request it. In these cases we simply fall back to MAJOR compaction return new MmMajorQueryCompactor(); -default: Review Comment: why remove this, what if the user mistypes? Issue Time Tracking --- Worklog Id: (was: 839367) Time Spent: 40m (was: 0.5h) > Query based Rebalance compaction on insert-only tables > -- > > Key: HIVE-26717 > URL: https://issues.apache.org/jira/browse/HIVE-26717 > Project: Hive > Issue Type: Sub-task > Components: Hive >Reporter: László Végh >Assignee: László Végh >Priority: Major > Labels: ACID, compaction, pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26802) Create qtest running QB compaction queries
[ https://issues.apache.org/jira/browse/HIVE-26802?focusedWorklogId=839363=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-839363 ] ASF GitHub Bot logged work on HIVE-26802: - Author: ASF GitHub Bot Created on: 16/Jan/23 10:44 Start Date: 16/Jan/23 10:44 Worklog Time Spent: 10m Work Description: sonarcloud[bot] commented on PR #3882: URL: https://github.com/apache/hive/pull/3882#issuecomment-1383847336 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive=3882) [](https://sonarcloud.io/project/issues?id=apache_hive=3882=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=3882=false=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive=3882=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=3882=false=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive=3882=false=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive=3882=false=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3882=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3882=false=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3882=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive=3882=false=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive=3882=false=CODE_SMELL) [4 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive=3882=false=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive=3882=coverage=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive=3882=duplicated_lines_density=list) No Duplication information Issue Time Tracking --- Worklog Id: (was: 839363) Time Spent: 4h 40m (was: 4.5h) > Create qtest running QB compaction queries > -- > > Key: HIVE-26802 > URL: https://issues.apache.org/jira/browse/HIVE-26802 > Project: Hive > Issue Type: Improvement >Reporter: Zoltán Rátkai >Assignee: Zoltán Rátkai >Priority: Minor > Labels: pull-request-available > Time Spent: 4h 40m > Remaining Estimate: 0h > > Create a qtest that runs the queries that query-based compaction runs. > Not so much to check for correct data but more to check the query plans, to > simplify tracing changes in compilation that might affect QB compaction. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (HIVE-26735) Ability to sort the data during rebalancing compaction
[ https://issues.apache.org/jira/browse/HIVE-26735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Végh reassigned HIVE-26735: -- Assignee: László Végh > Ability to sort the data during rebalancing compaction > -- > > Key: HIVE-26735 > URL: https://issues.apache.org/jira/browse/HIVE-26735 > Project: Hive > Issue Type: Sub-task >Reporter: László Végh >Assignee: László Végh >Priority: Major > > Ability to sort the data during rebalancing compaction -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work started] (HIVE-26735) Ability to sort the data during rebalancing compaction
[ https://issues.apache.org/jira/browse/HIVE-26735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-26735 started by László Végh. -- > Ability to sort the data during rebalancing compaction > -- > > Key: HIVE-26735 > URL: https://issues.apache.org/jira/browse/HIVE-26735 > Project: Hive > Issue Type: Sub-task >Reporter: László Végh >Assignee: László Végh >Priority: Major > > Ability to sort the data during rebalancing compaction -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26711) The very first REPL Load should make the Target Database read-only
[
https://issues.apache.org/jira/browse/HIVE-26711?focusedWorklogId=839353=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-839353
]
ASF GitHub Bot logged work on HIVE-26711:
-
Author: ASF GitHub Bot
Created on: 16/Jan/23 09:59
Start Date: 16/Jan/23 09:59
Worklog Time Spent: 10m
Work Description: shreenidhiSaigaonkar commented on code in PR #3736:
URL: https://github.com/apache/hive/pull/3736#discussion_r1071043268
##
ql/src/java/org/apache/hadoop/hive/ql/exec/repl/ReplLoadTask.java:
##
@@ -420,9 +422,40 @@ a database ( directory )
// Ignore if no file.
}
}
+
+if(isReadOnlyHookRegistered()) {
Review Comment:
Done
Issue Time Tracking
---
Worklog Id: (was: 839353)
Time Spent: 1h (was: 50m)
> The very first REPL Load should make the Target Database read-only
> --
>
> Key: HIVE-26711
> URL: https://issues.apache.org/jira/browse/HIVE-26711
> Project: Hive
> Issue Type: Task
>Reporter: Shreenidhi
>Assignee: Shreenidhi
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h
> Remaining Estimate: 0h
>
> Use EnforceReadOnly hook to set TARGET database read only during BootStrap
> load.
> Also ensure backward compatibility.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-26711) The very first REPL Load should make the Target Database read-only
[
https://issues.apache.org/jira/browse/HIVE-26711?focusedWorklogId=839348=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-839348
]
ASF GitHub Bot logged work on HIVE-26711:
-
Author: ASF GitHub Bot
Created on: 16/Jan/23 09:37
Start Date: 16/Jan/23 09:37
Worklog Time Spent: 10m
Work Description: pudidic commented on code in PR #3736:
URL: https://github.com/apache/hive/pull/3736#discussion_r1071019470
##
ql/src/java/org/apache/hadoop/hive/ql/exec/repl/ReplLoadTask.java:
##
@@ -420,9 +422,40 @@ a database ( directory )
// Ignore if no file.
}
}
+
+if(isReadOnlyHookRegistered()) {
Review Comment:
Please follow coding convention `if(isReadOnlyHookRegistered()) {`
Issue Time Tracking
---
Worklog Id: (was: 839348)
Time Spent: 50m (was: 40m)
> The very first REPL Load should make the Target Database read-only
> --
>
> Key: HIVE-26711
> URL: https://issues.apache.org/jira/browse/HIVE-26711
> Project: Hive
> Issue Type: Task
>Reporter: Shreenidhi
>Assignee: Shreenidhi
>Priority: Major
> Labels: pull-request-available
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Use EnforceReadOnly hook to set TARGET database read only during BootStrap
> load.
> Also ensure backward compatibility.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-26925) MV with iceberg storage format fails when contains 'PARTITIONED ON' clause due to column number/types difference.
[
https://issues.apache.org/jira/browse/HIVE-26925?focusedWorklogId=839337=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-839337
]
ASF GitHub Bot logged work on HIVE-26925:
-
Author: ASF GitHub Bot
Created on: 16/Jan/23 09:18
Start Date: 16/Jan/23 09:18
Worklog Time Spent: 10m
Work Description: sonarcloud[bot] commented on PR #3939:
URL: https://github.com/apache/hive/pull/3939#issuecomment-1383719697
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive=3939)
[](https://sonarcloud.io/project/issues?id=apache_hive=3939=false=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive=3939=false=BUG)
[0
Bugs](https://sonarcloud.io/project/issues?id=apache_hive=3939=false=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive=3939=false=VULNERABILITY)
[](https://sonarcloud.io/project/issues?id=apache_hive=3939=false=VULNERABILITY)
[0
Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive=3939=false=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3939=false=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3939=false=SECURITY_HOTSPOT)
[0 Security
Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3939=false=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive=3939=false=CODE_SMELL)
[](https://sonarcloud.io/project/issues?id=apache_hive=3939=false=CODE_SMELL)
[0 Code
Smells](https://sonarcloud.io/project/issues?id=apache_hive=3939=false=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive=3939=coverage=list)
No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive=3939=duplicated_lines_density=list)
No Duplication information
Issue Time Tracking
---
Worklog Id: (was: 839337)
Time Spent: 50m (was: 40m)
> MV with iceberg storage format fails when contains 'PARTITIONED ON' clause
> due to column number/types difference.
> -
>
> Key: HIVE-26925
> URL: https://issues.apache.org/jira/browse/HIVE-26925
> Project: Hive
> Issue Type: Bug
> Components: Iceberg integration
>Reporter: Dharmik Thakkar
>Assignee: Krisztian Kasa
>Priority: Critical
> Labels: pull-request-available
> Time Spent: 50m
> Remaining Estimate: 0h
>
> MV with iceberg storage format fails when contains 'PARTITIONED ON' clause
> due to column number/types difference.
> {code:java}
> !!! annotations iceberg
> >>> use iceberg_test_db_hive;
> No rows affected
> >>> set hive.exec.max.dynamic.partitions=2000;
> >>> set hive.exec.max.dynamic.partitions.pernode=2000;
> >>> drop materialized view if exists mv_agg_gby_col_partitioned;
> >>> create materialized view mv_agg_gby_col_partitioned PARTITIONED ON (t)
> >>> stored by iceberg stored as orc tblproperties ('format-version'='1') as
> >>> select b,f,sum(b), sum(f),t from all100k group by b,f,v,c,t;
> >>> analyze table mv_agg_gby_col_partitioned compute statistics for columns;
> >>> set hive.explain.user=false;
> >>> explain select b,f,sum(b) from all100k where t=93 group by c,v,f,b;
> !!! match row_contains
> alias:
[jira] [Work logged] (HIVE-26597) Fix unsetting of db prop repl.target.for in ReplicationSemanticAnalyzer
[
https://issues.apache.org/jira/browse/HIVE-26597?focusedWorklogId=839336=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-839336
]
ASF GitHub Bot logged work on HIVE-26597:
-
Author: ASF GitHub Bot
Created on: 16/Jan/23 09:14
Start Date: 16/Jan/23 09:14
Worklog Time Spent: 10m
Work Description: pudidic commented on PR #3788:
URL: https://github.com/apache/hive/pull/3788#issuecomment-1383713734
Looks good to me, but orc_llap_counters.q is failing. Please push an empty
commit with `git commit --allow-empty -m 'Re-trigger CI'`.
Issue Time Tracking
---
Worklog Id: (was: 839336)
Time Spent: 0.5h (was: 20m)
> Fix unsetting of db prop repl.target.for in ReplicationSemanticAnalyzer
> ---
>
> Key: HIVE-26597
> URL: https://issues.apache.org/jira/browse/HIVE-26597
> Project: Hive
> Issue Type: Bug
>Reporter: Teddy Choi
>Assignee: Rakshith C
>Priority: Major
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> when repl policy is set from A -> B
> * *repl.target.for* is set on B.
> when failover is initiated
> * *repl.failover.endpoint* = *'TARGET'* is set on B.
>
> now when reverse policy is set up from {*}A <- B{*};
> there is a check in
> [ReplicationSemanticAnalyzer#initReplDump|https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/parse/ReplicationSemanticAnalyzer.java#L196]
> which checks for existence of these two properties and if they are set,
> it unsets the *repl.target.for* property.
> Because of this optimisedBootstrap won't be triggered because it checks for
> the existence of *repl.target.for* property during repl dump on target
> [HERE|https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/repl/OptimisedBootstrapUtils.java#L93].
>
> Fix : remove the code which unsets repl.target.for in
> ReplicationSemanticAnalyzer, because second dump cycle of optimized bootstrap
> unsets it.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HIVE-26945) Test fixes for query*.q files
[ https://issues.apache.org/jira/browse/HIVE-26945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-26945: -- Labels: pull-request-available (was: ) > Test fixes for query*.q files > - > > Key: HIVE-26945 > URL: https://issues.apache.org/jira/browse/HIVE-26945 > Project: Hive > Issue Type: Sub-task >Reporter: Aman Raj >Assignee: Aman Raj >Priority: Critical > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > The tests has outdated q.out files which need to be updated. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26945) Test fixes for query*.q files
[ https://issues.apache.org/jira/browse/HIVE-26945?focusedWorklogId=839329=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-839329 ] ASF GitHub Bot logged work on HIVE-26945: - Author: ASF GitHub Bot Created on: 16/Jan/23 08:57 Start Date: 16/Jan/23 08:57 Worklog Time Spent: 10m Work Description: amanraj2520 opened a new pull request, #3954: URL: https://github.com/apache/hive/pull/3954 JIRA link : https://issues.apache.org/jira/browse/HIVE-26945 ### What changes were proposed in this pull request? ### Why are the changes needed? ### Does this PR introduce _any_ user-facing change? ### How was this patch tested? Issue Time Tracking --- Worklog Id: (was: 839329) Remaining Estimate: 0h Time Spent: 10m > Test fixes for query*.q files > - > > Key: HIVE-26945 > URL: https://issues.apache.org/jira/browse/HIVE-26945 > Project: Hive > Issue Type: Sub-task >Reporter: Aman Raj >Assignee: Aman Raj >Priority: Critical > Time Spent: 10m > Remaining Estimate: 0h > > The tests has outdated q.out files which need to be updated. -- This message was sent by Atlassian Jira (v8.20.10#820010)
