[jira] [Assigned] (HIVE-27013) Provide an option to enable iceberg manifest caching via table properties
[
https://issues.apache.org/jira/browse/HIVE-27013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ayush Saxena reassigned HIVE-27013:
---
Assignee: Ayush Saxena
> Provide an option to enable iceberg manifest caching via table properties
> -
>
> Key: HIVE-27013
> URL: https://issues.apache.org/jira/browse/HIVE-27013
> Project: Hive
> Issue Type: Improvement
> Components: Iceberg integration
>Reporter: Rajesh Balamohan
>Assignee: Ayush Saxena
>Priority: Major
>
> {color:#22}I tried the following thinking that it would work with iceberg
> manifest caching; but it didn't.{color}
> {noformat}
> alter table store_sales set
> tblproperties('io.manifest.cache-enabled'='true');{noformat}
> {color:#22}Creating this ticket as a placeholder to fix the same.{color}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HIVE-27013) Provide an option to enable iceberg manifest caching via table properties
[
https://issues.apache.org/jira/browse/HIVE-27013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rajesh Balamohan updated HIVE-27013:
Description:
{color:#22}I tried the following thinking that it would work with iceberg
manifest caching; but it didn't.{color}
{noformat}
alter table store_sales set
tblproperties('io.manifest.cache-enabled'='true');{noformat}
{color:#22}Creating this ticket as a placeholder to fix the same.{color}
was:
{color:#22}I tried the following thinking that it would work with iceberg
manifest caching; but it didn't.{color}
{color:#22}{noformat}{color}
{color:#22}alter table store_sales set
tblproperties('io.manifest.cac{color}{color:#22}he-enabled'='true');
\{noformat}{color}
{color:#22}Creating this ticket as a placeholder to fix the same.{color}
> Provide an option to enable iceberg manifest caching via table properties
> -
>
> Key: HIVE-27013
> URL: https://issues.apache.org/jira/browse/HIVE-27013
> Project: Hive
> Issue Type: Improvement
> Components: Iceberg integration
>Reporter: Rajesh Balamohan
>Priority: Major
>
> {color:#22}I tried the following thinking that it would work with iceberg
> manifest caching; but it didn't.{color}
> {noformat}
> alter table store_sales set
> tblproperties('io.manifest.cache-enabled'='true');{noformat}
> {color:#22}Creating this ticket as a placeholder to fix the same.{color}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HIVE-27012) Upgrade JavaEWAH to 1.1.7
[ https://issues.apache.org/jira/browse/HIVE-27012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-27012: -- Labels: pull-request-available (was: ) > Upgrade JavaEWAH to 1.1.7 > - > > Key: HIVE-27012 > URL: https://issues.apache.org/jira/browse/HIVE-27012 > Project: Hive > Issue Type: Task >Reporter: Devaspati Krishnatri >Assignee: Devaspati Krishnatri >Priority: Major > Labels: pull-request-available > Attachments: tree.txt > > Time Spent: 10m > Remaining Estimate: 0h > > Upgrade JavaEWAH to 1.1.7 > JavaEWAH:0.3.2 is pulling in CVE-2022-29580. > CVE-2022-29580 is a High Severity CVE with CVSSv3 Score 7.8 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27012) Upgrade JavaEWAH to 1.1.7
[ https://issues.apache.org/jira/browse/HIVE-27012?focusedWorklogId=843096=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-843096 ] ASF GitHub Bot logged work on HIVE-27012: - Author: ASF GitHub Bot Created on: 02/Feb/23 07:11 Start Date: 02/Feb/23 07:11 Worklog Time Spent: 10m Work Description: devaspatikrishnatri opened a new pull request, #4010: URL: https://github.com/apache/hive/pull/4010 ### What changes were proposed in this pull request? ### Why are the changes needed? ### Does this PR introduce _any_ user-facing change? No --> ### How was this patch tested? Issue Time Tracking --- Worklog Id: (was: 843096) Remaining Estimate: 0h Time Spent: 10m > Upgrade JavaEWAH to 1.1.7 > - > > Key: HIVE-27012 > URL: https://issues.apache.org/jira/browse/HIVE-27012 > Project: Hive > Issue Type: Task >Reporter: Devaspati Krishnatri >Assignee: Devaspati Krishnatri >Priority: Major > Attachments: tree.txt > > Time Spent: 10m > Remaining Estimate: 0h > > Upgrade JavaEWAH to 1.1.7 > JavaEWAH:0.3.2 is pulling in CVE-2022-29580. > CVE-2022-29580 is a High Severity CVE with CVSSv3 Score 7.8 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-23935) Fetching primaryKey through beeline fails with NPE
[ https://issues.apache.org/jira/browse/HIVE-23935?focusedWorklogId=843094=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-843094 ] ASF GitHub Bot logged work on HIVE-23935: - Author: ASF GitHub Bot Created on: 02/Feb/23 07:10 Start Date: 02/Feb/23 07:10 Worklog Time Spent: 10m Work Description: ayushtkn opened a new pull request, #4009: URL: https://github.com/apache/hive/pull/4009 https://issues.apache.org/jira/browse/HIVE-23935 Entire Trace - ``` 0: jdbc:hive2://localhost:1> !primarykeys Persons Error: MetaException(message:java.lang.NullPointerException) (state=,code=0) org.apache.hive.service.cli.HiveSQLException: MetaException(message:java.lang.NullPointerException) at org.apache.hive.jdbc.Utils.verifySuccess(Utils.java:360) at org.apache.hive.jdbc.Utils.verifySuccess(Utils.java:351) at org.apache.hive.jdbc.HiveDatabaseMetaData.getPrimaryKeys(HiveDatabaseMetaData.java:573) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hive.beeline.Reflector.invoke(Reflector.java:89) at org.apache.hive.beeline.Commands.metadata(Commands.java:125) at org.apache.hive.beeline.Commands.primarykeys(Commands.java:231) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hive.beeline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:57) at org.apache.hive.beeline.BeeLine.execCommandWithPrefix(BeeLine.java:1465) at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:1504) at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:1364) at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:1134) at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:1082) at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:546) at org.apache.hive.beeline.BeeLine.main(BeeLine.java:528) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.util.RunJar.run(RunJar.java:323) at org.apache.hadoop.util.RunJar.main(RunJar.java:236) Caused by: org.apache.hive.service.cli.HiveSQLException: MetaException(message:java.lang.NullPointerException) at org.apache.hive.service.cli.operation.GetPrimaryKeysOperation.runInternal(GetPrimaryKeysOperation.java:120) at org.apache.hive.service.cli.operation.Operation.run(Operation.java:277) at org.apache.hive.service.cli.session.HiveSessionImpl.getPrimaryKeys(HiveSessionImpl.java:997) at org.apache.hive.service.cli.CLIService.getPrimaryKeys(CLIService.java:416) at org.apache.hive.service.cli.thrift.ThriftCLIService.GetPrimaryKeys(ThriftCLIService.java:838) at org.apache.hive.service.rpc.thrift.TCLIService$Processor$GetPrimaryKeys.getResult(TCLIService.java:1717) at org.apache.hive.service.rpc.thrift.TCLIService$Processor$GetPrimaryKeys.getResult(TCLIService.java:1702) at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) at org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:56) at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: MetaException(message:java.lang.NullPointerException) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newMetaException(HiveMetaStore.java:7921) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.throwMetaException(HiveMetaStore.java:9105) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_primary_keys(HiveMetaStore.java:9067) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
[jira] [Updated] (HIVE-27012) Upgrade JavaEWAH to 1.1.7
[ https://issues.apache.org/jira/browse/HIVE-27012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaspati Krishnatri updated HIVE-27012: Attachment: tree.txt > Upgrade JavaEWAH to 1.1.7 > - > > Key: HIVE-27012 > URL: https://issues.apache.org/jira/browse/HIVE-27012 > Project: Hive > Issue Type: Task >Reporter: Devaspati Krishnatri >Assignee: Devaspati Krishnatri >Priority: Major > Attachments: tree.txt > > > Upgrade JavaEWAH to 1.1.7 > JavaEWAH:0.3.2 is pulling in CVE-2022-29580. > CVE-2022-29580 is a High Severity CVE with CVSSv3 Score 7.8 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (HIVE-27012) Upgrade JavaEWAH to 1.1.7
[ https://issues.apache.org/jira/browse/HIVE-27012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaspati Krishnatri reassigned HIVE-27012: --- Assignee: Devaspati Krishnatri > Upgrade JavaEWAH to 1.1.7 > - > > Key: HIVE-27012 > URL: https://issues.apache.org/jira/browse/HIVE-27012 > Project: Hive > Issue Type: Task >Reporter: Devaspati Krishnatri >Assignee: Devaspati Krishnatri >Priority: Major > > Upgrade JavaEWAH to 1.1.7 > JavaEWAH:0.3.2 is pulling in CVE-2022-29580. > CVE-2022-29580 is a High Severity CVE with CVSSv3 Score 7.8 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27004) DateTimeFormatterBuilder cannot parse 'UTC+' in Java versions higher than 8
[
https://issues.apache.org/jira/browse/HIVE-27004?focusedWorklogId=843086=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-843086
]
ASF GitHub Bot logged work on HIVE-27004:
-
Author: ASF GitHub Bot
Created on: 02/Feb/23 06:27
Start Date: 02/Feb/23 06:27
Worklog Time Spent: 10m
Work Description: sonarcloud[bot] commented on PR #4008:
URL: https://github.com/apache/hive/pull/4008#issuecomment-1413217123
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive=4008)
[](https://sonarcloud.io/project/issues?id=apache_hive=4008=false=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive=4008=false=BUG)
[0
Bugs](https://sonarcloud.io/project/issues?id=apache_hive=4008=false=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive=4008=false=VULNERABILITY)
[](https://sonarcloud.io/project/issues?id=apache_hive=4008=false=VULNERABILITY)
[0
Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive=4008=false=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4008=false=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4008=false=SECURITY_HOTSPOT)
[0 Security
Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4008=false=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive=4008=false=CODE_SMELL)
[](https://sonarcloud.io/project/issues?id=apache_hive=4008=false=CODE_SMELL)
[1 Code
Smell](https://sonarcloud.io/project/issues?id=apache_hive=4008=false=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive=4008=coverage=list)
No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive=4008=duplicated_lines_density=list)
No Duplication information
Issue Time Tracking
---
Worklog Id: (was: 843086)
Time Spent: 40m (was: 0.5h)
> DateTimeFormatterBuilder cannot parse 'UTC+' in Java versions higher than 8
> ---
>
> Key: HIVE-27004
> URL: https://issues.apache.org/jira/browse/HIVE-27004
> Project: Hive
> Issue Type: Bug
> Components: Hive
> Environment: Java version used
> {code:java}
> openjdk version "11.0.16.1" 2022-08-12 LTS OpenJDK Runtime Environment
> Corretto-11.0.16.9.1 (build 11.0.16.1+9-LTS) OpenJDK 64-Bit Server VM
> Corretto-11.0.16.9.1 (build 11.0.16.1+9-LTS, mixed mode{code}
>
>Reporter: Anmol Sundaram
>Assignee: Anmol Sundaram
>Priority: Minor
> Labels: pull-request-available
> Attachments: HIVE-27004.patch
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> Some of the unit tests related to _DateTimeFormatter_ were failing in Java
> versions greater than 8 while working in Java 8.
> Example of a failing Unit Test :
> _org.apache.hadoop.hive.common.type.TestTimestampTZ#testComparision_
>
> {code:java}
> java.time.format.DateTimeParseException: Text '2017-04-14 18:00:00 UTC+08:00'
> could not be parsed, unparsed text found at index 23 at
> java.base/java.time.format.DateTimeFormatter.parseResolved0(DateTimeFormatter.java:2049)
> at
> java.base/java.time.format.DateTimeFormatter.parse(DateTimeFormatter.java:1874)
> at
>
[jira] [Work logged] (HIVE-26889) Implement array_join udf to concatenate the elements of an array with a specified delimiter
[
https://issues.apache.org/jira/browse/HIVE-26889?focusedWorklogId=843079=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-843079
]
ASF GitHub Bot logged work on HIVE-26889:
-
Author: ASF GitHub Bot
Created on: 02/Feb/23 05:40
Start Date: 02/Feb/23 05:40
Worklog Time Spent: 10m
Work Description: sonarcloud[bot] commented on PR #3896:
URL: https://github.com/apache/hive/pull/3896#issuecomment-1413184097
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive=3896)
[](https://sonarcloud.io/project/issues?id=apache_hive=3896=false=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive=3896=false=BUG)
[0
Bugs](https://sonarcloud.io/project/issues?id=apache_hive=3896=false=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive=3896=false=VULNERABILITY)
[](https://sonarcloud.io/project/issues?id=apache_hive=3896=false=VULNERABILITY)
[0
Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive=3896=false=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3896=false=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3896=false=SECURITY_HOTSPOT)
[0 Security
Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3896=false=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive=3896=false=CODE_SMELL)
[](https://sonarcloud.io/project/issues?id=apache_hive=3896=false=CODE_SMELL)
[0 Code
Smells](https://sonarcloud.io/project/issues?id=apache_hive=3896=false=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive=3896=coverage=list)
No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive=3896=duplicated_lines_density=list)
No Duplication information
Issue Time Tracking
---
Worklog Id: (was: 843079)
Time Spent: 2h (was: 1h 50m)
> Implement array_join udf to concatenate the elements of an array with a
> specified delimiter
> ---
>
> Key: HIVE-26889
> URL: https://issues.apache.org/jira/browse/HIVE-26889
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: Taraka Rama Rao Lethavadla
>Assignee: Taraka Rama Rao Lethavadla
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 2h
> Remaining Estimate: 0h
>
> *array_join(array, delimiter, replaceNull)* - concatenate the elements of an
> array with a specified delimiter
> Example:
> {noformat}
> > SELECT array_join(array(1, 2, 3,4), ',') FROM src LIMIT 1;
> 1,2,3,4
> > SELECT array_join(array(1, 2, NULL, 4), ',',':') FROM src LIMIT 1;
> 1,2,:,4{noformat}
> Returns NULL if array is null/empty
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27004) DateTimeFormatterBuilder cannot parse 'UTC+' in Java versions higher than 8
[
https://issues.apache.org/jira/browse/HIVE-27004?focusedWorklogId=843076=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-843076
]
ASF GitHub Bot logged work on HIVE-27004:
-
Author: ASF GitHub Bot
Created on: 02/Feb/23 05:34
Start Date: 02/Feb/23 05:34
Worklog Time Spent: 10m
Work Description: AnmolSun opened a new pull request, #4008:
URL: https://github.com/apache/hive/pull/4008
…TC+' in Java versions higher than 8.
### What changes were proposed in this pull request?
Make changes in
_common/src/java/org/apache/hadoop/hive/common/type/TimestampTZUtil.java_ to
prevent
### Why are the changes needed?
Required to prevent parsing related error in Java ver greater than 8
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Tested at my setup with related unit tests that were failing on Java ver
greater than 8
Issue Time Tracking
---
Worklog Id: (was: 843076)
Time Spent: 0.5h (was: 20m)
> DateTimeFormatterBuilder cannot parse 'UTC+' in Java versions higher than 8
> ---
>
> Key: HIVE-27004
> URL: https://issues.apache.org/jira/browse/HIVE-27004
> Project: Hive
> Issue Type: Bug
> Components: Hive
> Environment: Java version used
> {code:java}
> openjdk version "11.0.16.1" 2022-08-12 LTS OpenJDK Runtime Environment
> Corretto-11.0.16.9.1 (build 11.0.16.1+9-LTS) OpenJDK 64-Bit Server VM
> Corretto-11.0.16.9.1 (build 11.0.16.1+9-LTS, mixed mode{code}
>
>Reporter: Anmol Sundaram
>Assignee: Anmol Sundaram
>Priority: Minor
> Labels: pull-request-available
> Attachments: HIVE-27004.patch
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Some of the unit tests related to _DateTimeFormatter_ were failing in Java
> versions greater than 8 while working in Java 8.
> Example of a failing Unit Test :
> _org.apache.hadoop.hive.common.type.TestTimestampTZ#testComparision_
>
> {code:java}
> java.time.format.DateTimeParseException: Text '2017-04-14 18:00:00 UTC+08:00'
> could not be parsed, unparsed text found at index 23 at
> java.base/java.time.format.DateTimeFormatter.parseResolved0(DateTimeFormatter.java:2049)
> at
> java.base/java.time.format.DateTimeFormatter.parse(DateTimeFormatter.java:1874)
> at
> org.apache.hadoop.hive.common.type.TimestampTZUtil.parse(TimestampTZUtil.java:76)
> at
> org.apache.hadoop.hive.common.type.TimestampTZUtil.parse(TimestampTZUtil.java:64)
> at
> org.apache.hadoop.hive.common.type.TestTimestampTZ.testComparison(TestTimestampTZ.java:44)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method){code}
>
> The {{appendZoneText(TextStyle)}} method of
> [DateTimeFormatteBuilder|https://github.com/apache/hive/blob/master/common/src/java/org/apache/hadoop/hive/common/type/TimestampTZUtil.java#L82]
> is not able to parse the {{+}} symbol in {{2017-04-14 18:00:00 UTC+08:00}}
> when running [the
> test|https://github.com/apache/hive/blob/master/common/src/test/org/apache/hadoop/hive/common/type/TestTimestampTZ.java#L37]
> in Java 11 , while it is working fine with Java 8.
> According to the
> [doc|https://developer.android.com/reference/java/time/format/DateTimeFormatterBuilder#appendZoneText(java.time.format.TextStyle)]
> , {{appendZoneText(TextStyle)}} should be able to parse {{either the textual
> zone name, the zone ID or the offset}} and UTC+08:00 should come under offset
> as per the same
> [doc|https://developer.android.com/reference/java/time/format/DateTimeFormatterBuilder#appendOffset(java.lang.String,%20java.lang.String)].
> It seems, however, that "UTC" was explicitly removed [when parsed as ZoneText
> for higher Java
> versions|https://github.com/openjdk/jdk/commit/5c3a01591c5c945926636fdc9f164d60b5b4f29e?diff=unified#diff-5fcf976db1c06e8f44a8671356d7e34fdfbf5b057baa852e7c3e015c8797c889R4263]
> As a workaround, we can use {{appendZoneOrOffsetId()}} rather than
> {{appendZoneText().}}
> This ensures the tests are passing for Java 8+ and based on my testing, I
> didn’t see any regression of the change.
> Sample repro code - jdoodle.com/ia/D5e
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-26035) Explore moving to directsql for ObjectStore::addPartitions
[
https://issues.apache.org/jira/browse/HIVE-26035?focusedWorklogId=843075=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-843075
]
ASF GitHub Bot logged work on HIVE-26035:
-
Author: ASF GitHub Bot
Created on: 02/Feb/23 05:27
Start Date: 02/Feb/23 05:27
Worklog Time Spent: 10m
Work Description: dengzhhu653 commented on PR #3905:
URL: https://github.com/apache/hive/pull/3905#issuecomment-1413175823
Leave a minor comment, others look good to me.
Issue Time Tracking
---
Worklog Id: (was: 843075)
Time Spent: 7h 50m (was: 7h 40m)
> Explore moving to directsql for ObjectStore::addPartitions
> --
>
> Key: HIVE-26035
> URL: https://issues.apache.org/jira/browse/HIVE-26035
> Project: Hive
> Issue Type: Bug
>Reporter: Rajesh Balamohan
>Assignee: Venugopal Reddy K
>Priority: Major
> Labels: pull-request-available
> Time Spent: 7h 50m
> Remaining Estimate: 0h
>
> Currently {{addPartitions}} uses datanuclues and is super slow for large
> number of partitions. It will be good to move to direct sql. Lots of repeated
> SQLs can be avoided as well (e.g SDS, SERDE, TABLE_PARAMS)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-26035) Explore moving to directsql for ObjectStore::addPartitions
[
https://issues.apache.org/jira/browse/HIVE-26035?focusedWorklogId=843074=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-843074
]
ASF GitHub Bot logged work on HIVE-26035:
-
Author: ASF GitHub Bot
Created on: 02/Feb/23 05:26
Start Date: 02/Feb/23 05:26
Worklog Time Spent: 10m
Work Description: dengzhhu653 commented on code in PR #3905:
URL: https://github.com/apache/hive/pull/3905#discussion_r1094044633
##
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/DirectSqlInsertPart.java:
##
@@ -0,0 +1,804 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.metastore;
+
+import static org.apache.commons.lang3.StringUtils.repeat;
+import static org.apache.hadoop.hive.metastore.Batchable.NO_BATCHING;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map;
+
+import javax.jdo.PersistenceManager;
+
+import org.apache.hadoop.hive.metastore.api.MetaException;
+import org.apache.hadoop.hive.metastore.model.MColumnDescriptor;
+import org.apache.hadoop.hive.metastore.model.MFieldSchema;
+import org.apache.hadoop.hive.metastore.model.MOrder;
+import org.apache.hadoop.hive.metastore.model.MPartition;
+import org.apache.hadoop.hive.metastore.model.MPartitionColumnPrivilege;
+import org.apache.hadoop.hive.metastore.model.MPartitionPrivilege;
+import org.apache.hadoop.hive.metastore.model.MSerDeInfo;
+import org.apache.hadoop.hive.metastore.model.MStorageDescriptor;
+import org.apache.hadoop.hive.metastore.model.MStringList;
+import org.datanucleus.ExecutionContext;
+import org.datanucleus.api.jdo.JDOPersistenceManager;
+import org.datanucleus.identity.DatastoreId;
+import org.datanucleus.metadata.AbstractClassMetaData;
+import org.datanucleus.metadata.IdentityType;
+
+/**
+ * This class contains the methods to insert into tables on the underlying
database using direct SQL
+ */
+class DirectSqlInsertPart {
+ private final PersistenceManager pm;
+ private final DatabaseProduct dbType;
+ private final int batchSize;
+
+ public DirectSqlInsertPart(PersistenceManager pm, DatabaseProduct dbType,
int batchSize) {
+this.pm = pm;
+this.dbType = dbType;
+this.batchSize = batchSize;
+ }
+
+ /**
+ * Interface to execute multiple row insert query in batch for direct SQL
+ */
+ interface BatchExecutionContext {
+void execute(String batchQueryText, int batchRowCount, int
batchParamCount) throws MetaException;
+ }
+
+ private Long getDataStoreId(Class modelClass) throws MetaException {
+ExecutionContext ec = ((JDOPersistenceManager) pm).getExecutionContext();
+AbstractClassMetaData cmd =
ec.getMetaDataManager().getMetaDataForClass(modelClass,
ec.getClassLoaderResolver());
+if (cmd.getIdentityType() == IdentityType.DATASTORE) {
+ return (Long) ec.getStoreManager().getValueGenerationStrategyValue(ec,
cmd, -1);
+} else {
+ throw new MetaException("Identity type is not datastore.");
+}
+ }
+
+ private void insertInBatch(String tableName, String columns, int
columnCount, int rowCount,
+ BatchExecutionContext batchExecutionContext) throws MetaException {
+if (rowCount == 0 || columnCount == 0) {
+ return;
+}
+int maxRowsInBatch = (batchSize == NO_BATCHING) ? rowCount : batchSize;
+int maxBatches = rowCount / maxRowsInBatch;
+int last = rowCount % maxRowsInBatch;
+String rowFormat = "(" + repeat(",?", columnCount).substring(1) + ")";
+String query = "";
+if (maxBatches > 0) {
+ query = dbType.getBatchInsertQuery(tableName, columns, rowFormat,
maxRowsInBatch);
+}
+int batchParamCount = maxRowsInBatch * columnCount;
+for (int batch = 0; batch < maxBatches; batch++) {
+ batchExecutionContext.execute(query, maxRowsInBatch, batchParamCount);
+}
+if (last != 0) {
+ query = dbType.getBatchInsertQuery(tableName, columns, rowFormat, last);
+ batchExecutionContext.execute(query, last, last * columnCount);
+}
+ }
+
+ private void executeQuery(String
[jira] [Work logged] (HIVE-27004) DateTimeFormatterBuilder cannot parse 'UTC+' in Java versions higher than 8
[
https://issues.apache.org/jira/browse/HIVE-27004?focusedWorklogId=843073=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-843073
]
ASF GitHub Bot logged work on HIVE-27004:
-
Author: ASF GitHub Bot
Created on: 02/Feb/23 05:24
Start Date: 02/Feb/23 05:24
Worklog Time Spent: 10m
Work Description: AnmolSun closed pull request #4007:
[HIVE-27004](https://issues.apache.org/jira/browse/HIVE-27004) :
DateTimeFormatterBuilder#appendZoneText cannot parse 'UTC+' in Java …
URL: https://github.com/apache/hive/pull/4007
Issue Time Tracking
---
Worklog Id: (was: 843073)
Time Spent: 20m (was: 10m)
> DateTimeFormatterBuilder cannot parse 'UTC+' in Java versions higher than 8
> ---
>
> Key: HIVE-27004
> URL: https://issues.apache.org/jira/browse/HIVE-27004
> Project: Hive
> Issue Type: Bug
> Components: Hive
> Environment: Java version used
> {code:java}
> openjdk version "11.0.16.1" 2022-08-12 LTS OpenJDK Runtime Environment
> Corretto-11.0.16.9.1 (build 11.0.16.1+9-LTS) OpenJDK 64-Bit Server VM
> Corretto-11.0.16.9.1 (build 11.0.16.1+9-LTS, mixed mode{code}
>
>Reporter: Anmol Sundaram
>Assignee: Anmol Sundaram
>Priority: Minor
> Labels: pull-request-available
> Attachments: HIVE-27004.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Some of the unit tests related to _DateTimeFormatter_ were failing in Java
> versions greater than 8 while working in Java 8.
> Example of a failing Unit Test :
> _org.apache.hadoop.hive.common.type.TestTimestampTZ#testComparision_
>
> {code:java}
> java.time.format.DateTimeParseException: Text '2017-04-14 18:00:00 UTC+08:00'
> could not be parsed, unparsed text found at index 23 at
> java.base/java.time.format.DateTimeFormatter.parseResolved0(DateTimeFormatter.java:2049)
> at
> java.base/java.time.format.DateTimeFormatter.parse(DateTimeFormatter.java:1874)
> at
> org.apache.hadoop.hive.common.type.TimestampTZUtil.parse(TimestampTZUtil.java:76)
> at
> org.apache.hadoop.hive.common.type.TimestampTZUtil.parse(TimestampTZUtil.java:64)
> at
> org.apache.hadoop.hive.common.type.TestTimestampTZ.testComparison(TestTimestampTZ.java:44)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method){code}
>
> The {{appendZoneText(TextStyle)}} method of
> [DateTimeFormatteBuilder|https://github.com/apache/hive/blob/master/common/src/java/org/apache/hadoop/hive/common/type/TimestampTZUtil.java#L82]
> is not able to parse the {{+}} symbol in {{2017-04-14 18:00:00 UTC+08:00}}
> when running [the
> test|https://github.com/apache/hive/blob/master/common/src/test/org/apache/hadoop/hive/common/type/TestTimestampTZ.java#L37]
> in Java 11 , while it is working fine with Java 8.
> According to the
> [doc|https://developer.android.com/reference/java/time/format/DateTimeFormatterBuilder#appendZoneText(java.time.format.TextStyle)]
> , {{appendZoneText(TextStyle)}} should be able to parse {{either the textual
> zone name, the zone ID or the offset}} and UTC+08:00 should come under offset
> as per the same
> [doc|https://developer.android.com/reference/java/time/format/DateTimeFormatterBuilder#appendOffset(java.lang.String,%20java.lang.String)].
> It seems, however, that "UTC" was explicitly removed [when parsed as ZoneText
> for higher Java
> versions|https://github.com/openjdk/jdk/commit/5c3a01591c5c945926636fdc9f164d60b5b4f29e?diff=unified#diff-5fcf976db1c06e8f44a8671356d7e34fdfbf5b057baa852e7c3e015c8797c889R4263]
> As a workaround, we can use {{appendZoneOrOffsetId()}} rather than
> {{appendZoneText().}}
> This ensures the tests are passing for Java 8+ and based on my testing, I
> didn’t see any regression of the change.
> Sample repro code - jdoodle.com/ia/D5e
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27004) DateTimeFormatterBuilder cannot parse 'UTC+' in Java versions higher than 8
[
https://issues.apache.org/jira/browse/HIVE-27004?focusedWorklogId=843072=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-843072
]
ASF GitHub Bot logged work on HIVE-27004:
-
Author: ASF GitHub Bot
Created on: 02/Feb/23 05:18
Start Date: 02/Feb/23 05:18
Worklog Time Spent: 10m
Work Description: AnmolSun opened a new pull request, #4007:
URL: https://github.com/apache/hive/pull/4007
[…versions](https://issues.apache.org/jira/browse/HIVE-27004) higher than 8.
### What changes were proposed in this pull request?
Changes in
_common/src/java/org/apache/hadoop/hive/common/type/TimestampTZUtil.java_
### Why are the changes needed?
DateTimeFormatterBuilder cannot parse 'UTC+' in Java versions higher than 8
and throws an error.
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
The unit tests at my setup which were failing for Java ver. greater than 8
worked.
Issue Time Tracking
---
Worklog Id: (was: 843072)
Remaining Estimate: 0h
Time Spent: 10m
> DateTimeFormatterBuilder cannot parse 'UTC+' in Java versions higher than 8
> ---
>
> Key: HIVE-27004
> URL: https://issues.apache.org/jira/browse/HIVE-27004
> Project: Hive
> Issue Type: Bug
> Components: Hive
> Environment: Java version used
> {code:java}
> openjdk version "11.0.16.1" 2022-08-12 LTS OpenJDK Runtime Environment
> Corretto-11.0.16.9.1 (build 11.0.16.1+9-LTS) OpenJDK 64-Bit Server VM
> Corretto-11.0.16.9.1 (build 11.0.16.1+9-LTS, mixed mode{code}
>
>Reporter: Anmol Sundaram
>Assignee: Anmol Sundaram
>Priority: Minor
> Attachments: HIVE-27004.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Some of the unit tests related to _DateTimeFormatter_ were failing in Java
> versions greater than 8 while working in Java 8.
> Example of a failing Unit Test :
> _org.apache.hadoop.hive.common.type.TestTimestampTZ#testComparision_
>
> {code:java}
> java.time.format.DateTimeParseException: Text '2017-04-14 18:00:00 UTC+08:00'
> could not be parsed, unparsed text found at index 23 at
> java.base/java.time.format.DateTimeFormatter.parseResolved0(DateTimeFormatter.java:2049)
> at
> java.base/java.time.format.DateTimeFormatter.parse(DateTimeFormatter.java:1874)
> at
> org.apache.hadoop.hive.common.type.TimestampTZUtil.parse(TimestampTZUtil.java:76)
> at
> org.apache.hadoop.hive.common.type.TimestampTZUtil.parse(TimestampTZUtil.java:64)
> at
> org.apache.hadoop.hive.common.type.TestTimestampTZ.testComparison(TestTimestampTZ.java:44)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method){code}
>
> The {{appendZoneText(TextStyle)}} method of
> [DateTimeFormatteBuilder|https://github.com/apache/hive/blob/master/common/src/java/org/apache/hadoop/hive/common/type/TimestampTZUtil.java#L82]
> is not able to parse the {{+}} symbol in {{2017-04-14 18:00:00 UTC+08:00}}
> when running [the
> test|https://github.com/apache/hive/blob/master/common/src/test/org/apache/hadoop/hive/common/type/TestTimestampTZ.java#L37]
> in Java 11 , while it is working fine with Java 8.
> According to the
> [doc|https://developer.android.com/reference/java/time/format/DateTimeFormatterBuilder#appendZoneText(java.time.format.TextStyle)]
> , {{appendZoneText(TextStyle)}} should be able to parse {{either the textual
> zone name, the zone ID or the offset}} and UTC+08:00 should come under offset
> as per the same
> [doc|https://developer.android.com/reference/java/time/format/DateTimeFormatterBuilder#appendOffset(java.lang.String,%20java.lang.String)].
> It seems, however, that "UTC" was explicitly removed [when parsed as ZoneText
> for higher Java
> versions|https://github.com/openjdk/jdk/commit/5c3a01591c5c945926636fdc9f164d60b5b4f29e?diff=unified#diff-5fcf976db1c06e8f44a8671356d7e34fdfbf5b057baa852e7c3e015c8797c889R4263]
> As a workaround, we can use {{appendZoneOrOffsetId()}} rather than
> {{appendZoneText().}}
> This ensures the tests are passing for Java 8+ and based on my testing, I
> didn’t see any regression of the change.
> Sample repro code - jdoodle.com/ia/D5e
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HIVE-27004) DateTimeFormatterBuilder cannot parse 'UTC+' in Java versions higher than 8
[
https://issues.apache.org/jira/browse/HIVE-27004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HIVE-27004:
--
Labels: pull-request-available (was: )
> DateTimeFormatterBuilder cannot parse 'UTC+' in Java versions higher than 8
> ---
>
> Key: HIVE-27004
> URL: https://issues.apache.org/jira/browse/HIVE-27004
> Project: Hive
> Issue Type: Bug
> Components: Hive
> Environment: Java version used
> {code:java}
> openjdk version "11.0.16.1" 2022-08-12 LTS OpenJDK Runtime Environment
> Corretto-11.0.16.9.1 (build 11.0.16.1+9-LTS) OpenJDK 64-Bit Server VM
> Corretto-11.0.16.9.1 (build 11.0.16.1+9-LTS, mixed mode{code}
>
>Reporter: Anmol Sundaram
>Assignee: Anmol Sundaram
>Priority: Minor
> Labels: pull-request-available
> Attachments: HIVE-27004.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Some of the unit tests related to _DateTimeFormatter_ were failing in Java
> versions greater than 8 while working in Java 8.
> Example of a failing Unit Test :
> _org.apache.hadoop.hive.common.type.TestTimestampTZ#testComparision_
>
> {code:java}
> java.time.format.DateTimeParseException: Text '2017-04-14 18:00:00 UTC+08:00'
> could not be parsed, unparsed text found at index 23 at
> java.base/java.time.format.DateTimeFormatter.parseResolved0(DateTimeFormatter.java:2049)
> at
> java.base/java.time.format.DateTimeFormatter.parse(DateTimeFormatter.java:1874)
> at
> org.apache.hadoop.hive.common.type.TimestampTZUtil.parse(TimestampTZUtil.java:76)
> at
> org.apache.hadoop.hive.common.type.TimestampTZUtil.parse(TimestampTZUtil.java:64)
> at
> org.apache.hadoop.hive.common.type.TestTimestampTZ.testComparison(TestTimestampTZ.java:44)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method){code}
>
> The {{appendZoneText(TextStyle)}} method of
> [DateTimeFormatteBuilder|https://github.com/apache/hive/blob/master/common/src/java/org/apache/hadoop/hive/common/type/TimestampTZUtil.java#L82]
> is not able to parse the {{+}} symbol in {{2017-04-14 18:00:00 UTC+08:00}}
> when running [the
> test|https://github.com/apache/hive/blob/master/common/src/test/org/apache/hadoop/hive/common/type/TestTimestampTZ.java#L37]
> in Java 11 , while it is working fine with Java 8.
> According to the
> [doc|https://developer.android.com/reference/java/time/format/DateTimeFormatterBuilder#appendZoneText(java.time.format.TextStyle)]
> , {{appendZoneText(TextStyle)}} should be able to parse {{either the textual
> zone name, the zone ID or the offset}} and UTC+08:00 should come under offset
> as per the same
> [doc|https://developer.android.com/reference/java/time/format/DateTimeFormatterBuilder#appendOffset(java.lang.String,%20java.lang.String)].
> It seems, however, that "UTC" was explicitly removed [when parsed as ZoneText
> for higher Java
> versions|https://github.com/openjdk/jdk/commit/5c3a01591c5c945926636fdc9f164d60b5b4f29e?diff=unified#diff-5fcf976db1c06e8f44a8671356d7e34fdfbf5b057baa852e7c3e015c8797c889R4263]
> As a workaround, we can use {{appendZoneOrOffsetId()}} rather than
> {{appendZoneText().}}
> This ensures the tests are passing for Java 8+ and based on my testing, I
> didn’t see any regression of the change.
> Sample repro code - jdoodle.com/ia/D5e
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-26921) Add failover_type, failover_endpoint to replication metrics metadata
[
https://issues.apache.org/jira/browse/HIVE-26921?focusedWorklogId=843071=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-843071
]
ASF GitHub Bot logged work on HIVE-26921:
-
Author: ASF GitHub Bot
Created on: 02/Feb/23 05:10
Start Date: 02/Feb/23 05:10
Worklog Time Spent: 10m
Work Description: pudidic commented on code in PR #3999:
URL: https://github.com/apache/hive/pull/3999#discussion_r1094030970
##
standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/common/repl/ReplConst.java:
##
@@ -82,4 +84,10 @@ public class ReplConst {
public static final String BOOTSTRAP_DUMP_STATE_KEY_PREFIX =
"bootstrap.dump.state.";
public static final String READ_ONLY_HOOK =
"org.apache.hadoop.hive.ql.hooks.EnforceReadOnlyDatabaseHook";
+
+ /**
+ * Type of failover
+ */
+ public static final String PLANNED_FAILOVER = "PLANNED";
Review Comment:
It's usually better to create an enum for types.
https://docs.oracle.com/javase/tutorial/java/javaOO/enum.html
Issue Time Tracking
---
Worklog Id: (was: 843071)
Time Spent: 40m (was: 0.5h)
> Add failover_type, failover_endpoint to replication metrics metadata
>
>
> Key: HIVE-26921
> URL: https://issues.apache.org/jira/browse/HIVE-26921
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Reporter: Amit Saonerkar
>Assignee: Amit Saonerkar
>Priority: Major
> Labels: pull-request-available
> Time Spent: 40m
> Remaining Estimate: 0h
>
> Corresponding to CDPD-46494
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HIVE-27004) DateTimeFormatterBuilder cannot parse 'UTC+' in Java versions higher than 8
[
https://issues.apache.org/jira/browse/HIVE-27004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anmol Sundaram updated HIVE-27004:
--
Attachment: HIVE-27004.patch
Status: Patch Available (was: Open)
> DateTimeFormatterBuilder cannot parse 'UTC+' in Java versions higher than 8
> ---
>
> Key: HIVE-27004
> URL: https://issues.apache.org/jira/browse/HIVE-27004
> Project: Hive
> Issue Type: Bug
> Components: Hive
> Environment: Java version used
> {code:java}
> openjdk version "11.0.16.1" 2022-08-12 LTS OpenJDK Runtime Environment
> Corretto-11.0.16.9.1 (build 11.0.16.1+9-LTS) OpenJDK 64-Bit Server VM
> Corretto-11.0.16.9.1 (build 11.0.16.1+9-LTS, mixed mode{code}
>
>Reporter: Anmol Sundaram
>Assignee: Anmol Sundaram
>Priority: Minor
> Attachments: HIVE-27004.patch
>
>
> Some of the unit tests related to _DateTimeFormatter_ were failing in Java
> versions greater than 8 while working in Java 8.
> Example of a failing Unit Test :
> _org.apache.hadoop.hive.common.type.TestTimestampTZ#testComparision_
>
> {code:java}
> java.time.format.DateTimeParseException: Text '2017-04-14 18:00:00 UTC+08:00'
> could not be parsed, unparsed text found at index 23 at
> java.base/java.time.format.DateTimeFormatter.parseResolved0(DateTimeFormatter.java:2049)
> at
> java.base/java.time.format.DateTimeFormatter.parse(DateTimeFormatter.java:1874)
> at
> org.apache.hadoop.hive.common.type.TimestampTZUtil.parse(TimestampTZUtil.java:76)
> at
> org.apache.hadoop.hive.common.type.TimestampTZUtil.parse(TimestampTZUtil.java:64)
> at
> org.apache.hadoop.hive.common.type.TestTimestampTZ.testComparison(TestTimestampTZ.java:44)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method){code}
>
> The {{appendZoneText(TextStyle)}} method of
> [DateTimeFormatteBuilder|https://github.com/apache/hive/blob/master/common/src/java/org/apache/hadoop/hive/common/type/TimestampTZUtil.java#L82]
> is not able to parse the {{+}} symbol in {{2017-04-14 18:00:00 UTC+08:00}}
> when running [the
> test|https://github.com/apache/hive/blob/master/common/src/test/org/apache/hadoop/hive/common/type/TestTimestampTZ.java#L37]
> in Java 11 , while it is working fine with Java 8.
> According to the
> [doc|https://developer.android.com/reference/java/time/format/DateTimeFormatterBuilder#appendZoneText(java.time.format.TextStyle)]
> , {{appendZoneText(TextStyle)}} should be able to parse {{either the textual
> zone name, the zone ID or the offset}} and UTC+08:00 should come under offset
> as per the same
> [doc|https://developer.android.com/reference/java/time/format/DateTimeFormatterBuilder#appendOffset(java.lang.String,%20java.lang.String)].
> It seems, however, that "UTC" was explicitly removed [when parsed as ZoneText
> for higher Java
> versions|https://github.com/openjdk/jdk/commit/5c3a01591c5c945926636fdc9f164d60b5b4f29e?diff=unified#diff-5fcf976db1c06e8f44a8671356d7e34fdfbf5b057baa852e7c3e015c8797c889R4263]
> As a workaround, we can use {{appendZoneOrOffsetId()}} rather than
> {{appendZoneText().}}
> This ensures the tests are passing for Java 8+ and based on my testing, I
> didn’t see any regression of the change.
> Sample repro code - jdoodle.com/ia/D5e
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HIVE-27004) DateTimeFormatterBuilder cannot parse 'UTC+' in Java versions higher than 8
[
https://issues.apache.org/jira/browse/HIVE-27004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anmol Sundaram updated HIVE-27004:
--
Attachment: (was: HIVE-27004.patch)
> DateTimeFormatterBuilder cannot parse 'UTC+' in Java versions higher than 8
> ---
>
> Key: HIVE-27004
> URL: https://issues.apache.org/jira/browse/HIVE-27004
> Project: Hive
> Issue Type: Bug
> Components: Hive
> Environment: Java version used
> {code:java}
> openjdk version "11.0.16.1" 2022-08-12 LTS OpenJDK Runtime Environment
> Corretto-11.0.16.9.1 (build 11.0.16.1+9-LTS) OpenJDK 64-Bit Server VM
> Corretto-11.0.16.9.1 (build 11.0.16.1+9-LTS, mixed mode{code}
>
>Reporter: Anmol Sundaram
>Assignee: Anmol Sundaram
>Priority: Minor
> Attachments: HIVE-27004.patch
>
>
> Some of the unit tests related to _DateTimeFormatter_ were failing in Java
> versions greater than 8 while working in Java 8.
> Example of a failing Unit Test :
> _org.apache.hadoop.hive.common.type.TestTimestampTZ#testComparision_
>
> {code:java}
> java.time.format.DateTimeParseException: Text '2017-04-14 18:00:00 UTC+08:00'
> could not be parsed, unparsed text found at index 23 at
> java.base/java.time.format.DateTimeFormatter.parseResolved0(DateTimeFormatter.java:2049)
> at
> java.base/java.time.format.DateTimeFormatter.parse(DateTimeFormatter.java:1874)
> at
> org.apache.hadoop.hive.common.type.TimestampTZUtil.parse(TimestampTZUtil.java:76)
> at
> org.apache.hadoop.hive.common.type.TimestampTZUtil.parse(TimestampTZUtil.java:64)
> at
> org.apache.hadoop.hive.common.type.TestTimestampTZ.testComparison(TestTimestampTZ.java:44)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method){code}
>
> The {{appendZoneText(TextStyle)}} method of
> [DateTimeFormatteBuilder|https://github.com/apache/hive/blob/master/common/src/java/org/apache/hadoop/hive/common/type/TimestampTZUtil.java#L82]
> is not able to parse the {{+}} symbol in {{2017-04-14 18:00:00 UTC+08:00}}
> when running [the
> test|https://github.com/apache/hive/blob/master/common/src/test/org/apache/hadoop/hive/common/type/TestTimestampTZ.java#L37]
> in Java 11 , while it is working fine with Java 8.
> According to the
> [doc|https://developer.android.com/reference/java/time/format/DateTimeFormatterBuilder#appendZoneText(java.time.format.TextStyle)]
> , {{appendZoneText(TextStyle)}} should be able to parse {{either the textual
> zone name, the zone ID or the offset}} and UTC+08:00 should come under offset
> as per the same
> [doc|https://developer.android.com/reference/java/time/format/DateTimeFormatterBuilder#appendOffset(java.lang.String,%20java.lang.String)].
> It seems, however, that "UTC" was explicitly removed [when parsed as ZoneText
> for higher Java
> versions|https://github.com/openjdk/jdk/commit/5c3a01591c5c945926636fdc9f164d60b5b4f29e?diff=unified#diff-5fcf976db1c06e8f44a8671356d7e34fdfbf5b057baa852e7c3e015c8797c889R4263]
> As a workaround, we can use {{appendZoneOrOffsetId()}} rather than
> {{appendZoneText().}}
> This ensures the tests are passing for Java 8+ and based on my testing, I
> didn’t see any regression of the change.
> Sample repro code - jdoodle.com/ia/D5e
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27010) Reduce compilation time
[ https://issues.apache.org/jira/browse/HIVE-27010?focusedWorklogId=843057=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-843057 ] ASF GitHub Bot logged work on HIVE-27010: - Author: ASF GitHub Bot Created on: 02/Feb/23 03:23 Start Date: 02/Feb/23 03:23 Worklog Time Spent: 10m Work Description: rbalamohan commented on PR #4005: URL: https://github.com/apache/hive/pull/4005#issuecomment-1413097340 Thanks for the review @ayushtkn. Issue Time Tracking --- Worklog Id: (was: 843057) Time Spent: 40m (was: 0.5h) > Reduce compilation time > --- > > Key: HIVE-27010 > URL: https://issues.apache.org/jira/browse/HIVE-27010 > Project: Hive > Issue Type: Improvement >Reporter: Rajesh Balamohan >Assignee: Rajesh Balamohan >Priority: Major > Labels: performance, pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > Context: Post HIVE-24645, compilation time for queries has increased. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HIVE-27010) Reduce compilation time
[ https://issues.apache.org/jira/browse/HIVE-27010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan resolved HIVE-27010. - Resolution: Fixed > Reduce compilation time > --- > > Key: HIVE-27010 > URL: https://issues.apache.org/jira/browse/HIVE-27010 > Project: Hive > Issue Type: Improvement >Reporter: Rajesh Balamohan >Assignee: Rajesh Balamohan >Priority: Major > Labels: performance, pull-request-available > Time Spent: 50m > Remaining Estimate: 0h > > Context: Post HIVE-24645, compilation time for queries has increased. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27010) Reduce compilation time
[ https://issues.apache.org/jira/browse/HIVE-27010?focusedWorklogId=843058=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-843058 ] ASF GitHub Bot logged work on HIVE-27010: - Author: ASF GitHub Bot Created on: 02/Feb/23 03:23 Start Date: 02/Feb/23 03:23 Worklog Time Spent: 10m Work Description: rbalamohan merged PR #4005: URL: https://github.com/apache/hive/pull/4005 Issue Time Tracking --- Worklog Id: (was: 843058) Time Spent: 50m (was: 40m) > Reduce compilation time > --- > > Key: HIVE-27010 > URL: https://issues.apache.org/jira/browse/HIVE-27010 > Project: Hive > Issue Type: Improvement >Reporter: Rajesh Balamohan >Assignee: Rajesh Balamohan >Priority: Major > Labels: performance, pull-request-available > Time Spent: 50m > Remaining Estimate: 0h > > Context: Post HIVE-24645, compilation time for queries has increased. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27007) Iceberg: Use BasicStats from iceberg table's currrentSnapshot.summary() for query planning
[ https://issues.apache.org/jira/browse/HIVE-27007?focusedWorklogId=843023=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-843023 ] ASF GitHub Bot logged work on HIVE-27007: - Author: ASF GitHub Bot Created on: 01/Feb/23 21:10 Start Date: 01/Feb/23 21:10 Worklog Time Spent: 10m Work Description: sonarcloud[bot] commented on PR #4000: URL: https://github.com/apache/hive/pull/4000#issuecomment-1412727326 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive=4000) [](https://sonarcloud.io/project/issues?id=apache_hive=4000=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=4000=false=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive=4000=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=4000=false=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive=4000=false=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive=4000=false=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4000=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4000=false=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4000=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive=4000=false=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive=4000=false=CODE_SMELL) [2 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive=4000=false=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive=4000=coverage=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive=4000=duplicated_lines_density=list) No Duplication information Issue Time Tracking --- Worklog Id: (was: 843023) Time Spent: 2h 20m (was: 2h 10m) > Iceberg: Use BasicStats from iceberg table's currrentSnapshot.summary() for > query planning > -- > > Key: HIVE-27007 > URL: https://issues.apache.org/jira/browse/HIVE-27007 > Project: Hive > Issue Type: Improvement >Reporter: Simhadri Govindappa >Assignee: Simhadri Govindappa >Priority: Major > Labels: pull-request-available > Time Spent: 2h 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26838) Add a new event to improve cache performance in external systems that communicates with HMS.
[ https://issues.apache.org/jira/browse/HIVE-26838?focusedWorklogId=843018=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-843018 ] ASF GitHub Bot logged work on HIVE-26838: - Author: ASF GitHub Bot Created on: 01/Feb/23 20:35 Start Date: 01/Feb/23 20:35 Worklog Time Spent: 10m Work Description: sonarcloud[bot] commented on PR #3861: URL: https://github.com/apache/hive/pull/3861#issuecomment-1412688178 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive=3861) [](https://sonarcloud.io/project/issues?id=apache_hive=3861=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=3861=false=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive=3861=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=3861=false=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive=3861=false=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive=3861=false=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3861=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3861=false=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3861=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive=3861=false=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive=3861=false=CODE_SMELL) [4 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive=3861=false=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive=3861=coverage=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive=3861=duplicated_lines_density=list) No Duplication information Issue Time Tracking --- Worklog Id: (was: 843018) Time Spent: 2h 40m (was: 2.5h) > Add a new event to improve cache performance in external systems that > communicates with HMS. > > > Key: HIVE-26838 > URL: https://issues.apache.org/jira/browse/HIVE-26838 > Project: Hive > Issue Type: New Feature > Components: Hive, Standalone Metastore >Reporter: Sai Hemanth Gantasala >Assignee: Sai Hemanth Gantasala >Priority: Major > Labels: pull-request-available > Time Spent: 2h 40m > Remaining Estimate: 0h > > Adding support for a new event "Reload event" in the HMS (HiveMetaStore). > This event can be used by external services that depend on HMS for metadata > operations to improve its cache performance. In the distributed environment > where there are replicas of an external service (with its own cache in each > of these replicas) talking to HMS for metadata operations, the reload event > can be used to address the cache performance and ensure consistency among all > the replicas for a given table/partition. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27007) Iceberg: Use BasicStats from iceberg table's currrentSnapshot.summary() for query planning
[
https://issues.apache.org/jira/browse/HIVE-27007?focusedWorklogId=842992=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-842992
]
ASF GitHub Bot logged work on HIVE-27007:
-
Author: ASF GitHub Bot
Created on: 01/Feb/23 18:44
Start Date: 01/Feb/23 18:44
Worklog Time Spent: 10m
Work Description: akshat0395 commented on code in PR #4000:
URL: https://github.com/apache/hive/pull/4000#discussion_r1093602583
##
ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java:
##
@@ -454,6 +468,26 @@ private static Statistics collectStatistics(HiveConf conf,
PrunedPartitionList p
return stats;
}
+ private static Map getValidBasisStatsFromStorageHandler(Table
table) {
Review Comment:
Seems like a typo in method name, Should it be
`getValidBasicStatsFromStorageHandler`?
Issue Time Tracking
---
Worklog Id: (was: 842992)
Time Spent: 2h 10m (was: 2h)
> Iceberg: Use BasicStats from iceberg table's currrentSnapshot.summary() for
> query planning
> --
>
> Key: HIVE-27007
> URL: https://issues.apache.org/jira/browse/HIVE-27007
> Project: Hive
> Issue Type: Improvement
>Reporter: Simhadri Govindappa
>Assignee: Simhadri Govindappa
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HIVE-27004) DateTimeFormatterBuilder cannot parse 'UTC+' in Java versions higher than 8
[
https://issues.apache.org/jira/browse/HIVE-27004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17683115#comment-17683115
]
Sai Hemanth Gantasala commented on HIVE-27004:
--
[~AnmolSun] - Can you upload your patch in the form of GitHub PR? That would be
easier to review and merge the patch.
> DateTimeFormatterBuilder cannot parse 'UTC+' in Java versions higher than 8
> ---
>
> Key: HIVE-27004
> URL: https://issues.apache.org/jira/browse/HIVE-27004
> Project: Hive
> Issue Type: Bug
> Components: Hive
> Environment: Java version used
> {code:java}
> openjdk version "11.0.16.1" 2022-08-12 LTS OpenJDK Runtime Environment
> Corretto-11.0.16.9.1 (build 11.0.16.1+9-LTS) OpenJDK 64-Bit Server VM
> Corretto-11.0.16.9.1 (build 11.0.16.1+9-LTS, mixed mode{code}
>
>Reporter: Anmol Sundaram
>Assignee: Anmol Sundaram
>Priority: Minor
> Attachments: HIVE-27004.patch
>
>
> Some of the unit tests related to _DateTimeFormatter_ were failing in Java
> versions greater than 8 while working in Java 8.
> Example of a failing Unit Test :
> _org.apache.hadoop.hive.common.type.TestTimestampTZ#testComparision_
>
> {code:java}
> java.time.format.DateTimeParseException: Text '2017-04-14 18:00:00 UTC+08:00'
> could not be parsed, unparsed text found at index 23 at
> java.base/java.time.format.DateTimeFormatter.parseResolved0(DateTimeFormatter.java:2049)
> at
> java.base/java.time.format.DateTimeFormatter.parse(DateTimeFormatter.java:1874)
> at
> org.apache.hadoop.hive.common.type.TimestampTZUtil.parse(TimestampTZUtil.java:76)
> at
> org.apache.hadoop.hive.common.type.TimestampTZUtil.parse(TimestampTZUtil.java:64)
> at
> org.apache.hadoop.hive.common.type.TestTimestampTZ.testComparison(TestTimestampTZ.java:44)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method){code}
>
> The {{appendZoneText(TextStyle)}} method of
> [DateTimeFormatteBuilder|https://github.com/apache/hive/blob/master/common/src/java/org/apache/hadoop/hive/common/type/TimestampTZUtil.java#L82]
> is not able to parse the {{+}} symbol in {{2017-04-14 18:00:00 UTC+08:00}}
> when running [the
> test|https://github.com/apache/hive/blob/master/common/src/test/org/apache/hadoop/hive/common/type/TestTimestampTZ.java#L37]
> in Java 11 , while it is working fine with Java 8.
> According to the
> [doc|https://developer.android.com/reference/java/time/format/DateTimeFormatterBuilder#appendZoneText(java.time.format.TextStyle)]
> , {{appendZoneText(TextStyle)}} should be able to parse {{either the textual
> zone name, the zone ID or the offset}} and UTC+08:00 should come under offset
> as per the same
> [doc|https://developer.android.com/reference/java/time/format/DateTimeFormatterBuilder#appendOffset(java.lang.String,%20java.lang.String)].
> It seems, however, that "UTC" was explicitly removed [when parsed as ZoneText
> for higher Java
> versions|https://github.com/openjdk/jdk/commit/5c3a01591c5c945926636fdc9f164d60b5b4f29e?diff=unified#diff-5fcf976db1c06e8f44a8671356d7e34fdfbf5b057baa852e7c3e015c8797c889R4263]
> As a workaround, we can use {{appendZoneOrOffsetId()}} rather than
> {{appendZoneText().}}
> This ensures the tests are passing for Java 8+ and based on my testing, I
> didn’t see any regression of the change.
> Sample repro code - jdoodle.com/ia/D5e
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HIVE-27004) DateTimeFormatterBuilder cannot parse 'UTC+' in Java versions higher than 8
[
https://issues.apache.org/jira/browse/HIVE-27004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17683110#comment-17683110
]
Stamatis Zampetakis commented on HIVE-27004:
Hey team, I can review but not this week; too busy with other stuff. Please
ping me next week if nobody looks into this in the meantime.
> DateTimeFormatterBuilder cannot parse 'UTC+' in Java versions higher than 8
> ---
>
> Key: HIVE-27004
> URL: https://issues.apache.org/jira/browse/HIVE-27004
> Project: Hive
> Issue Type: Bug
> Components: Hive
> Environment: Java version used
> {code:java}
> openjdk version "11.0.16.1" 2022-08-12 LTS OpenJDK Runtime Environment
> Corretto-11.0.16.9.1 (build 11.0.16.1+9-LTS) OpenJDK 64-Bit Server VM
> Corretto-11.0.16.9.1 (build 11.0.16.1+9-LTS, mixed mode{code}
>
>Reporter: Anmol Sundaram
>Assignee: Anmol Sundaram
>Priority: Minor
> Attachments: HIVE-27004.patch
>
>
> Some of the unit tests related to _DateTimeFormatter_ were failing in Java
> versions greater than 8 while working in Java 8.
> Example of a failing Unit Test :
> _org.apache.hadoop.hive.common.type.TestTimestampTZ#testComparision_
>
> {code:java}
> java.time.format.DateTimeParseException: Text '2017-04-14 18:00:00 UTC+08:00'
> could not be parsed, unparsed text found at index 23 at
> java.base/java.time.format.DateTimeFormatter.parseResolved0(DateTimeFormatter.java:2049)
> at
> java.base/java.time.format.DateTimeFormatter.parse(DateTimeFormatter.java:1874)
> at
> org.apache.hadoop.hive.common.type.TimestampTZUtil.parse(TimestampTZUtil.java:76)
> at
> org.apache.hadoop.hive.common.type.TimestampTZUtil.parse(TimestampTZUtil.java:64)
> at
> org.apache.hadoop.hive.common.type.TestTimestampTZ.testComparison(TestTimestampTZ.java:44)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method){code}
>
> The {{appendZoneText(TextStyle)}} method of
> [DateTimeFormatteBuilder|https://github.com/apache/hive/blob/master/common/src/java/org/apache/hadoop/hive/common/type/TimestampTZUtil.java#L82]
> is not able to parse the {{+}} symbol in {{2017-04-14 18:00:00 UTC+08:00}}
> when running [the
> test|https://github.com/apache/hive/blob/master/common/src/test/org/apache/hadoop/hive/common/type/TestTimestampTZ.java#L37]
> in Java 11 , while it is working fine with Java 8.
> According to the
> [doc|https://developer.android.com/reference/java/time/format/DateTimeFormatterBuilder#appendZoneText(java.time.format.TextStyle)]
> , {{appendZoneText(TextStyle)}} should be able to parse {{either the textual
> zone name, the zone ID or the offset}} and UTC+08:00 should come under offset
> as per the same
> [doc|https://developer.android.com/reference/java/time/format/DateTimeFormatterBuilder#appendOffset(java.lang.String,%20java.lang.String)].
> It seems, however, that "UTC" was explicitly removed [when parsed as ZoneText
> for higher Java
> versions|https://github.com/openjdk/jdk/commit/5c3a01591c5c945926636fdc9f164d60b5b4f29e?diff=unified#diff-5fcf976db1c06e8f44a8671356d7e34fdfbf5b057baa852e7c3e015c8797c889R4263]
> As a workaround, we can use {{appendZoneOrOffsetId()}} rather than
> {{appendZoneText().}}
> This ensures the tests are passing for Java 8+ and based on my testing, I
> didn’t see any regression of the change.
> Sample repro code - jdoodle.com/ia/D5e
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-26971) Test fix in load_data_using_job.q.out (Done as part of HIVE-20593 addendum patch)
[
https://issues.apache.org/jira/browse/HIVE-26971?focusedWorklogId=842969=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-842969
]
ASF GitHub Bot logged work on HIVE-26971:
-
Author: ASF GitHub Bot
Created on: 01/Feb/23 17:45
Start Date: 01/Feb/23 17:45
Worklog Time Spent: 10m
Work Description: zabetak commented on PR #3973:
URL: https://github.com/apache/hive/pull/3973#issuecomment-1412466675
@amanraj2520 You created HIVE-26971 so there is a specific JIRA for the test
failure; not sure what you mean that there is no specific JIRA for this.
The commit
https://github.com/apache/hive/commit/dca6ef04e376ea1e18cd8986c3baf210b51d7677
mentions HIVE-20593 but as I wrote above the same reason does not seem to apply
here.
HIVE-20593 is in branch-3 for a while now so if the problem was HIVE-20593
then the test should fail from the beginning that we started running CI in
branch-3; not after merging 15 patches. There somethings more to it.
Issue Time Tracking
---
Worklog Id: (was: 842969)
Time Spent: 1h 20m (was: 1h 10m)
> Test fix in load_data_using_job.q.out (Done as part of HIVE-20593 addendum
> patch)
> -
>
> Key: HIVE-26971
> URL: https://issues.apache.org/jira/browse/HIVE-26971
> Project: Hive
> Issue Type: Sub-task
>Reporter: Aman Raj
>Assignee: Aman Raj
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> This fix was done is master using this commit id : dca6ef04 (HIVE-20593
> addendum)
>
> There are test failures are as follows :
> h4. Stacktrace
> java.lang.AssertionError:
> Client Execution succeeded but contained differences (error code = 1) after
> executing load_data_using_job.q
> 3031,3032d3030
> < PREHOOK: Input: default@orc_test_txn_{_}temp_table_for_load_data{_}_
> < PREHOOK: Output: default@orc_test_txn
> 3035d3032
> < POSTHOOK: Input: default@orc_test_txn_{_}temp_table_for_load_data{_}_
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (HIVE-27011) Default value of PartitionManagementTask frequency should be set to higher value
[ https://issues.apache.org/jira/browse/HIVE-27011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Riju Trivedi reassigned HIVE-27011: --- > Default value of PartitionManagementTask frequency should be set to higher > value > > > Key: HIVE-27011 > URL: https://issues.apache.org/jira/browse/HIVE-27011 > Project: Hive > Issue Type: Improvement > Components: Metastore >Reporter: Riju Trivedi >Assignee: Riju Trivedi >Priority: Major > > Default for "metastore.partition.management.task.frequency" is 5 mins, less > than ideal for Prod scenarios. When there are a large number of > databases/tables, it takes a lot of time for PartitionManagementTask to scan > all tables and partitions and doesn't complete within 5 mins. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27007) Iceberg: Use BasicStats from iceberg table's currrentSnapshot.summary() for query planning
[
https://issues.apache.org/jira/browse/HIVE-27007?focusedWorklogId=842901=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-842901
]
ASF GitHub Bot logged work on HIVE-27007:
-
Author: ASF GitHub Bot
Created on: 01/Feb/23 14:31
Start Date: 01/Feb/23 14:31
Worklog Time Spent: 10m
Work Description: simhadri-g commented on code in PR #4000:
URL: https://github.com/apache/hive/pull/4000#discussion_r1093299476
##
ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java:
##
@@ -277,23 +277,31 @@ private static Statistics collectStatistics(HiveConf
conf, PrunedPartitionList p
boolean metaTable = table.getMetaTable() != null;
if (!table.isPartitioned()) {
+ long ds, nr, fs;
+ if (table.isNonNative() &&
table.getStorageHandler().canProvideBasicStatistics()) {
+Map icebergBasicStatMap =
table.getStorageHandler().getBasicStatistics(Partish.buildFor(table));
+ds = Long.parseLong(icebergBasicStatMap.get("totalSize"));
+nr = Long.parseLong(icebergBasicStatMap.get("numRows"));
+fs = Long.parseLong(icebergBasicStatMap.get("numFiles"));
Review Comment:
Thanks, @kasakrisz , I have updated the PR to catch numberFormatException
and fall back to the pre-existing implementation of basic stats in case the
storage handler cannot provide the stats.
Please have a look at the updated PR.
Issue Time Tracking
---
Worklog Id: (was: 842901)
Time Spent: 2h (was: 1h 50m)
> Iceberg: Use BasicStats from iceberg table's currrentSnapshot.summary() for
> query planning
> --
>
> Key: HIVE-27007
> URL: https://issues.apache.org/jira/browse/HIVE-27007
> Project: Hive
> Issue Type: Improvement
>Reporter: Simhadri Govindappa
>Assignee: Simhadri Govindappa
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27007) Iceberg: Use BasicStats from iceberg table's currrentSnapshot.summary() for query planning
[
https://issues.apache.org/jira/browse/HIVE-27007?focusedWorklogId=842897=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-842897
]
ASF GitHub Bot logged work on HIVE-27007:
-
Author: ASF GitHub Bot
Created on: 01/Feb/23 14:27
Start Date: 01/Feb/23 14:27
Worklog Time Spent: 10m
Work Description: simhadri-g commented on code in PR #4000:
URL: https://github.com/apache/hive/pull/4000#discussion_r1093299476
##
ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java:
##
@@ -277,23 +277,31 @@ private static Statistics collectStatistics(HiveConf
conf, PrunedPartitionList p
boolean metaTable = table.getMetaTable() != null;
if (!table.isPartitioned()) {
+ long ds, nr, fs;
+ if (table.isNonNative() &&
table.getStorageHandler().canProvideBasicStatistics()) {
+Map icebergBasicStatMap =
table.getStorageHandler().getBasicStatistics(Partish.buildFor(table));
+ds = Long.parseLong(icebergBasicStatMap.get("totalSize"));
+nr = Long.parseLong(icebergBasicStatMap.get("numRows"));
+fs = Long.parseLong(icebergBasicStatMap.get("numFiles"));
Review Comment:
Thanks @kasakrisz , i have updated the PR to catch numberFormatException and
fall back to the pre existing implementation of basic stats.
Issue Time Tracking
---
Worklog Id: (was: 842897)
Time Spent: 1h 50m (was: 1h 40m)
> Iceberg: Use BasicStats from iceberg table's currrentSnapshot.summary() for
> query planning
> --
>
> Key: HIVE-27007
> URL: https://issues.apache.org/jira/browse/HIVE-27007
> Project: Hive
> Issue Type: Improvement
>Reporter: Simhadri Govindappa
>Assignee: Simhadri Govindappa
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-26921) Add failover_type, failover_endpoint to replication metrics metadata
[ https://issues.apache.org/jira/browse/HIVE-26921?focusedWorklogId=842890=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-842890 ] ASF GitHub Bot logged work on HIVE-26921: - Author: ASF GitHub Bot Created on: 01/Feb/23 14:09 Start Date: 01/Feb/23 14:09 Worklog Time Spent: 10m Work Description: sonarcloud[bot] commented on PR #3999: URL: https://github.com/apache/hive/pull/3999#issuecomment-1412121563 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive=3999) [](https://sonarcloud.io/project/issues?id=apache_hive=3999=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=3999=false=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive=3999=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=3999=false=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive=3999=false=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive=3999=false=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3999=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3999=false=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3999=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive=3999=false=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive=3999=false=CODE_SMELL) [3 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive=3999=false=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive=3999=coverage=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive=3999=duplicated_lines_density=list) No Duplication information Issue Time Tracking --- Worklog Id: (was: 842890) Time Spent: 0.5h (was: 20m) > Add failover_type, failover_endpoint to replication metrics metadata > > > Key: HIVE-26921 > URL: https://issues.apache.org/jira/browse/HIVE-26921 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Reporter: Amit Saonerkar >Assignee: Amit Saonerkar >Priority: Major > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > > Corresponding to CDPD-46494 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26889) Implement array_join udf to concatenate the elements of an array with a specified delimiter
[
https://issues.apache.org/jira/browse/HIVE-26889?focusedWorklogId=842886=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-842886
]
ASF GitHub Bot logged work on HIVE-26889:
-
Author: ASF GitHub Bot
Created on: 01/Feb/23 14:02
Start Date: 01/Feb/23 14:02
Worklog Time Spent: 10m
Work Description: sonarcloud[bot] commented on PR #3896:
URL: https://github.com/apache/hive/pull/3896#issuecomment-1412109496
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive=3896)
[](https://sonarcloud.io/project/issues?id=apache_hive=3896=false=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive=3896=false=BUG)
[0
Bugs](https://sonarcloud.io/project/issues?id=apache_hive=3896=false=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive=3896=false=VULNERABILITY)
[](https://sonarcloud.io/project/issues?id=apache_hive=3896=false=VULNERABILITY)
[0
Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive=3896=false=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3896=false=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3896=false=SECURITY_HOTSPOT)
[0 Security
Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3896=false=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive=3896=false=CODE_SMELL)
[](https://sonarcloud.io/project/issues?id=apache_hive=3896=false=CODE_SMELL)
[0 Code
Smells](https://sonarcloud.io/project/issues?id=apache_hive=3896=false=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive=3896=coverage=list)
No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive=3896=duplicated_lines_density=list)
No Duplication information
Issue Time Tracking
---
Worklog Id: (was: 842886)
Time Spent: 1h 50m (was: 1h 40m)
> Implement array_join udf to concatenate the elements of an array with a
> specified delimiter
> ---
>
> Key: HIVE-26889
> URL: https://issues.apache.org/jira/browse/HIVE-26889
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: Taraka Rama Rao Lethavadla
>Assignee: Taraka Rama Rao Lethavadla
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> *array_join(array, delimiter, replaceNull)* - concatenate the elements of an
> array with a specified delimiter
> Example:
> {noformat}
> > SELECT array_join(array(1, 2, 3,4), ',') FROM src LIMIT 1;
> 1,2,3,4
> > SELECT array_join(array(1, 2, NULL, 4), ',',':') FROM src LIMIT 1;
> 1,2,:,4{noformat}
> Returns NULL if array is null/empty
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HIVE-27004) DateTimeFormatterBuilder cannot parse 'UTC+' in Java versions higher than 8
[
https://issues.apache.org/jira/browse/HIVE-27004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17683008#comment-17683008
]
Anmol Sundaram commented on HIVE-27004:
---
Thanks a lot [~zabetak] and [~hemanth619] . Would really appreciate your help
in reviewing the patch.
> DateTimeFormatterBuilder cannot parse 'UTC+' in Java versions higher than 8
> ---
>
> Key: HIVE-27004
> URL: https://issues.apache.org/jira/browse/HIVE-27004
> Project: Hive
> Issue Type: Bug
> Components: Hive
> Environment: Java version used
> {code:java}
> openjdk version "11.0.16.1" 2022-08-12 LTS OpenJDK Runtime Environment
> Corretto-11.0.16.9.1 (build 11.0.16.1+9-LTS) OpenJDK 64-Bit Server VM
> Corretto-11.0.16.9.1 (build 11.0.16.1+9-LTS, mixed mode{code}
>
>Reporter: Anmol Sundaram
>Assignee: Anmol Sundaram
>Priority: Minor
> Attachments: HIVE-27004.patch
>
>
> Some of the unit tests related to _DateTimeFormatter_ were failing in Java
> versions greater than 8 while working in Java 8.
> Example of a failing Unit Test :
> _org.apache.hadoop.hive.common.type.TestTimestampTZ#testComparision_
>
> {code:java}
> java.time.format.DateTimeParseException: Text '2017-04-14 18:00:00 UTC+08:00'
> could not be parsed, unparsed text found at index 23 at
> java.base/java.time.format.DateTimeFormatter.parseResolved0(DateTimeFormatter.java:2049)
> at
> java.base/java.time.format.DateTimeFormatter.parse(DateTimeFormatter.java:1874)
> at
> org.apache.hadoop.hive.common.type.TimestampTZUtil.parse(TimestampTZUtil.java:76)
> at
> org.apache.hadoop.hive.common.type.TimestampTZUtil.parse(TimestampTZUtil.java:64)
> at
> org.apache.hadoop.hive.common.type.TestTimestampTZ.testComparison(TestTimestampTZ.java:44)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method){code}
>
> The {{appendZoneText(TextStyle)}} method of
> [DateTimeFormatteBuilder|https://github.com/apache/hive/blob/master/common/src/java/org/apache/hadoop/hive/common/type/TimestampTZUtil.java#L82]
> is not able to parse the {{+}} symbol in {{2017-04-14 18:00:00 UTC+08:00}}
> when running [the
> test|https://github.com/apache/hive/blob/master/common/src/test/org/apache/hadoop/hive/common/type/TestTimestampTZ.java#L37]
> in Java 11 , while it is working fine with Java 8.
> According to the
> [doc|https://developer.android.com/reference/java/time/format/DateTimeFormatterBuilder#appendZoneText(java.time.format.TextStyle)]
> , {{appendZoneText(TextStyle)}} should be able to parse {{either the textual
> zone name, the zone ID or the offset}} and UTC+08:00 should come under offset
> as per the same
> [doc|https://developer.android.com/reference/java/time/format/DateTimeFormatterBuilder#appendOffset(java.lang.String,%20java.lang.String)].
> It seems, however, that "UTC" was explicitly removed [when parsed as ZoneText
> for higher Java
> versions|https://github.com/openjdk/jdk/commit/5c3a01591c5c945926636fdc9f164d60b5b4f29e?diff=unified#diff-5fcf976db1c06e8f44a8671356d7e34fdfbf5b057baa852e7c3e015c8797c889R4263]
> As a workaround, we can use {{appendZoneOrOffsetId()}} rather than
> {{appendZoneText().}}
> This ensures the tests are passing for Java 8+ and based on my testing, I
> didn’t see any regression of the change.
> Sample repro code - jdoodle.com/ia/D5e
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27007) Iceberg: Use BasicStats from iceberg table's currrentSnapshot.summary() for query planning
[ https://issues.apache.org/jira/browse/HIVE-27007?focusedWorklogId=842847=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-842847 ] ASF GitHub Bot logged work on HIVE-27007: - Author: ASF GitHub Bot Created on: 01/Feb/23 11:48 Start Date: 01/Feb/23 11:48 Worklog Time Spent: 10m Work Description: sonarcloud[bot] commented on PR #4000: URL: https://github.com/apache/hive/pull/4000#issuecomment-1411928657 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive=4000) [](https://sonarcloud.io/project/issues?id=apache_hive=4000=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=4000=false=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive=4000=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=4000=false=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive=4000=false=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive=4000=false=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4000=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4000=false=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4000=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive=4000=false=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive=4000=false=CODE_SMELL) [2 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive=4000=false=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive=4000=coverage=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive=4000=duplicated_lines_density=list) No Duplication information Issue Time Tracking --- Worklog Id: (was: 842847) Time Spent: 1h 40m (was: 1.5h) > Iceberg: Use BasicStats from iceberg table's currrentSnapshot.summary() for > query planning > -- > > Key: HIVE-27007 > URL: https://issues.apache.org/jira/browse/HIVE-27007 > Project: Hive > Issue Type: Improvement >Reporter: Simhadri Govindappa >Assignee: Simhadri Govindappa >Priority: Major > Labels: pull-request-available > Time Spent: 1h 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26971) Test fix in load_data_using_job.q.out (Done as part of HIVE-20593 addendum patch)
[
https://issues.apache.org/jira/browse/HIVE-26971?focusedWorklogId=842845=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-842845
]
ASF GitHub Bot logged work on HIVE-26971:
-
Author: ASF GitHub Bot
Created on: 01/Feb/23 11:38
Start Date: 01/Feb/23 11:38
Worklog Time Spent: 10m
Work Description: amanraj2520 commented on PR #3973:
URL: https://github.com/apache/hive/pull/3973#issuecomment-1411917470
@zabetak There is no specific JIRA created for this. So I have mentioned the
commit id of master. Thanks for merging it.
Issue Time Tracking
---
Worklog Id: (was: 842845)
Time Spent: 1h 10m (was: 1h)
> Test fix in load_data_using_job.q.out (Done as part of HIVE-20593 addendum
> patch)
> -
>
> Key: HIVE-26971
> URL: https://issues.apache.org/jira/browse/HIVE-26971
> Project: Hive
> Issue Type: Sub-task
>Reporter: Aman Raj
>Assignee: Aman Raj
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> This fix was done is master using this commit id : dca6ef04 (HIVE-20593
> addendum)
>
> There are test failures are as follows :
> h4. Stacktrace
> java.lang.AssertionError:
> Client Execution succeeded but contained differences (error code = 1) after
> executing load_data_using_job.q
> 3031,3032d3030
> < PREHOOK: Input: default@orc_test_txn_{_}temp_table_for_load_data{_}_
> < PREHOOK: Output: default@orc_test_txn
> 3035d3032
> < POSTHOOK: Input: default@orc_test_txn_{_}temp_table_for_load_data{_}_
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-26971) Test fix in load_data_using_job.q.out (Done as part of HIVE-20593 addendum patch)
[
https://issues.apache.org/jira/browse/HIVE-26971?focusedWorklogId=842844=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-842844
]
ASF GitHub Bot logged work on HIVE-26971:
-
Author: ASF GitHub Bot
Created on: 01/Feb/23 11:36
Start Date: 01/Feb/23 11:36
Worklog Time Spent: 10m
Work Description: amanraj2520 commented on PR #3973:
URL: https://github.com/apache/hive/pull/3973#issuecomment-1411915543
Hi @zabetak Thanks for your review. This was done as part of HIVE-20593
addendum patch. Please follow this commit id :
https://github.com/apache/hive/commit/dca6ef04e376ea1e18cd8986c3baf210b51d7677.
Can you please mark this as complete and close this. Apologies for not
mentioning the commit id.
Issue Time Tracking
---
Worklog Id: (was: 842844)
Time Spent: 1h (was: 50m)
> Test fix in load_data_using_job.q.out (Done as part of HIVE-20593 addendum
> patch)
> -
>
> Key: HIVE-26971
> URL: https://issues.apache.org/jira/browse/HIVE-26971
> Project: Hive
> Issue Type: Sub-task
>Reporter: Aman Raj
>Assignee: Aman Raj
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h
> Remaining Estimate: 0h
>
> This fix was done is master using this commit id : dca6ef04 (HIVE-20593
> addendum)
>
> There are test failures are as follows :
> h4. Stacktrace
> java.lang.AssertionError:
> Client Execution succeeded but contained differences (error code = 1) after
> executing load_data_using_job.q
> 3031,3032d3030
> < PREHOOK: Input: default@orc_test_txn_{_}temp_table_for_load_data{_}_
> < PREHOOK: Output: default@orc_test_txn
> 3035d3032
> < POSTHOOK: Input: default@orc_test_txn_{_}temp_table_for_load_data{_}_
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HIVE-26971) Test fix in load_data_using_job.q.out (Done as part of HIVE-20593 addendum patch)
[
https://issues.apache.org/jira/browse/HIVE-26971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aman Raj updated HIVE-26971:
Description:
This fix was done is master using this commit id : dca6ef04 (HIVE-20593
addendum)
There are test failures are as follows :
h4. Stacktrace
java.lang.AssertionError:
Client Execution succeeded but contained differences (error code = 1) after
executing load_data_using_job.q
3031,3032d3030
< PREHOOK: Input: default@orc_test_txn_{_}temp_table_for_load_data{_}_
< PREHOOK: Output: default@orc_test_txn
3035d3032
< POSTHOOK: Input: default@orc_test_txn_{_}temp_table_for_load_data{_}_
was:
This fix was done is master using this commit id : dca6ef04 (
There are test failures are as follows :
h4. Stacktrace
java.lang.AssertionError:
Client Execution succeeded but contained differences (error code = 1) after
executing load_data_using_job.q
3031,3032d3030
< PREHOOK: Input: default@orc_test_txn_{_}temp_table_for_load_data{_}_
< PREHOOK: Output: default@orc_test_txn
3035d3032
< POSTHOOK: Input: default@orc_test_txn_{_}temp_table_for_load_data{_}_
> Test fix in load_data_using_job.q.out (Done as part of HIVE-20593 addendum
> patch)
> -
>
> Key: HIVE-26971
> URL: https://issues.apache.org/jira/browse/HIVE-26971
> Project: Hive
> Issue Type: Sub-task
>Reporter: Aman Raj
>Assignee: Aman Raj
>Priority: Major
> Labels: pull-request-available
> Time Spent: 50m
> Remaining Estimate: 0h
>
> This fix was done is master using this commit id : dca6ef04 (HIVE-20593
> addendum)
>
> There are test failures are as follows :
> h4. Stacktrace
> java.lang.AssertionError:
> Client Execution succeeded but contained differences (error code = 1) after
> executing load_data_using_job.q
> 3031,3032d3030
> < PREHOOK: Input: default@orc_test_txn_{_}temp_table_for_load_data{_}_
> < PREHOOK: Output: default@orc_test_txn
> 3035d3032
> < POSTHOOK: Input: default@orc_test_txn_{_}temp_table_for_load_data{_}_
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HIVE-26971) Test fix in load_data_using_job.q.out (Done as part of HIVE-20593 addendum patch)
[
https://issues.apache.org/jira/browse/HIVE-26971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aman Raj updated HIVE-26971:
Description:
This fix was done is master using this commit id : dca6ef04 (
There are test failures are as follows :
h4. Stacktrace
java.lang.AssertionError:
Client Execution succeeded but contained differences (error code = 1) after
executing load_data_using_job.q
3031,3032d3030
< PREHOOK: Input: default@orc_test_txn_{_}temp_table_for_load_data{_}_
< PREHOOK: Output: default@orc_test_txn
3035d3032
< POSTHOOK: Input: default@orc_test_txn_{_}temp_table_for_load_data{_}_
was:
There are test failures are as follows :
h4. Stacktrace
java.lang.AssertionError:
Client Execution succeeded but contained differences (error code = 1) after
executing load_data_using_job.q
3031,3032d3030
< PREHOOK: Input: default@orc_test_txn_{_}temp_table_for_load_data{_}_
< PREHOOK: Output: default@orc_test_txn
3035d3032
< POSTHOOK: Input: default@orc_test_txn_{_}temp_table_for_load_data{_}_
> Test fix in load_data_using_job.q.out (Done as part of HIVE-20593 addendum
> patch)
> -
>
> Key: HIVE-26971
> URL: https://issues.apache.org/jira/browse/HIVE-26971
> Project: Hive
> Issue Type: Sub-task
>Reporter: Aman Raj
>Assignee: Aman Raj
>Priority: Major
> Labels: pull-request-available
> Time Spent: 50m
> Remaining Estimate: 0h
>
> This fix was done is master using this commit id : dca6ef04 (
>
> There are test failures are as follows :
> h4. Stacktrace
> java.lang.AssertionError:
> Client Execution succeeded but contained differences (error code = 1) after
> executing load_data_using_job.q
> 3031,3032d3030
> < PREHOOK: Input: default@orc_test_txn_{_}temp_table_for_load_data{_}_
> < PREHOOK: Output: default@orc_test_txn
> 3035d3032
> < POSTHOOK: Input: default@orc_test_txn_{_}temp_table_for_load_data{_}_
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HIVE-26971) Test fix in load_data_using_job.q.out (Done as part of HIVE-20593 addendum patch)
[
https://issues.apache.org/jira/browse/HIVE-26971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aman Raj updated HIVE-26971:
Summary: Test fix in load_data_using_job.q.out (Done as part of HIVE-20593
addendum patch) (was: Backport of HIVE-20593)
> Test fix in load_data_using_job.q.out (Done as part of HIVE-20593 addendum
> patch)
> -
>
> Key: HIVE-26971
> URL: https://issues.apache.org/jira/browse/HIVE-26971
> Project: Hive
> Issue Type: Sub-task
>Reporter: Aman Raj
>Assignee: Aman Raj
>Priority: Major
> Labels: pull-request-available
> Time Spent: 50m
> Remaining Estimate: 0h
>
> There are test failures are as follows :
> h4. Stacktrace
> java.lang.AssertionError:
> Client Execution succeeded but contained differences (error code = 1) after
> executing load_data_using_job.q
> 3031,3032d3030
> < PREHOOK: Input: default@orc_test_txn_{_}temp_table_for_load_data{_}_
> < PREHOOK: Output: default@orc_test_txn
> 3035d3032
> < POSTHOOK: Input: default@orc_test_txn_{_}temp_table_for_load_data{_}_
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-26971) Backport of HIVE-20593
[
https://issues.apache.org/jira/browse/HIVE-26971?focusedWorklogId=842835=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-842835
]
ASF GitHub Bot logged work on HIVE-26971:
-
Author: ASF GitHub Bot
Created on: 01/Feb/23 10:56
Start Date: 01/Feb/23 10:56
Worklog Time Spent: 10m
Work Description: zabetak commented on PR #3973:
URL: https://github.com/apache/hive/pull/3973#issuecomment-1411863174
@amanraj2520 I merged the changes but I have some comments.
The PR as well as the JIRA summary are misleading; the changes are NOT a
backport of HIVE-20593. The changes is a simple update of a .q.out file. Please
change the title here and the JIRA summary accordingly (see my commit message).
Moreover, the relation with HIVE-20593 is unclear. HIVE-20593 has been for a
while in branch-3 but the test started failing recently. For instance when we
enabled the tests in
[branch-3](http://ci.hive.apache.org/job/hive-precommit/job/branch-3/1/#showFailuresLink)
the test was not failing. Please be more meticulous and precise in further
investigations.
Issue Time Tracking
---
Worklog Id: (was: 842835)
Time Spent: 50m (was: 40m)
> Backport of HIVE-20593
> --
>
> Key: HIVE-26971
> URL: https://issues.apache.org/jira/browse/HIVE-26971
> Project: Hive
> Issue Type: Sub-task
>Reporter: Aman Raj
>Assignee: Aman Raj
>Priority: Major
> Labels: pull-request-available
> Time Spent: 50m
> Remaining Estimate: 0h
>
> There are test failures are as follows :
> h4. Stacktrace
> java.lang.AssertionError:
> Client Execution succeeded but contained differences (error code = 1) after
> executing load_data_using_job.q
> 3031,3032d3030
> < PREHOOK: Input: default@orc_test_txn_{_}temp_table_for_load_data{_}_
> < PREHOOK: Output: default@orc_test_txn
> 3035d3032
> < POSTHOOK: Input: default@orc_test_txn_{_}temp_table_for_load_data{_}_
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-26986) A DAG created by OperatorGraph is not equal to the Tez DAG.
[ https://issues.apache.org/jira/browse/HIVE-26986?focusedWorklogId=842833=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-842833 ] ASF GitHub Bot logged work on HIVE-26986: - Author: ASF GitHub Bot Created on: 01/Feb/23 10:50 Start Date: 01/Feb/23 10:50 Worklog Time Spent: 10m Work Description: sonarcloud[bot] commented on PR #3998: URL: https://github.com/apache/hive/pull/3998#issuecomment-1411855911 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive=3998) [](https://sonarcloud.io/project/issues?id=apache_hive=3998=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=3998=false=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive=3998=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=3998=false=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive=3998=false=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive=3998=false=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3998=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3998=false=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3998=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive=3998=false=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive=3998=false=CODE_SMELL) [14 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive=3998=false=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive=3998=coverage=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive=3998=duplicated_lines_density=list) No Duplication information Issue Time Tracking --- Worklog Id: (was: 842833) Time Spent: 0.5h (was: 20m) > A DAG created by OperatorGraph is not equal to the Tez DAG. > --- > > Key: HIVE-26986 > URL: https://issues.apache.org/jira/browse/HIVE-26986 > Project: Hive > Issue Type: Bug >Affects Versions: 4.0.0-alpha-2 >Reporter: Seonggon Namgung >Assignee: Seonggon Namgung >Priority: Major > Labels: pull-request-available > Attachments: Query71 OperatorGraph.png, Query71 TezDAG.png > > Time Spent: 0.5h > Remaining Estimate: 0h > > A DAG created by OperatorGraph is not equal to the corresponding DAG that is > submitted to Tez. > Because of this problem, ParallelEdgeFixer reports a pair of normal edges to > a parallel edge. > We observe this problem by comparing OperatorGraph and Tez DAG when running > TPC-DS query 71 on 1TB ORC format managed table. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26166) Make website GDPR compliant
[ https://issues.apache.org/jira/browse/HIVE-26166?focusedWorklogId=842827=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-842827 ] ASF GitHub Bot logged work on HIVE-26166: - Author: ASF GitHub Bot Created on: 01/Feb/23 10:27 Start Date: 01/Feb/23 10:27 Worklog Time Spent: 10m Work Description: zabetak commented on PR #3629: URL: https://github.com/apache/hive/pull/3629#issuecomment-1411826382 The PR is not pertinent for this branch. It is sufficient to get https://github.com/apache/hive-site/pull/1 in the hive-site repo. Issue Time Tracking --- Worklog Id: (was: 842827) Time Spent: 1h (was: 50m) > Make website GDPR compliant > --- > > Key: HIVE-26166 > URL: https://issues.apache.org/jira/browse/HIVE-26166 > Project: Hive > Issue Type: Task > Components: Website >Reporter: Stamatis Zampetakis >Priority: Major > Labels: pull-request-available > Time Spent: 1h > Remaining Estimate: 0h > > Per the email that was sent out from privacy we need to make the Hive website > GDPR compliant. > # The link to privacy policy needs to be updated from > [https://hive.apache.org/privacy_policy.html] to > [https://privacy.apache.org/policies/privacy-policy-public.html] > # The google analytics service must be removed -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26166) Make website GDPR compliant
[ https://issues.apache.org/jira/browse/HIVE-26166?focusedWorklogId=842828=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-842828 ] ASF GitHub Bot logged work on HIVE-26166: - Author: ASF GitHub Bot Created on: 01/Feb/23 10:27 Start Date: 01/Feb/23 10:27 Worklog Time Spent: 10m Work Description: zabetak closed pull request #3629: HIVE-26166 Make website GDPR compliant URL: https://github.com/apache/hive/pull/3629 Issue Time Tracking --- Worklog Id: (was: 842828) Time Spent: 1h 10m (was: 1h) > Make website GDPR compliant > --- > > Key: HIVE-26166 > URL: https://issues.apache.org/jira/browse/HIVE-26166 > Project: Hive > Issue Type: Task > Components: Website >Reporter: Stamatis Zampetakis >Priority: Major > Labels: pull-request-available > Time Spent: 1h 10m > Remaining Estimate: 0h > > Per the email that was sent out from privacy we need to make the Hive website > GDPR compliant. > # The link to privacy policy needs to be updated from > [https://hive.apache.org/privacy_policy.html] to > [https://privacy.apache.org/policies/privacy-policy-public.html] > # The google analytics service must be removed -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26984) Deprecate public HiveConf constructors
[
https://issues.apache.org/jira/browse/HIVE-26984?focusedWorklogId=842826=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-842826
]
ASF GitHub Bot logged work on HIVE-26984:
-
Author: ASF GitHub Bot
Created on: 01/Feb/23 10:23
Start Date: 01/Feb/23 10:23
Worklog Time Spent: 10m
Work Description: zabetak commented on code in PR #3983:
URL: https://github.com/apache/hive/pull/3983#discussion_r1093029187
##
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java:
##
@@ -6322,29 +6322,67 @@ public ZooKeeperHiveHelper getZKConfig() {
.trustStorePassword(trustStorePassword).build();
}
+ public static HiveConf create() {
+return new HiveConf();
+ }
+
+ public static HiveConf create(Class cls) {
+return new HiveConf(cls);
+ }
+
+ public static HiveConf create(Configuration other, Class cls) {
+return new HiveConf(other, cls);
+ }
+
+
+ public static HiveConf create(HiveConf other) {
+return new HiveConf(other);
+ }
+
+ /**
+ * Instantiating HiveConf is deprecated. Please use
+ * HiveConf#create() to construct a Configuration,
+ * this method will become private eventually.
Review Comment:
There shouldn't be many projects creating new Hive configuration instances
so it may be fine doing it here as part of this change.
Consider though if we would like to backport the change in other Hive
branches (Apache or internal); there breaking changes may not be tolerable.
Issue Time Tracking
---
Worklog Id: (was: 842826)
Time Spent: 1.5h (was: 1h 20m)
> Deprecate public HiveConf constructors
> --
>
> Key: HIVE-26984
> URL: https://issues.apache.org/jira/browse/HIVE-26984
> Project: Hive
> Issue Type: Improvement
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> From time to time we investigate configuration object problems that are hard
> to investigate. We can improve this area, e.g. with HIVE-26985, but first, we
> need to introduce a public static factory method to hook into the creation
> process. I can see this pattern in another projects as well, like:
> HBaseConfiguration.
> Creating custom HiveConf subclasses can be useful because putting optional
> (say: if else branches or whatever) stuff into the original HiveConf object's
> hot codepaths can turn it less performant instantly.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-26990) Upgrade Iceberg to 1.1.0
[ https://issues.apache.org/jira/browse/HIVE-26990?focusedWorklogId=842806=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-842806 ] ASF GitHub Bot logged work on HIVE-26990: - Author: ASF GitHub Bot Created on: 01/Feb/23 09:14 Start Date: 01/Feb/23 09:14 Worklog Time Spent: 10m Work Description: sonarcloud[bot] commented on PR #3968: URL: https://github.com/apache/hive/pull/3968#issuecomment-1411714071 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive=3968) [](https://sonarcloud.io/project/issues?id=apache_hive=3968=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=3968=false=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive=3968=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=3968=false=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive=3968=false=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive=3968=false=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3968=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3968=false=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3968=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive=3968=false=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive=3968=false=CODE_SMELL) [2 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive=3968=false=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive=3968=coverage=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive=3968=duplicated_lines_density=list) No Duplication information Issue Time Tracking --- Worklog Id: (was: 842806) Time Spent: 0.5h (was: 20m) > Upgrade Iceberg to 1.1.0 > > > Key: HIVE-26990 > URL: https://issues.apache.org/jira/browse/HIVE-26990 > Project: Hive > Issue Type: Improvement > Components: Iceberg integration >Reporter: Zsolt Miskolczi >Assignee: Zsolt Miskolczi >Priority: Major > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > > Iceberg integration currently uses Iceberg 1.0.0 > Upgrade it to 1.1.0 to be able to utilise new features. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26996) typos in hive-exec
[ https://issues.apache.org/jira/browse/HIVE-26996?focusedWorklogId=842802=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-842802 ] ASF GitHub Bot logged work on HIVE-26996: - Author: ASF GitHub Bot Created on: 01/Feb/23 09:12 Start Date: 01/Feb/23 09:12 Worklog Time Spent: 10m Work Description: sonarcloud[bot] commented on PR #3994: URL: https://github.com/apache/hive/pull/3994#issuecomment-1411710995 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive=3994) [](https://sonarcloud.io/project/issues?id=apache_hive=3994=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=3994=false=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive=3994=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=3994=false=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive=3994=false=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive=3994=false=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3994=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3994=false=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3994=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive=3994=false=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive=3994=false=CODE_SMELL) [4 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive=3994=false=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive=3994=coverage=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive=3994=duplicated_lines_density=list) No Duplication information Issue Time Tracking --- Worklog Id: (was: 842802) Time Spent: 50m (was: 40m) > typos in hive-exec > -- > > Key: HIVE-26996 > URL: https://issues.apache.org/jira/browse/HIVE-26996 > Project: Hive > Issue Type: Improvement > Components: Query Planning, Query Processor >Affects Versions: All Versions >Reporter: Michal Lorek >Assignee: Michal Lorek >Priority: Minor > Labels: pull-request-available > Fix For: 4.0.0 > > Time Spent: 50m > Remaining Estimate: 0h > > typos and grammar errors in hive-exec module -- This message was sent by Atlassian Jira (v8.20.10#820010)
