[jira] [Updated] (HIVE-25577) unix_timestamp() is ignoring the time zone value
[
https://issues.apache.org/jira/browse/HIVE-25577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sankar Hariappan updated HIVE-25577:
Affects Version/s: 4.0.0
3.1.2
> unix_timestamp() is ignoring the time zone value
>
>
> Key: HIVE-25577
> URL: https://issues.apache.org/jira/browse/HIVE-25577
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.2, 4.0.0
>Reporter: Ashish Sharma
>Assignee: Ashish Sharma
>Priority: Minor
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> set hive.local.time.zone=Asia/Bangkok;
> Query - SELECT FROM_UNIXTIME(UNIX_TIMESTAMP('2000-01-07 00:00:00
> GMT','-MM-dd HH:mm:ss z'));
> Result - 2000-01-07 00:00:00 ICT
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-25577) unix_timestamp() is ignoring the time zone value
[
https://issues.apache.org/jira/browse/HIVE-25577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sankar Hariappan updated HIVE-25577:
Component/s: UDF
> unix_timestamp() is ignoring the time zone value
>
>
> Key: HIVE-25577
> URL: https://issues.apache.org/jira/browse/HIVE-25577
> Project: Hive
> Issue Type: Bug
> Components: UDF
>Affects Versions: 3.1.2, 4.0.0
>Reporter: Ashish Sharma
>Assignee: Ashish Sharma
>Priority: Minor
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> set hive.local.time.zone=Asia/Bangkok;
> Query - SELECT FROM_UNIXTIME(UNIX_TIMESTAMP('2000-01-07 00:00:00
> GMT','-MM-dd HH:mm:ss z'));
> Result - 2000-01-07 00:00:00 ICT
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HIVE-25577) unix_timestamp() is ignoring the time zone value
[
https://issues.apache.org/jira/browse/HIVE-25577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sankar Hariappan resolved HIVE-25577.
-
Fix Version/s: 4.0.0
Target Version/s: 4.0.0
Resolution: Fixed
PR merged to master.
Thanks [~ashish-kumar-sharma] for the patch!
> unix_timestamp() is ignoring the time zone value
>
>
> Key: HIVE-25577
> URL: https://issues.apache.org/jira/browse/HIVE-25577
> Project: Hive
> Issue Type: Bug
>Reporter: Ashish Sharma
>Assignee: Ashish Sharma
>Priority: Minor
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> set hive.local.time.zone=Asia/Bangkok;
> Query - SELECT FROM_UNIXTIME(UNIX_TIMESTAMP('2000-01-07 00:00:00
> GMT','-MM-dd HH:mm:ss z'));
> Result - 2000-01-07 00:00:00 ICT
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25577) unix_timestamp() is ignoring the time zone value
[
https://issues.apache.org/jira/browse/HIVE-25577?focusedWorklogId=660100=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-660100
]
ASF GitHub Bot logged work on HIVE-25577:
-
Author: ASF GitHub Bot
Created on: 05/Oct/21 05:09
Start Date: 05/Oct/21 05:09
Worklog Time Spent: 10m
Work Description: sankarh merged pull request #2686:
URL: https://github.com/apache/hive/pull/2686
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 660100)
Time Spent: 1h (was: 50m)
> unix_timestamp() is ignoring the time zone value
>
>
> Key: HIVE-25577
> URL: https://issues.apache.org/jira/browse/HIVE-25577
> Project: Hive
> Issue Type: Bug
>Reporter: Ashish Sharma
>Assignee: Ashish Sharma
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 1h
> Remaining Estimate: 0h
>
> set hive.local.time.zone=Asia/Bangkok;
> Query - SELECT FROM_UNIXTIME(UNIX_TIMESTAMP('2000-01-07 00:00:00
> GMT','-MM-dd HH:mm:ss z'));
> Result - 2000-01-07 00:00:00 ICT
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25576) Add config to parse date with older date format
[
https://issues.apache.org/jira/browse/HIVE-25576?focusedWorklogId=660079=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-660079
]
ASF GitHub Bot logged work on HIVE-25576:
-
Author: ASF GitHub Bot
Created on: 05/Oct/21 03:59
Start Date: 05/Oct/21 03:59
Worklog Time Spent: 10m
Work Description: ashish-kumar-sharma commented on pull request #2690:
URL: https://github.com/apache/hive/pull/2690#issuecomment-934037741
@zabetak @sankarh Could guys please review the PR?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 660079)
Time Spent: 20m (was: 10m)
> Add config to parse date with older date format
> ---
>
> Key: HIVE-25576
> URL: https://issues.apache.org/jira/browse/HIVE-25576
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 4.0.0
>Reporter: Ashish Sharma
>Assignee: Ashish Sharma
>Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> *History*
> *Hive 1.2* -
> VM time zone set to Asia/Bangkok
> *Query* - SELECT FROM_UNIXTIME(UNIX_TIMESTAMP('1800-01-01 00:00:00
> UTC','-MM-dd HH:mm:ss z'));
> *Result* - 1800-01-01 07:00:00
> *Implementation details* -
> SimpleDateFormat formatter = new SimpleDateFormat(pattern);
> Long unixtime = formatter.parse(textval).getTime() / 1000;
> Date date = new Date(unixtime * 1000L);
> https://docs.oracle.com/javase/8/docs/api/java/util/Date.html . In official
> documentation they have mention that "Unfortunately, the API for these
> functions was not amenable to internationalization and The corresponding
> methods in Date are deprecated" . Due to that this is producing wrong result
> *Master branch* -
> set hive.local.time.zone=Asia/Bangkok;
> *Query* - SELECT FROM_UNIXTIME(UNIX_TIMESTAMP('1800-01-01 00:00:00
> UTC','-MM-dd HH:mm:ss z'));
> *Result* - 1800-01-01 06:42:04
> *Implementation details* -
> DateTimeFormatter dtformatter = new DateTimeFormatterBuilder()
> .parseCaseInsensitive()
> .appendPattern(pattern)
> .toFormatter();
> ZonedDateTime zonedDateTime =
> ZonedDateTime.parse(textval,dtformatter).withZoneSameInstant(ZoneId.of(timezone));
> Long dttime = zonedDateTime.toInstant().getEpochSecond();
> *Problem*-
> Now *SimpleDateFormat* has been replaced with *DateTimeFormatter* which is
> giving the correct result but it is not backword compatible. Which is causing
> issue at time for migration to new version. Because the older data written is
> using Hive 1.x or 2.x is not compatible with *DateTimeFormatter*.
> *Solution*
> Introduce an config "hive.legacy.timeParserPolicy" with following values -
> 1. *True*- use *SimpleDateFormat*
> 2. *False* - use *DateTimeFormatter*
> Note: apache spark also face the same issue
> https://issues.apache.org/jira/browse/SPARK-30668
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-25546) Enable incremental rebuild of Materialized views with insert only source tables
[
https://issues.apache.org/jira/browse/HIVE-25546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-25546:
--
Resolution: Fixed
Status: Resolved (was: Patch Available)
Pushed to master. Thanks [~zabetak], [~asolimando] for review.
> Enable incremental rebuild of Materialized views with insert only source
> tables
> ---
>
> Key: HIVE-25546
> URL: https://issues.apache.org/jira/browse/HIVE-25546
> Project: Hive
> Issue Type: Improvement
> Components: CBO, Materialized views
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Labels: pull-request-available
> Time Spent: 3h 10m
> Remaining Estimate: 0h
>
> {code}
> create table t1(a int, b int, c int) stored as parquet TBLPROPERTIES
> ('transactional'='true', 'transactional_properties'='insert_only');
> create materialized view mat1 stored as orc TBLPROPERTIES
> ('transactional'='true') as
> select a, b, c from t1 where a > 10;
> {code}
> Currently materialized view *mat1* can not be rebuilt incrementally because
> it has an insert only source table (t1). Such tables does not have
> ROW_ID.write_id which is required to identify newly inserted records since
> the last rebuild.
> HIVE-25406 adds the ability to query write_id.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25546) Enable incremental rebuild of Materialized views with insert only source tables
[
https://issues.apache.org/jira/browse/HIVE-25546?focusedWorklogId=660074=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-660074
]
ASF GitHub Bot logged work on HIVE-25546:
-
Author: ASF GitHub Bot
Created on: 05/Oct/21 03:28
Start Date: 05/Oct/21 03:28
Worklog Time Spent: 10m
Work Description: kasakrisz merged pull request #2663:
URL: https://github.com/apache/hive/pull/2663
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 660074)
Time Spent: 3h 10m (was: 3h)
> Enable incremental rebuild of Materialized views with insert only source
> tables
> ---
>
> Key: HIVE-25546
> URL: https://issues.apache.org/jira/browse/HIVE-25546
> Project: Hive
> Issue Type: Improvement
> Components: CBO, Materialized views
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Labels: pull-request-available
> Time Spent: 3h 10m
> Remaining Estimate: 0h
>
> {code}
> create table t1(a int, b int, c int) stored as parquet TBLPROPERTIES
> ('transactional'='true', 'transactional_properties'='insert_only');
> create materialized view mat1 stored as orc TBLPROPERTIES
> ('transactional'='true') as
> select a, b, c from t1 where a > 10;
> {code}
> Currently materialized view *mat1* can not be rebuilt incrementally because
> it has an insert only source table (t1). Such tables does not have
> ROW_ID.write_id which is required to identify newly inserted records since
> the last rebuild.
> HIVE-25406 adds the ability to query write_id.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HIVE-25572) Exception while querying materialized view invalidation info
[
https://issues.apache.org/jira/browse/HIVE-25572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa resolved HIVE-25572.
---
Resolution: Fixed
Pushed to master. Thanks [~pvary] for review.
> Exception while querying materialized view invalidation info
>
>
> Key: HIVE-25572
> URL: https://issues.apache.org/jira/browse/HIVE-25572
> Project: Hive
> Issue Type: Bug
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> {code:java}
> 2021-09-29T00:33:02,971 WARN [main] txn.TxnHandler: Unable to retrieve
> materialization invalidation information: completed transaction components.
> java.sql.SQLSyntaxErrorException: Syntax error: Encountered "" at line
> 1, column 234.
> at
> org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown
> Source) ~[derby-10.14.1.0.jar:?]
> at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown
> Source) ~[derby-10.14.1.0.jar:?]
> at
> org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown
> Source) ~[derby-10.14.1.0.jar:?]
> at
> org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown
> Source) ~[derby-10.14.1.0.jar:?]
> at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown
> Source) ~[derby-10.14.1.0.jar:?]
> at org.apache.derby.impl.jdbc.ConnectionChild.handleException(Unknown
> Source) ~[derby-10.14.1.0.jar:?]
> at org.apache.derby.impl.jdbc.EmbedPreparedStatement.(Unknown
> Source) ~[derby-10.14.1.0.jar:?]
> at org.apache.derby.impl.jdbc.EmbedPreparedStatement42.(Unknown
> Source) ~[derby-10.14.1.0.jar:?]
> at org.apache.derby.jdbc.Driver42.newEmbedPreparedStatement(Unknown
> Source) ~[derby-10.14.1.0.jar:?]
> at org.apache.derby.impl.jdbc.EmbedConnection.prepareStatement(Unknown
> Source) ~[derby-10.14.1.0.jar:?]
> at org.apache.derby.impl.jdbc.EmbedConnection.prepareStatement(Unknown
> Source) ~[derby-10.14.1.0.jar:?]
> at
> com.zaxxer.hikari.pool.ProxyConnection.prepareStatement(ProxyConnection.java:311)
> ~[HikariCP-2.6.1.jar:?]
> at
> com.zaxxer.hikari.pool.HikariProxyConnection.prepareStatement(HikariProxyConnection.java)
> ~[HikariCP-2.6.1.jar:?]
> at
> org.apache.hadoop.hive.metastore.tools.SQLGenerator.prepareStmtWithParameters(SQLGenerator.java:169)
> ~[classes/:?]
> at
> org.apache.hadoop.hive.metastore.txn.TxnHandler.executeBoolean(TxnHandler.java:2598)
> [classes/:?]
> at
> org.apache.hadoop.hive.metastore.txn.TxnHandler.getMaterializationInvalidationInfo(TxnHandler.java:2575)
> [classes/:?]
> at
> org.apache.hadoop.hive.metastore.txn.TestTxnHandler.testGetMaterializationInvalidationInfo(TestTxnHandler.java:1910)
> [test-classes/:?]
> at
> org.apache.hadoop.hive.metastore.txn.TestTxnHandler.testGetMaterializationInvalidationInfo(TestTxnHandler.java:1875)
> [test-classes/:?]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ~[?:1.8.0_112]
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> ~[?:1.8.0_112]
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[?:1.8.0_112]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_112]
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> [junit-4.13.jar:4.13]
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> [junit-4.13.jar:4.13]
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> [junit-4.13.jar:4.13]
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> [junit-4.13.jar:4.13]
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> [junit-4.13.jar:4.13]
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> [junit-4.13.jar:4.13]
> at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
> [junit-4.13.jar:4.13]
> at
> org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
> [junit-4.13.jar:4.13]
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
> [junit-4.13.jar:4.13]
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
> [junit-4.13.jar:4.13]
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
> [junit-4.13.jar:4.13]
> at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
>
[jira] [Work logged] (HIVE-25572) Exception while querying materialized view invalidation info
[
https://issues.apache.org/jira/browse/HIVE-25572?focusedWorklogId=660072=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-660072
]
ASF GitHub Bot logged work on HIVE-25572:
-
Author: ASF GitHub Bot
Created on: 05/Oct/21 03:25
Start Date: 05/Oct/21 03:25
Worklog Time Spent: 10m
Work Description: kasakrisz merged pull request #2682:
URL: https://github.com/apache/hive/pull/2682
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 660072)
Time Spent: 1h 40m (was: 1.5h)
> Exception while querying materialized view invalidation info
>
>
> Key: HIVE-25572
> URL: https://issues.apache.org/jira/browse/HIVE-25572
> Project: Hive
> Issue Type: Bug
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> {code:java}
> 2021-09-29T00:33:02,971 WARN [main] txn.TxnHandler: Unable to retrieve
> materialization invalidation information: completed transaction components.
> java.sql.SQLSyntaxErrorException: Syntax error: Encountered "" at line
> 1, column 234.
> at
> org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown
> Source) ~[derby-10.14.1.0.jar:?]
> at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown
> Source) ~[derby-10.14.1.0.jar:?]
> at
> org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown
> Source) ~[derby-10.14.1.0.jar:?]
> at
> org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown
> Source) ~[derby-10.14.1.0.jar:?]
> at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown
> Source) ~[derby-10.14.1.0.jar:?]
> at org.apache.derby.impl.jdbc.ConnectionChild.handleException(Unknown
> Source) ~[derby-10.14.1.0.jar:?]
> at org.apache.derby.impl.jdbc.EmbedPreparedStatement.(Unknown
> Source) ~[derby-10.14.1.0.jar:?]
> at org.apache.derby.impl.jdbc.EmbedPreparedStatement42.(Unknown
> Source) ~[derby-10.14.1.0.jar:?]
> at org.apache.derby.jdbc.Driver42.newEmbedPreparedStatement(Unknown
> Source) ~[derby-10.14.1.0.jar:?]
> at org.apache.derby.impl.jdbc.EmbedConnection.prepareStatement(Unknown
> Source) ~[derby-10.14.1.0.jar:?]
> at org.apache.derby.impl.jdbc.EmbedConnection.prepareStatement(Unknown
> Source) ~[derby-10.14.1.0.jar:?]
> at
> com.zaxxer.hikari.pool.ProxyConnection.prepareStatement(ProxyConnection.java:311)
> ~[HikariCP-2.6.1.jar:?]
> at
> com.zaxxer.hikari.pool.HikariProxyConnection.prepareStatement(HikariProxyConnection.java)
> ~[HikariCP-2.6.1.jar:?]
> at
> org.apache.hadoop.hive.metastore.tools.SQLGenerator.prepareStmtWithParameters(SQLGenerator.java:169)
> ~[classes/:?]
> at
> org.apache.hadoop.hive.metastore.txn.TxnHandler.executeBoolean(TxnHandler.java:2598)
> [classes/:?]
> at
> org.apache.hadoop.hive.metastore.txn.TxnHandler.getMaterializationInvalidationInfo(TxnHandler.java:2575)
> [classes/:?]
> at
> org.apache.hadoop.hive.metastore.txn.TestTxnHandler.testGetMaterializationInvalidationInfo(TestTxnHandler.java:1910)
> [test-classes/:?]
> at
> org.apache.hadoop.hive.metastore.txn.TestTxnHandler.testGetMaterializationInvalidationInfo(TestTxnHandler.java:1875)
> [test-classes/:?]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ~[?:1.8.0_112]
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> ~[?:1.8.0_112]
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[?:1.8.0_112]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_112]
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> [junit-4.13.jar:4.13]
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> [junit-4.13.jar:4.13]
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> [junit-4.13.jar:4.13]
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> [junit-4.13.jar:4.13]
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> [junit-4.13.jar:4.13]
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
[jira] [Work logged] (HIVE-23633) Metastore some JDO query objects do not close properly
[
https://issues.apache.org/jira/browse/HIVE-23633?focusedWorklogId=660004=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-660004
]
ASF GitHub Bot logged work on HIVE-23633:
-
Author: ASF GitHub Bot
Created on: 05/Oct/21 00:30
Start Date: 05/Oct/21 00:30
Worklog Time Spent: 10m
Work Description: dengzhhu653 commented on a change in pull request #2344:
URL: https://github.com/apache/hive/pull/2344#discussion_r721803993
##
File path:
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java
##
@@ -368,20 +363,22 @@ private void executeNoResult(final String queryText)
throws SQLException {
}
public Database getDatabase(String catName, String dbName) throws
MetaException{
-Query queryDbSelector = null;
-Query queryDbParams = null;
-try {
+String queryTextDbSelector= "select "
++ "\"DB_ID\", \"NAME\", \"DB_LOCATION_URI\", \"DESC\", "
++ "\"OWNER_NAME\", \"OWNER_TYPE\", \"CTLG_NAME\" , \"CREATE_TIME\",
\"DB_MANAGED_LOCATION_URI\", "
++ "\"TYPE\", \"DATACONNECTOR_NAME\", \"REMOTE_DBNAME\""
++ "FROM "+ DBS
++ " where \"NAME\" = ? and \"CTLG_NAME\" = ? ";
+String queryTextDbParams = "select \"PARAM_KEY\", \"PARAM_VALUE\" "
++ " from " + DATABASE_PARAMS + " "
++ " WHERE \"DB_ID\" = ? "
++ " AND \"PARAM_KEY\" IS NOT NULL";
Review comment:
Thank you for the review, @belugabehr! My idea is to make the codes be
clean and the same pattern by replacing the standalone `close` to
try-with-resource.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 660004)
Time Spent: 15h 40m (was: 15.5h)
> Metastore some JDO query objects do not close properly
> --
>
> Key: HIVE-23633
> URL: https://issues.apache.org/jira/browse/HIVE-23633
> Project: Hive
> Issue Type: Bug
>Reporter: Zhihua Deng
>Assignee: Zhihua Deng
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-23633.01.patch
>
> Time Spent: 15h 40m
> Remaining Estimate: 0h
>
> After patched [HIVE-10895|https://issues.apache.org/jira/browse/HIVE-10895],
> The metastore still has seen a memory leak on db resources: many
> StatementImpls left unclosed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-23633) Metastore some JDO query objects do not close properly
[ https://issues.apache.org/jira/browse/HIVE-23633?focusedWorklogId=659711=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-659711 ] ASF GitHub Bot logged work on HIVE-23633: - Author: ASF GitHub Bot Created on: 04/Oct/21 16:49 Start Date: 04/Oct/21 16:49 Worklog Time Spent: 10m Work Description: pvary commented on pull request #2344: URL: https://github.com/apache/hive/pull/2344#issuecomment-933667212 > I'm a little confused here. What version of JDO is Hive using? > > It looks to me like `Query` should already be `Autocloseable` > > https://github.com/datanucleus/javax.jdo/blob/fa061bf68e347b8fafca369ab589b7625d8579f3/src/main/java/javax/jdo/Query.java#L68 > > Thanks. Hehe. I have run the exact same rounds with @dengzhhu653. `Autocloseable`close throws an `Exception`. We do not want to handle this individually, so we need the wrapper -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 659711) Time Spent: 15.5h (was: 15h 20m) > Metastore some JDO query objects do not close properly > -- > > Key: HIVE-23633 > URL: https://issues.apache.org/jira/browse/HIVE-23633 > Project: Hive > Issue Type: Bug >Reporter: Zhihua Deng >Assignee: Zhihua Deng >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-23633.01.patch > > Time Spent: 15.5h > Remaining Estimate: 0h > > After patched [HIVE-10895|https://issues.apache.org/jira/browse/HIVE-10895], > The metastore still has seen a memory leak on db resources: many > StatementImpls left unclosed. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-18321) Support REBUILD for MVs backed by custom storage handlers
[ https://issues.apache.org/jira/browse/HIVE-18321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17424026#comment-17424026 ] Vipin commented on HIVE-18321: -- [~ashutoshc] [~jcamachorodriguez] https://issues.apache.org/jira/browse/HIVE-25565 Is this issue related to this? I am having this issue and any pointer to this will be great help please!!! > Support REBUILD for MVs backed by custom storage handlers > - > > Key: HIVE-18321 > URL: https://issues.apache.org/jira/browse/HIVE-18321 > Project: Hive > Issue Type: Bug > Components: Druid integration, Materialized views >Affects Versions: 3.0.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Fix For: 3.0.0 > > > For instance, currently REBUILD will fail for Druid backed MVs. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25565) Materialized view Rebuild issue Aws EMR
[
https://issues.apache.org/jira/browse/HIVE-25565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vipin updated HIVE-25565:
-
Affects Version/s: 3.1.2
> Materialized view Rebuild issue Aws EMR
> ---
>
> Key: HIVE-25565
> URL: https://issues.apache.org/jira/browse/HIVE-25565
> Project: Hive
> Issue Type: Bug
> Components: Materialized views
>Affects Versions: 3.1.2
> Environment: Aws EMR - 6.3.0
> Spark - 3.1.1
> Hive - 3.1.2
>Reporter: Vipin
>Priority: Major
>
> We have Materialized views built on top of Hudi tables which are hive-sync'd.
> Hive uses AWS Glue for its metastore catalog.
> We are running into issue whenever we are trying to "**rebuild**" Hive
> materialized views.
> Please note, creation of materialized views works fine. It's only rebuild
> which is failing.
> However, it does seem the rebuild actually seems to work behind the scenes
> but its throws some exception causing EMR steps to fail.
> Can anyone please guide us here, about any config changes that we need to do
> or anything. Any help will be great.
>
> The stack trace of the exception -
> {quote} FAILED: Hive Internal Error:
> org.apache.hadoop.hive.ql.metadata.HiveException(Error while invoking
> FailureHook. hooks: java.lang.NullPointerException at
> org.apache.hadoop.hive.ql.reexec.ReExecutionOverlayPlugin$LocalHook.run(ReExecutionOverlayPlugin.java:45)
> at
> org.apache.hadoop.hive.ql.HookRunner.invokeGeneralHook(HookRunner.java:296)
> at org.apache.hadoop.hive.ql.HookRunner.runFailureHooks(HookRunner.java:283)
> at org.apache.hadoop.hive.ql.Driver.invokeFailureHooks(Driver.java:2616) at
> org.apache.hadoop.hive.ql.Driver.execute(Driver.java:2386) at
> org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:2011) at
> org.apache.hadoop.hive.ql.Driver.run(Driver.java:1709) at
> org.apache.hadoop.hive.ql.Driver.run(Driver.java:1703) at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:157) at
> org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:224)
> at
> org.apache.hive.service.cli.operation.SQLOperation.access$700(SQLOperation.java:87)
> at
> org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork$1.run(SQLOperation.java:316)
> at java.security.AccessController.doPrivileged(Native Method) at
> javax.security.auth.Subject.doAs(Subject.java:422) at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
> at
> org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork.run(SQLOperation.java:330)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)) >
> org.apache.hadoop.hive.ql.metadata.HiveException: Error while invoking
> FailureHook. hooks: > java.lang.NullPointerException > at
> org.apache.hadoop.hive.ql.reexec.ReExecutionOverlayPlugin$LocalHook.run(ReExecutionOverlayPlugin.java:45)>
> at
> org.apache.hadoop.hive.ql.HookRunner.invokeGeneralHook(HookRunner.java:296)>
> at org.apache.hadoop.hive.ql.HookRunner.runFailureHooks(HookRunner.java:283)>
> at org.apache.hadoop.hive.ql.Driver.invokeFailureHooks(Driver.java:2616)> at
> org.apache.hadoop.hive.ql.Driver.execute(Driver.java:2386)> at
> org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:2011)> at
> org.apache.hadoop.hive.ql.Driver.run(Driver.java:1709)> at
> org.apache.hadoop.hive.ql.Driver.run(Driver.java:1703)> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:157)> at
> org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:224)>
> at
> org.apache.hive.service.cli.operation.SQLOperation.access$700(SQLOperation.java:87)>
> at
> org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork$1.run(SQLOperation.java:316)>
> at java.security.AccessController.doPrivileged(Native Method)> at
> javax.security.auth.Subject.doAs(Subject.java:422)> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)>
> at
> org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork.run(SQLOperation.java:330)>
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)>
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)>
> at java.lang.Thread.run(Thread.java:748)> > at
> org.apache.hadoop.hive.ql.HookRunner.invokeGeneralHook(HookRunner.java:302)>
> at
[jira] [Updated] (HIVE-25565) Materialized view Rebuild of Non-Txn table issue Aws EMR
[
https://issues.apache.org/jira/browse/HIVE-25565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vipin updated HIVE-25565:
-
Summary: Materialized view Rebuild of Non-Txn table issue Aws EMR (was:
Materialized view Rebuild issue Aws EMR)
> Materialized view Rebuild of Non-Txn table issue Aws EMR
>
>
> Key: HIVE-25565
> URL: https://issues.apache.org/jira/browse/HIVE-25565
> Project: Hive
> Issue Type: Bug
> Components: Materialized views
>Affects Versions: 3.1.2
> Environment: Aws EMR - 6.3.0
> Spark - 3.1.1
> Hive - 3.1.2
>Reporter: Vipin
>Priority: Major
>
> We have Materialized views built on top of Hudi tables which are hive-sync'd.
> Hive uses AWS Glue for its metastore catalog.
> We are running into issue whenever we are trying to "**rebuild**" Hive
> materialized views.
> Please note, creation of materialized views works fine. It's only rebuild
> which is failing.
> However, it does seem the rebuild actually seems to work behind the scenes
> but its throws some exception causing EMR steps to fail.
> Can anyone please guide us here, about any config changes that we need to do
> or anything. Any help will be great.
>
> The stack trace of the exception -
> {quote} FAILED: Hive Internal Error:
> org.apache.hadoop.hive.ql.metadata.HiveException(Error while invoking
> FailureHook. hooks: java.lang.NullPointerException at
> org.apache.hadoop.hive.ql.reexec.ReExecutionOverlayPlugin$LocalHook.run(ReExecutionOverlayPlugin.java:45)
> at
> org.apache.hadoop.hive.ql.HookRunner.invokeGeneralHook(HookRunner.java:296)
> at org.apache.hadoop.hive.ql.HookRunner.runFailureHooks(HookRunner.java:283)
> at org.apache.hadoop.hive.ql.Driver.invokeFailureHooks(Driver.java:2616) at
> org.apache.hadoop.hive.ql.Driver.execute(Driver.java:2386) at
> org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:2011) at
> org.apache.hadoop.hive.ql.Driver.run(Driver.java:1709) at
> org.apache.hadoop.hive.ql.Driver.run(Driver.java:1703) at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:157) at
> org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:224)
> at
> org.apache.hive.service.cli.operation.SQLOperation.access$700(SQLOperation.java:87)
> at
> org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork$1.run(SQLOperation.java:316)
> at java.security.AccessController.doPrivileged(Native Method) at
> javax.security.auth.Subject.doAs(Subject.java:422) at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
> at
> org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork.run(SQLOperation.java:330)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)) >
> org.apache.hadoop.hive.ql.metadata.HiveException: Error while invoking
> FailureHook. hooks: > java.lang.NullPointerException > at
> org.apache.hadoop.hive.ql.reexec.ReExecutionOverlayPlugin$LocalHook.run(ReExecutionOverlayPlugin.java:45)>
> at
> org.apache.hadoop.hive.ql.HookRunner.invokeGeneralHook(HookRunner.java:296)>
> at org.apache.hadoop.hive.ql.HookRunner.runFailureHooks(HookRunner.java:283)>
> at org.apache.hadoop.hive.ql.Driver.invokeFailureHooks(Driver.java:2616)> at
> org.apache.hadoop.hive.ql.Driver.execute(Driver.java:2386)> at
> org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:2011)> at
> org.apache.hadoop.hive.ql.Driver.run(Driver.java:1709)> at
> org.apache.hadoop.hive.ql.Driver.run(Driver.java:1703)> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:157)> at
> org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:224)>
> at
> org.apache.hive.service.cli.operation.SQLOperation.access$700(SQLOperation.java:87)>
> at
> org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork$1.run(SQLOperation.java:316)>
> at java.security.AccessController.doPrivileged(Native Method)> at
> javax.security.auth.Subject.doAs(Subject.java:422)> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)>
> at
> org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork.run(SQLOperation.java:330)>
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)>
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)>
> at
[jira] [Updated] (HIVE-25565) Materialized view Rebuild issue Aws EMR
[
https://issues.apache.org/jira/browse/HIVE-25565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vipin updated HIVE-25565:
-
Component/s: Materialized views
> Materialized view Rebuild issue Aws EMR
> ---
>
> Key: HIVE-25565
> URL: https://issues.apache.org/jira/browse/HIVE-25565
> Project: Hive
> Issue Type: Bug
> Components: Materialized views
> Environment: Aws EMR - 6.3.0
> Spark - 3.1.1
> Hive - 3.1.2
>Reporter: Vipin
>Priority: Major
>
> We have Materialized views built on top of Hudi tables which are hive-sync'd.
> Hive uses AWS Glue for its metastore catalog.
> We are running into issue whenever we are trying to "**rebuild**" Hive
> materialized views.
> Please note, creation of materialized views works fine. It's only rebuild
> which is failing.
> However, it does seem the rebuild actually seems to work behind the scenes
> but its throws some exception causing EMR steps to fail.
> Can anyone please guide us here, about any config changes that we need to do
> or anything. Any help will be great.
>
> The stack trace of the exception -
> {quote} FAILED: Hive Internal Error:
> org.apache.hadoop.hive.ql.metadata.HiveException(Error while invoking
> FailureHook. hooks: java.lang.NullPointerException at
> org.apache.hadoop.hive.ql.reexec.ReExecutionOverlayPlugin$LocalHook.run(ReExecutionOverlayPlugin.java:45)
> at
> org.apache.hadoop.hive.ql.HookRunner.invokeGeneralHook(HookRunner.java:296)
> at org.apache.hadoop.hive.ql.HookRunner.runFailureHooks(HookRunner.java:283)
> at org.apache.hadoop.hive.ql.Driver.invokeFailureHooks(Driver.java:2616) at
> org.apache.hadoop.hive.ql.Driver.execute(Driver.java:2386) at
> org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:2011) at
> org.apache.hadoop.hive.ql.Driver.run(Driver.java:1709) at
> org.apache.hadoop.hive.ql.Driver.run(Driver.java:1703) at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:157) at
> org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:224)
> at
> org.apache.hive.service.cli.operation.SQLOperation.access$700(SQLOperation.java:87)
> at
> org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork$1.run(SQLOperation.java:316)
> at java.security.AccessController.doPrivileged(Native Method) at
> javax.security.auth.Subject.doAs(Subject.java:422) at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
> at
> org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork.run(SQLOperation.java:330)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)) >
> org.apache.hadoop.hive.ql.metadata.HiveException: Error while invoking
> FailureHook. hooks: > java.lang.NullPointerException > at
> org.apache.hadoop.hive.ql.reexec.ReExecutionOverlayPlugin$LocalHook.run(ReExecutionOverlayPlugin.java:45)>
> at
> org.apache.hadoop.hive.ql.HookRunner.invokeGeneralHook(HookRunner.java:296)>
> at org.apache.hadoop.hive.ql.HookRunner.runFailureHooks(HookRunner.java:283)>
> at org.apache.hadoop.hive.ql.Driver.invokeFailureHooks(Driver.java:2616)> at
> org.apache.hadoop.hive.ql.Driver.execute(Driver.java:2386)> at
> org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:2011)> at
> org.apache.hadoop.hive.ql.Driver.run(Driver.java:1709)> at
> org.apache.hadoop.hive.ql.Driver.run(Driver.java:1703)> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:157)> at
> org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:224)>
> at
> org.apache.hive.service.cli.operation.SQLOperation.access$700(SQLOperation.java:87)>
> at
> org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork$1.run(SQLOperation.java:316)>
> at java.security.AccessController.doPrivileged(Native Method)> at
> javax.security.auth.Subject.doAs(Subject.java:422)> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)>
> at
> org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork.run(SQLOperation.java:330)>
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)>
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)>
> at java.lang.Thread.run(Thread.java:748)> > at
> org.apache.hadoop.hive.ql.HookRunner.invokeGeneralHook(HookRunner.java:302)>
> at
[jira] [Work logged] (HIVE-25522) NullPointerException in TxnHandler
[
https://issues.apache.org/jira/browse/HIVE-25522?focusedWorklogId=659693=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-659693
]
ASF GitHub Bot logged work on HIVE-25522:
-
Author: ASF GitHub Bot
Created on: 04/Oct/21 16:32
Start Date: 04/Oct/21 16:32
Worklog Time Spent: 10m
Work Description: szehon-ho commented on pull request #2647:
URL: https://github.com/apache/hive/pull/2647#issuecomment-933653613
Few tests left to fix, TestTxnXXX that now need to initialize TxnStore
manually (init is longer lazy part of getTxnStore).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 659693)
Time Spent: 6h 20m (was: 6h 10m)
> NullPointerException in TxnHandler
> --
>
> Key: HIVE-25522
> URL: https://issues.apache.org/jira/browse/HIVE-25522
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore
>Affects Versions: 3.1.2, 4.0.0
>Reporter: Szehon Ho
>Assignee: Szehon Ho
>Priority: Major
> Labels: pull-request-available
> Time Spent: 6h 20m
> Remaining Estimate: 0h
>
> Environment: Using Iceberg on Hive 3.1.2 standalone metastore. Iceberg
> issues a lot of lock() calls for commits.
> We hit randomly a strange NPE that fails Iceberg commits.
> {noformat}
> 2021-08-21T11:08:05,665 ERROR [pool-6-thread-195]
> metastore.RetryingHMSHandler: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.metastore.txn.TxnHandler.enqueueLockWithRetry(TxnHandler.java:1903)
> at
> org.apache.hadoop.hive.metastore.txn.TxnHandler.lock(TxnHandler.java:1827)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.lock(HiveMetaStore.java:7217)
> at jdk.internal.reflect.GeneratedMethodAccessor52.invoke(Unknown Source)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
> at com.sun.proxy.$Proxy27.lock(Unknown Source)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$lock.getResult(ThriftHiveMetastore.java:18111)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$lock.getResult(ThriftHiveMetastore.java:18095)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> at
> org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:111)
> at
> org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:107)
> at java.base/java.security.AccessController.doPrivileged(Native Method)
> at java.base/javax.security.auth.Subject.doAs(Subject.java:423)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729)
> at
> org.apache.hadoop.hive.metastore.TUGIBasedProcessor.process(TUGIBasedProcessor.java:119)
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at java.base/java.lang.Thread.run(Thread.java:834)
> 2021-08-21T11:08:05,665 ERROR [pool-6-thread-195] server.TThreadPoolServer:
> Error occurred during processing of message.
> java.lang.NullPointerException: null
> at
> org.apache.hadoop.hive.metastore.txn.TxnHandler.enqueueLockWithRetry(TxnHandler.java:1903)
> ~[hive-exec-3.1.2.jar:3.1.2]
> at
> org.apache.hadoop.hive.metastore.txn.TxnHandler.lock(TxnHandler.java:1827)
> ~[hive-exec-3.1.2.jar:3.1.2]
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.lock(HiveMetaStore.java:7217)
> ~[hive-exec-3.1.2.jar:3.1.2]
> at jdk.internal.reflect.GeneratedMethodAccessor52.invoke(Unknown
> Source) ~[?:?]
> at
> jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[?:?]
> at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
>
[jira] [Commented] (HIVE-25487) Caused by :org.apache.hive.com.esotericsoftware.kryo.KryoException:Unable to find class :S_4
[ https://issues.apache.org/jira/browse/HIVE-25487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17423998#comment-17423998 ] László Pintér commented on HIVE-25487: -- [~chinanihc] I see the `pull-request-available` label on this jira, but I can't find the associated PR. Could you please share it with me? Thanks > Caused by :org.apache.hive.com.esotericsoftware.kryo.KryoException:Unable to > find class :S_4 > - > > Key: HIVE-25487 > URL: https://issues.apache.org/jira/browse/HIVE-25487 > Project: Hive > Issue Type: Bug > Components: Hive, Tez >Affects Versions: 3.1.1 >Reporter: chengxinpeng >Assignee: Ashutosh Chauhan >Priority: Blocker > Labels: patch, pull-request-available > Attachments: 微信图片_20210827223829.jpg > > Original Estimate: 168h > Remaining Estimate: 168h > > at java.lang.CLassloader.loadClass(Classloader.java:351) > at java.lang.Class.forName0(Native Hethod) > at java.lang.Class.forName(Class.Java:348) > at org > apache.hive.com.esotericsoftware.kryo.utii.DefaultClassResolver.readName(DefaultClassResolver.java:154) > > ... 63 more > 2021-08-26 09:27:57,158[INFO[App Shared Poo1 -#1 > ]|dag.RootInputinitializerManager: Failed Inputinitiatizer for Input: > _dumy_tahle on vertex vertex_162774552112_1545_1_00 []Map 1] > 2021-08-26 09:27:57,159[ERROR(Dispatcher thread (Central) impl.VertexImpl|: > Vertex Input:dumy table initializer failed, > vertexs=vertex_1627745521112_1545_1_00 [Map 1] > org.apache.tez.dag.app.dag.impl.AMUserCodeException: > java.lang.RuntimeException: failed to load plan: > hdfs://nameservicetenant/tmp/hive/hive/8fblf9db-f922-4e31-af4a-12abb4ba405/hive_2021-08-26 > at > org.apache.tez.dag.app.dag.RootInputInitializerManager.runInitializerAndProcessResult > RootInputinitializerManager.Tava:158) > at > org.apache.tez.dag.app.dag.RootInputInitializerManager.lambda$createAndStartInitializing$1(RootInputInitializerManager.java:132) > at java.util.concurrent.Executors$RunnableAdapter.cal1(Executors.java:511) > at > com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutuzeTask.java:125) > > at > com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:69) > > at > com.google.common.util.concurrent.TrustedListenableFutureTaak.run(TruseedListenableFutureTank.Java:78) > > at > java.uti1.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.jave:1149) > > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > > at java.lang.Thread.run(Thread.java:748) > Caused by: java.lang.RuntimeException Failed to load plan: > hdfs://nameservicetenant/tmp/hive/hive/8fblf9db-f922-4e31-af4a-12abb4ba405/hive_2021-08-26 > > at org apache.hadoop.hive.ql.exec.Utilities.getBaseWork(Utilities.java:528) > at org.apache.hadoop.hive.ql.exec.Utilities.getMapWork (Utilities.java:359) > at org.apache.hadoop.hive.ql.io.HiveInputFormat.init(HiveInputFormat. > java:442) > at > org.apache.hadoop.hive.ql.io.CombineHiveInputformat.getSplits(CombineHiveInputFormat.java:508) > > at > org.apache.tez.mapreduce.hadoop.MRInputHelpers.genecateOldSplits(MRInputHelpers. > java:489) > at > org.apache.tez.mapreduce.hadoop.MRInputHelpers.generateinputSplitsToMem(MRInputHelpers.java:338) > at > org.apache.tez.mapreduce.common.MRInputAMSp1itGenerator.initialize(MRInputAMSplitGenerator.java:121) > at > org.apache.tez.dag.app.dag.RootInputInitializerianager.lambda$runInitializer$2(RootInputInitializerManager, > java:173) > at java.security.AccessController.doPrivileged (Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876) > at > org.apache.tez.dag.app.dag.RootInputInitializerManager.runInitializer(RootInputInitializerManager.java:166) > at > org.apache.tez.dag.app.dag.RootinputInitializerManager.runInitializerAndProcessResult(RootInputInitializertanager.java:147) > ...8 more > Caused by: org.apache.hive.com.esotericsoftwaze.kryo.KryoException: Unable to > find class S_4 > Serialization trace: > parentOperators (org.apache.hadoop.hive.ql.exec.fileSinkOperator) > childOperators (org.apache.hadoop.hive.ql.exec.UDTFOperacor) > childOperators (org.apache.hadoop.hive.ql.exec.SelectOperator) > childOperators (org.apache.hadoop.hive.ql.exec.TableScanOperator) > aliasTowork (org.apache.hadoop.hive.ql.plan.Mapwork) > at > org.apache.hive.com.esotericsoftware.kryo.uti1.DefaultClassResolver.readName(DefaultClassResolver.java:133) > > at >
[jira] [Work logged] (HIVE-23633) Metastore some JDO query objects do not close properly
[ https://issues.apache.org/jira/browse/HIVE-23633?focusedWorklogId=659638=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-659638 ] ASF GitHub Bot logged work on HIVE-23633: - Author: ASF GitHub Bot Created on: 04/Oct/21 15:18 Start Date: 04/Oct/21 15:18 Worklog Time Spent: 10m Work Description: belugabehr edited a comment on pull request #2344: URL: https://github.com/apache/hive/pull/2344#issuecomment-933589151 I'm a little confused here. What version of JDO is Hive using? It looks to me like `Query` should already be `Autocloseable` https://github.com/datanucleus/javax.jdo/blob/fa061bf68e347b8fafca369ab589b7625d8579f3/src/main/java/javax/jdo/Query.java#L68 Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 659638) Time Spent: 15h 20m (was: 15h 10m) > Metastore some JDO query objects do not close properly > -- > > Key: HIVE-23633 > URL: https://issues.apache.org/jira/browse/HIVE-23633 > Project: Hive > Issue Type: Bug >Reporter: Zhihua Deng >Assignee: Zhihua Deng >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-23633.01.patch > > Time Spent: 15h 20m > Remaining Estimate: 0h > > After patched [HIVE-10895|https://issues.apache.org/jira/browse/HIVE-10895], > The metastore still has seen a memory leak on db resources: many > StatementImpls left unclosed. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-23633) Metastore some JDO query objects do not close properly
[ https://issues.apache.org/jira/browse/HIVE-23633?focusedWorklogId=659637=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-659637 ] ASF GitHub Bot logged work on HIVE-23633: - Author: ASF GitHub Bot Created on: 04/Oct/21 15:17 Start Date: 04/Oct/21 15:17 Worklog Time Spent: 10m Work Description: belugabehr commented on pull request #2344: URL: https://github.com/apache/hive/pull/2344#issuecomment-933589151 I'm a little confused here. What version of JDO is Hive using? It looks to me like `Query` should already be `Autocloesable` https://github.com/datanucleus/javax.jdo/blob/fa061bf68e347b8fafca369ab589b7625d8579f3/src/main/java/javax/jdo/Query.java#L68 Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 659637) Time Spent: 15h 10m (was: 15h) > Metastore some JDO query objects do not close properly > -- > > Key: HIVE-23633 > URL: https://issues.apache.org/jira/browse/HIVE-23633 > Project: Hive > Issue Type: Bug >Reporter: Zhihua Deng >Assignee: Zhihua Deng >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-23633.01.patch > > Time Spent: 15h 10m > Remaining Estimate: 0h > > After patched [HIVE-10895|https://issues.apache.org/jira/browse/HIVE-10895], > The metastore still has seen a memory leak on db resources: many > StatementImpls left unclosed. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-23633) Metastore some JDO query objects do not close properly
[
https://issues.apache.org/jira/browse/HIVE-23633?focusedWorklogId=659630=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-659630
]
ASF GitHub Bot logged work on HIVE-23633:
-
Author: ASF GitHub Bot
Created on: 04/Oct/21 15:08
Start Date: 04/Oct/21 15:08
Worklog Time Spent: 10m
Work Description: belugabehr commented on a change in pull request #2344:
URL: https://github.com/apache/hive/pull/2344#discussion_r721458020
##
File path:
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java
##
@@ -368,20 +363,22 @@ private void executeNoResult(final String queryText)
throws SQLException {
}
public Database getDatabase(String catName, String dbName) throws
MetaException{
-Query queryDbSelector = null;
-Query queryDbParams = null;
-try {
+String queryTextDbSelector= "select "
++ "\"DB_ID\", \"NAME\", \"DB_LOCATION_URI\", \"DESC\", "
++ "\"OWNER_NAME\", \"OWNER_TYPE\", \"CTLG_NAME\" , \"CREATE_TIME\",
\"DB_MANAGED_LOCATION_URI\", "
++ "\"TYPE\", \"DATACONNECTOR_NAME\", \"REMOTE_DBNAME\""
++ "FROM "+ DBS
++ " where \"NAME\" = ? and \"CTLG_NAME\" = ? ";
+String queryTextDbParams = "select \"PARAM_KEY\", \"PARAM_VALUE\" "
++ " from " + DATABASE_PARAMS + " "
++ " WHERE \"DB_ID\" = ? "
++ " AND \"PARAM_KEY\" IS NOT NULL";
Review comment:
Not related to this PR directly, but there *has* to be a better way of
doing this. This should come from an external resource.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 659630)
Time Spent: 15h (was: 14h 50m)
> Metastore some JDO query objects do not close properly
> --
>
> Key: HIVE-23633
> URL: https://issues.apache.org/jira/browse/HIVE-23633
> Project: Hive
> Issue Type: Bug
>Reporter: Zhihua Deng
>Assignee: Zhihua Deng
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-23633.01.patch
>
> Time Spent: 15h
> Remaining Estimate: 0h
>
> After patched [HIVE-10895|https://issues.apache.org/jira/browse/HIVE-10895],
> The metastore still has seen a memory leak on db resources: many
> StatementImpls left unclosed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HIVE-25571) Fix Metastore script for Oracle Database
[
https://issues.apache.org/jira/browse/HIVE-25571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Naveen Gangam resolved HIVE-25571.
--
Resolution: Fixed
Fix has been committed to master. Closing the jira. Thank you for the patch.
> Fix Metastore script for Oracle Database
>
>
> Key: HIVE-25571
> URL: https://issues.apache.org/jira/browse/HIVE-25571
> Project: Hive
> Issue Type: Bug
>Reporter: Ayush Saxena
>Assignee: Ayush Saxena
>Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Error:1
> {noformat}
> 354/359 CREATE UNIQUE INDEX DBPRIVILEGEINDEX ON DC_PRIVS
> (AUTHORIZER,NAME,PRINCIPAL_NAME,PRINCIPAL_TYPE,DC_PRIV,GRANTOR,GRANTOR_TYPE);
> Error: ORA-00955: name is already used by an existing object
> (state=42000,code=955)
> Aborting command set because "force" is false and command failed: "CREATE
> UNIQUE INDEX DBPRIVILEGEINDEX ON DC_PRIVS
> (AUTHORIZER,NAME,PRINCIPAL_NAME,PRINCIPAL_TYPE,DC_PRIV,GRANTOR,GRANTOR_TYPE);"
> [ERROR] 2021-09-29 09:18:59.075 [main] MetastoreSchemaTool - Schema
> initialization FAILED! Metastore state would be inconsistent!
> Schema initialization FAILED! Metastore state would be inconsistent!{noformat}
> Error:2
> {noformat}
> Error: ORA-00900: invalid SQL statement (state=42000,code=900)
> Aborting command set because "force" is false and command failed: "===
> -- HIVE-24396
> -- Create DataCo{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25571) Fix Metastore script for Oracle Database
[
https://issues.apache.org/jira/browse/HIVE-25571?focusedWorklogId=659575=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-659575
]
ASF GitHub Bot logged work on HIVE-25571:
-
Author: ASF GitHub Bot
Created on: 04/Oct/21 13:53
Start Date: 04/Oct/21 13:53
Worklog Time Spent: 10m
Work Description: nrg4878 commented on pull request #2681:
URL: https://github.com/apache/hive/pull/2681#issuecomment-933508948
Fix has been merged to master. Please close the PR. Thank you for the patch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 659575)
Time Spent: 20m (was: 10m)
> Fix Metastore script for Oracle Database
>
>
> Key: HIVE-25571
> URL: https://issues.apache.org/jira/browse/HIVE-25571
> Project: Hive
> Issue Type: Bug
>Reporter: Ayush Saxena
>Assignee: Ayush Saxena
>Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Error:1
> {noformat}
> 354/359 CREATE UNIQUE INDEX DBPRIVILEGEINDEX ON DC_PRIVS
> (AUTHORIZER,NAME,PRINCIPAL_NAME,PRINCIPAL_TYPE,DC_PRIV,GRANTOR,GRANTOR_TYPE);
> Error: ORA-00955: name is already used by an existing object
> (state=42000,code=955)
> Aborting command set because "force" is false and command failed: "CREATE
> UNIQUE INDEX DBPRIVILEGEINDEX ON DC_PRIVS
> (AUTHORIZER,NAME,PRINCIPAL_NAME,PRINCIPAL_TYPE,DC_PRIV,GRANTOR,GRANTOR_TYPE);"
> [ERROR] 2021-09-29 09:18:59.075 [main] MetastoreSchemaTool - Schema
> initialization FAILED! Metastore state would be inconsistent!
> Schema initialization FAILED! Metastore state would be inconsistent!{noformat}
> Error:2
> {noformat}
> Error: ORA-00900: invalid SQL statement (state=42000,code=900)
> Aborting command set because "force" is false and command failed: "===
> -- HIVE-24396
> -- Create DataCo{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HIVE-25574) Replace clob with varchar when storing creation metadata
[
https://issues.apache.org/jira/browse/HIVE-25574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa resolved HIVE-25574.
---
Resolution: Fixed
Pushed to master. Thanks [~pvary] for review.
> Replace clob with varchar when storing creation metadata
>
>
> Key: HIVE-25574
> URL: https://issues.apache.org/jira/browse/HIVE-25574
> Project: Hive
> Issue Type: Bug
> Components: Standalone Metastore
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Follow up of HIVE-21940.
> {code}
> table="MV_CREATION_METADATA" detachable="true">
> ...
>
>
>
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25574) Replace clob with varchar when storing creation metadata
[
https://issues.apache.org/jira/browse/HIVE-25574?focusedWorklogId=659562=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-659562
]
ASF GitHub Bot logged work on HIVE-25574:
-
Author: ASF GitHub Bot
Created on: 04/Oct/21 13:30
Start Date: 04/Oct/21 13:30
Worklog Time Spent: 10m
Work Description: kasakrisz merged pull request #2683:
URL: https://github.com/apache/hive/pull/2683
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 659562)
Time Spent: 0.5h (was: 20m)
> Replace clob with varchar when storing creation metadata
>
>
> Key: HIVE-25574
> URL: https://issues.apache.org/jira/browse/HIVE-25574
> Project: Hive
> Issue Type: Bug
> Components: Standalone Metastore
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Follow up of HIVE-21940.
> {code}
> table="MV_CREATION_METADATA" detachable="true">
> ...
>
>
>
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25011) Concurrency: Do not acquire locks for EXPLAIN
[
https://issues.apache.org/jira/browse/HIVE-25011?focusedWorklogId=659525=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-659525
]
ASF GitHub Bot logged work on HIVE-25011:
-
Author: ASF GitHub Bot
Created on: 04/Oct/21 12:10
Start Date: 04/Oct/21 12:10
Worklog Time Spent: 10m
Work Description: pvary commented on pull request #2660:
URL: https://github.com/apache/hive/pull/2660#issuecomment-933421264
We need to clean up the commit message / Jira title to reflect the things we
do in the change, but the change itself looks good to me.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 659525)
Time Spent: 0.5h (was: 20m)
> Concurrency: Do not acquire locks for EXPLAIN
> -
>
> Key: HIVE-25011
> URL: https://issues.apache.org/jira/browse/HIVE-25011
> Project: Hive
> Issue Type: Improvement
> Components: Locking, Transactions
>Affects Versions: 4.0.0
>Reporter: Gopal Vijayaraghavan
>Assignee: Gopal Vijayaraghavan
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-25011.1.patch, HIVE-25011.2.patch
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> {code}
> EXPLAIN UPDATE ...
> {code}
> should not be in conflict with another active ongoing UPDATE operation.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25587) Disable Iceberg table migration for unsupported source file formats
[ https://issues.apache.org/jira/browse/HIVE-25587?focusedWorklogId=659521=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-659521 ] ASF GitHub Bot logged work on HIVE-25587: - Author: ASF GitHub Bot Created on: 04/Oct/21 11:56 Start Date: 04/Oct/21 11:56 Worklog Time Spent: 10m Work Description: marton-bod commented on a change in pull request #2696: URL: https://github.com/apache/hive/pull/2696#discussion_r721291904 ## File path: iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergMetaHook.java ## @@ -95,14 +98,17 @@ AlterTableType.ADDCOLS, AlterTableType.REPLACE_COLUMNS, AlterTableType.RENAME_COLUMN, AlterTableType.ADDPROPS, AlterTableType.DROPPROPS, AlterTableType.SETPARTITIONSPEC, AlterTableType.UPDATE_COLUMNS); + private static final List MIGRATION_ALLOWED_SOURCE_FORMATS = Arrays.stream(FileFormat.values()) + .filter(f -> !f.equals(FileFormat.METADATA)) + .map(FileFormat::name).map(String::toLowerCase).collect(Collectors.toList()); Review comment: Good point, will change this to list the 3 supported types only and explicitly -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 659521) Time Spent: 1h 20m (was: 1h 10m) > Disable Iceberg table migration for unsupported source file formats > --- > > Key: HIVE-25587 > URL: https://issues.apache.org/jira/browse/HIVE-25587 > Project: Hive > Issue Type: Improvement >Reporter: Marton Bod >Assignee: Marton Bod >Priority: Major > Labels: pull-request-available > Time Spent: 1h 20m > Remaining Estimate: 0h > > Currently, we only support migrating ORC, Parquet and Avro tables to Iceberg. > However, there is no check in the code to fail early for other formats (e.g. > text, json, rcfile), which can lead to wasted effort at best, and leaving the > source table unusable at worst. Therefore, we should check the source format > early and shortcircuit for unsupported types. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-25587) Disable Iceberg table migration for unsupported source file formats
[ https://issues.apache.org/jira/browse/HIVE-25587?focusedWorklogId=659520=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-659520 ] ASF GitHub Bot logged work on HIVE-25587: - Author: ASF GitHub Bot Created on: 04/Oct/21 11:54 Start Date: 04/Oct/21 11:54 Worklog Time Spent: 10m Work Description: szlta commented on a change in pull request #2696: URL: https://github.com/apache/hive/pull/2696#discussion_r721290259 ## File path: iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergMetaHook.java ## @@ -95,14 +98,17 @@ AlterTableType.ADDCOLS, AlterTableType.REPLACE_COLUMNS, AlterTableType.RENAME_COLUMN, AlterTableType.ADDPROPS, AlterTableType.DROPPROPS, AlterTableType.SETPARTITIONSPEC, AlterTableType.UPDATE_COLUMNS); + private static final List MIGRATION_ALLOWED_SOURCE_FORMATS = Arrays.stream(FileFormat.values()) + .filter(f -> !f.equals(FileFormat.METADATA)) + .map(FileFormat::name).map(String::toLowerCase).collect(Collectors.toList()); Review comment: I think it would probably be better to list the formats instead i.e. Lists.newArrayList(FileFormat.PARQUET, .. etc). That way it's more readable and safer from a very unlikely scenario a new enum value is added in the future. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 659520) Time Spent: 1h 10m (was: 1h) > Disable Iceberg table migration for unsupported source file formats > --- > > Key: HIVE-25587 > URL: https://issues.apache.org/jira/browse/HIVE-25587 > Project: Hive > Issue Type: Improvement >Reporter: Marton Bod >Assignee: Marton Bod >Priority: Major > Labels: pull-request-available > Time Spent: 1h 10m > Remaining Estimate: 0h > > Currently, we only support migrating ORC, Parquet and Avro tables to Iceberg. > However, there is no check in the code to fail early for other formats (e.g. > text, json, rcfile), which can lead to wasted effort at best, and leaving the > source table unusable at worst. Therefore, we should check the source format > early and shortcircuit for unsupported types. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HIVE-25550) Increase the RM_PROGRESS column max length to fit metrics stat
[
https://issues.apache.org/jira/browse/HIVE-25550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ayush Saxena resolved HIVE-25550.
-
Resolution: Fixed
> Increase the RM_PROGRESS column max length to fit metrics stat
> --
>
> Key: HIVE-25550
> URL: https://issues.apache.org/jira/browse/HIVE-25550
> Project: Hive
> Issue Type: Bug
>Reporter: Ayush Saxena
>Assignee: Ayush Saxena
>Priority: Major
> Labels: pull-request-available
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Presently it fails with the following trace:
> {noformat}
> [[Event Name: EVENT_ALLOC_WRITE_ID; Total Number: 213; Total Time: 85347.0;
> Mean: 400.6901408450704; Median: 392.0; Standard Deviation:
> 33.99178239314741; Variance: 1155.4412702630862; Kurtosis: 83.69411620601193;
> Skewness: 83.69411620601193; 25th Percentile: 384.0; 50th Percentile: 392.0;
> 75th Percentile: 408.0; 90th Percentile: 417.0; Top 5 EventIds(EventId=Time)
> {1498476=791, 1498872=533, 1497805=508, 1498808=500, 1499027=492};]]}"}]}" in
> column ""RM_PROGRESS"" that has maximum length of 4000. Please correct your
> data!
> at
> org.datanucleus.store.rdbms.mapping.datastore.CharRDBMSMapping.setString(CharRDBMSMapping.java:254)
> ~[datanucleus-rdbms-4.1.19.jar:?]
> at
> org.datanucleus.store.rdbms.mapping.java.SingleFieldMapping.setString(SingleFieldMapping.java:180)
> ~{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25571) Fix Metastore script for Oracle Database
[
https://issues.apache.org/jira/browse/HIVE-25571?focusedWorklogId=659494=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-659494
]
ASF GitHub Bot logged work on HIVE-25571:
-
Author: ASF GitHub Bot
Created on: 04/Oct/21 10:35
Start Date: 04/Oct/21 10:35
Worklog Time Spent: 10m
Work Description: ayushtkn commented on pull request #2681:
URL: https://github.com/apache/hive/pull/2681#issuecomment-933356318
Thanx @nrg4878 for the review. Can you help commit
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 659494)
Remaining Estimate: 0h
Time Spent: 10m
> Fix Metastore script for Oracle Database
>
>
> Key: HIVE-25571
> URL: https://issues.apache.org/jira/browse/HIVE-25571
> Project: Hive
> Issue Type: Bug
>Reporter: Ayush Saxena
>Assignee: Ayush Saxena
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Error:1
> {noformat}
> 354/359 CREATE UNIQUE INDEX DBPRIVILEGEINDEX ON DC_PRIVS
> (AUTHORIZER,NAME,PRINCIPAL_NAME,PRINCIPAL_TYPE,DC_PRIV,GRANTOR,GRANTOR_TYPE);
> Error: ORA-00955: name is already used by an existing object
> (state=42000,code=955)
> Aborting command set because "force" is false and command failed: "CREATE
> UNIQUE INDEX DBPRIVILEGEINDEX ON DC_PRIVS
> (AUTHORIZER,NAME,PRINCIPAL_NAME,PRINCIPAL_TYPE,DC_PRIV,GRANTOR,GRANTOR_TYPE);"
> [ERROR] 2021-09-29 09:18:59.075 [main] MetastoreSchemaTool - Schema
> initialization FAILED! Metastore state would be inconsistent!
> Schema initialization FAILED! Metastore state would be inconsistent!{noformat}
> Error:2
> {noformat}
> Error: ORA-00900: invalid SQL statement (state=42000,code=900)
> Aborting command set because "force" is false and command failed: "===
> -- HIVE-24396
> -- Create DataCo{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-25576) Add config to parse date with older date format
[
https://issues.apache.org/jira/browse/HIVE-25576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashish Sharma updated HIVE-25576:
-
Summary: Add config to parse date with older date format (was: Add
"hive.legacy.timeParserPolicy" config to parse date with older date fromat)
> Add config to parse date with older date format
> ---
>
> Key: HIVE-25576
> URL: https://issues.apache.org/jira/browse/HIVE-25576
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 4.0.0
>Reporter: Ashish Sharma
>Assignee: Ashish Sharma
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> *History*
> *Hive 1.2* -
> VM time zone set to Asia/Bangkok
> *Query* - SELECT FROM_UNIXTIME(UNIX_TIMESTAMP('1800-01-01 00:00:00
> UTC','-MM-dd HH:mm:ss z'));
> *Result* - 1800-01-01 07:00:00
> *Implementation details* -
> SimpleDateFormat formatter = new SimpleDateFormat(pattern);
> Long unixtime = formatter.parse(textval).getTime() / 1000;
> Date date = new Date(unixtime * 1000L);
> https://docs.oracle.com/javase/8/docs/api/java/util/Date.html . In official
> documentation they have mention that "Unfortunately, the API for these
> functions was not amenable to internationalization and The corresponding
> methods in Date are deprecated" . Due to that this is producing wrong result
> *Master branch* -
> set hive.local.time.zone=Asia/Bangkok;
> *Query* - SELECT FROM_UNIXTIME(UNIX_TIMESTAMP('1800-01-01 00:00:00
> UTC','-MM-dd HH:mm:ss z'));
> *Result* - 1800-01-01 06:42:04
> *Implementation details* -
> DateTimeFormatter dtformatter = new DateTimeFormatterBuilder()
> .parseCaseInsensitive()
> .appendPattern(pattern)
> .toFormatter();
> ZonedDateTime zonedDateTime =
> ZonedDateTime.parse(textval,dtformatter).withZoneSameInstant(ZoneId.of(timezone));
> Long dttime = zonedDateTime.toInstant().getEpochSecond();
> *Problem*-
> Now *SimpleDateFormat* has been replaced with *DateTimeFormatter* which is
> giving the correct result but it is not backword compatible. Which is causing
> issue at time for migration to new version. Because the older data written is
> using Hive 1.x or 2.x is not compatible with *DateTimeFormatter*.
> *Solution*
> Introduce an config "hive.legacy.timeParserPolicy" with following values -
> 1. *True*- use *SimpleDateFormat*
> 2. *False* - use *DateTimeFormatter*
> Note: apache spark also face the same issue
> https://issues.apache.org/jira/browse/SPARK-30668
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-25576) Add "hive.legacy.timeParserPolicy" config to parse date with older date fromat
[
https://issues.apache.org/jira/browse/HIVE-25576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashish Sharma updated HIVE-25576:
-
Summary: Add "hive.legacy.timeParserPolicy" config to parse date with older
date fromat (was: Raise exception instead of silent change for new
DateTimeformatter)
> Add "hive.legacy.timeParserPolicy" config to parse date with older date fromat
> --
>
> Key: HIVE-25576
> URL: https://issues.apache.org/jira/browse/HIVE-25576
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 4.0.0
>Reporter: Ashish Sharma
>Assignee: Ashish Sharma
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> *History*
> *Hive 1.2* -
> VM time zone set to Asia/Bangkok
> *Query* - SELECT FROM_UNIXTIME(UNIX_TIMESTAMP('1800-01-01 00:00:00
> UTC','-MM-dd HH:mm:ss z'));
> *Result* - 1800-01-01 07:00:00
> *Implementation details* -
> SimpleDateFormat formatter = new SimpleDateFormat(pattern);
> Long unixtime = formatter.parse(textval).getTime() / 1000;
> Date date = new Date(unixtime * 1000L);
> https://docs.oracle.com/javase/8/docs/api/java/util/Date.html . In official
> documentation they have mention that "Unfortunately, the API for these
> functions was not amenable to internationalization and The corresponding
> methods in Date are deprecated" . Due to that this is producing wrong result
> *Master branch* -
> set hive.local.time.zone=Asia/Bangkok;
> *Query* - SELECT FROM_UNIXTIME(UNIX_TIMESTAMP('1800-01-01 00:00:00
> UTC','-MM-dd HH:mm:ss z'));
> *Result* - 1800-01-01 06:42:04
> *Implementation details* -
> DateTimeFormatter dtformatter = new DateTimeFormatterBuilder()
> .parseCaseInsensitive()
> .appendPattern(pattern)
> .toFormatter();
> ZonedDateTime zonedDateTime =
> ZonedDateTime.parse(textval,dtformatter).withZoneSameInstant(ZoneId.of(timezone));
> Long dttime = zonedDateTime.toInstant().getEpochSecond();
> *Problem*-
> Now *SimpleDateFormat* has been replaced with *DateTimeFormatter* which is
> giving the correct result but it is not backword compatible. Which is causing
> issue at time for migration to new version. Because the older data written is
> using Hive 1.x or 2.x is not compatible with *DateTimeFormatter*.
> *Solution*
> Introduce an config "hive.legacy.timeParserPolicy" with following values -
> 1. *True*- use *SimpleDateFormat*
> 2. *False* - use *DateTimeFormatter*
> Note: apache spark also face the same issue
> https://issues.apache.org/jira/browse/SPARK-30668
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work started] (HIVE-25576) Raise exception instead of silent change for new DateTimeformatter
[
https://issues.apache.org/jira/browse/HIVE-25576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HIVE-25576 started by Ashish Sharma.
> Raise exception instead of silent change for new DateTimeformatter
> --
>
> Key: HIVE-25576
> URL: https://issues.apache.org/jira/browse/HIVE-25576
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 4.0.0
>Reporter: Ashish Sharma
>Assignee: Ashish Sharma
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> *History*
> *Hive 1.2* -
> VM time zone set to Asia/Bangkok
> *Query* - SELECT FROM_UNIXTIME(UNIX_TIMESTAMP('1800-01-01 00:00:00
> UTC','-MM-dd HH:mm:ss z'));
> *Result* - 1800-01-01 07:00:00
> *Implementation details* -
> SimpleDateFormat formatter = new SimpleDateFormat(pattern);
> Long unixtime = formatter.parse(textval).getTime() / 1000;
> Date date = new Date(unixtime * 1000L);
> https://docs.oracle.com/javase/8/docs/api/java/util/Date.html . In official
> documentation they have mention that "Unfortunately, the API for these
> functions was not amenable to internationalization and The corresponding
> methods in Date are deprecated" . Due to that this is producing wrong result
> *Master branch* -
> set hive.local.time.zone=Asia/Bangkok;
> *Query* - SELECT FROM_UNIXTIME(UNIX_TIMESTAMP('1800-01-01 00:00:00
> UTC','-MM-dd HH:mm:ss z'));
> *Result* - 1800-01-01 06:42:04
> *Implementation details* -
> DateTimeFormatter dtformatter = new DateTimeFormatterBuilder()
> .parseCaseInsensitive()
> .appendPattern(pattern)
> .toFormatter();
> ZonedDateTime zonedDateTime =
> ZonedDateTime.parse(textval,dtformatter).withZoneSameInstant(ZoneId.of(timezone));
> Long dttime = zonedDateTime.toInstant().getEpochSecond();
> *Problem*-
> Now *SimpleDateFormat* has been replaced with *DateTimeFormatter* which is
> giving the correct result but it is not backword compatible. Which is causing
> issue at time for migration to new version. Because the older data written is
> using Hive 1.x or 2.x is not compatible with *DateTimeFormatter*.
> *Solution*
> Introduce an config "hive.legacy.timeParserPolicy" with following values -
> 1. *True*- use *SimpleDateFormat*
> 2. *False* - use *DateTimeFormatter*
> Note: apache spark also face the same issue
> https://issues.apache.org/jira/browse/SPARK-30668
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25550) Increase the RM_PROGRESS column max length to fit metrics stat
[
https://issues.apache.org/jira/browse/HIVE-25550?focusedWorklogId=659454=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-659454
]
ASF GitHub Bot logged work on HIVE-25550:
-
Author: ASF GitHub Bot
Created on: 04/Oct/21 08:45
Start Date: 04/Oct/21 08:45
Worklog Time Spent: 10m
Work Description: aasha merged pull request #2668:
URL: https://github.com/apache/hive/pull/2668
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 659454)
Time Spent: 50m (was: 40m)
> Increase the RM_PROGRESS column max length to fit metrics stat
> --
>
> Key: HIVE-25550
> URL: https://issues.apache.org/jira/browse/HIVE-25550
> Project: Hive
> Issue Type: Bug
>Reporter: Ayush Saxena
>Assignee: Ayush Saxena
>Priority: Major
> Labels: pull-request-available
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Presently it fails with the following trace:
> {noformat}
> [[Event Name: EVENT_ALLOC_WRITE_ID; Total Number: 213; Total Time: 85347.0;
> Mean: 400.6901408450704; Median: 392.0; Standard Deviation:
> 33.99178239314741; Variance: 1155.4412702630862; Kurtosis: 83.69411620601193;
> Skewness: 83.69411620601193; 25th Percentile: 384.0; 50th Percentile: 392.0;
> 75th Percentile: 408.0; 90th Percentile: 417.0; Top 5 EventIds(EventId=Time)
> {1498476=791, 1498872=533, 1497805=508, 1498808=500, 1499027=492};]]}"}]}" in
> column ""RM_PROGRESS"" that has maximum length of 4000. Please correct your
> data!
> at
> org.datanucleus.store.rdbms.mapping.datastore.CharRDBMSMapping.setString(CharRDBMSMapping.java:254)
> ~[datanucleus-rdbms-4.1.19.jar:?]
> at
> org.datanucleus.store.rdbms.mapping.java.SingleFieldMapping.setString(SingleFieldMapping.java:180)
> ~{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-25586) Add Iceberg to list of allowed serdes for incompatible col type changes
[ https://issues.apache.org/jira/browse/HIVE-25586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ádám Szita updated HIVE-25586: -- Fix Version/s: 4.0.0 Resolution: Fixed Status: Resolved (was: Patch Available) Pushed to master. Thanks for the review [~mbod]! > Add Iceberg to list of allowed serdes for incompatible col type changes > --- > > Key: HIVE-25586 > URL: https://issues.apache.org/jira/browse/HIVE-25586 > Project: Hive > Issue Type: Improvement >Reporter: Ádám Szita >Assignee: Ádám Szita >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-25586) Add Iceberg to list of allowed serdes for incompatible col type changes
[ https://issues.apache.org/jira/browse/HIVE-25586?focusedWorklogId=659450=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-659450 ] ASF GitHub Bot logged work on HIVE-25586: - Author: ASF GitHub Bot Created on: 04/Oct/21 08:27 Start Date: 04/Oct/21 08:27 Worklog Time Spent: 10m Work Description: szlta merged pull request #2695: URL: https://github.com/apache/hive/pull/2695 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 659450) Remaining Estimate: 0h Time Spent: 10m > Add Iceberg to list of allowed serdes for incompatible col type changes > --- > > Key: HIVE-25586 > URL: https://issues.apache.org/jira/browse/HIVE-25586 > Project: Hive > Issue Type: Improvement >Reporter: Ádám Szita >Assignee: Ádám Szita >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25586) Add Iceberg to list of allowed serdes for incompatible col type changes
[ https://issues.apache.org/jira/browse/HIVE-25586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-25586: -- Labels: pull-request-available (was: ) > Add Iceberg to list of allowed serdes for incompatible col type changes > --- > > Key: HIVE-25586 > URL: https://issues.apache.org/jira/browse/HIVE-25586 > Project: Hive > Issue Type: Improvement >Reporter: Ádám Szita >Assignee: Ádám Szita >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25586) Add Iceberg to list of allowed serdes for incompatible col type changes
[ https://issues.apache.org/jira/browse/HIVE-25586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ádám Szita updated HIVE-25586: -- Status: Patch Available (was: Open) > Add Iceberg to list of allowed serdes for incompatible col type changes > --- > > Key: HIVE-25586 > URL: https://issues.apache.org/jira/browse/HIVE-25586 > Project: Hive > Issue Type: Improvement >Reporter: Ádám Szita >Assignee: Ádám Szita >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25564) Enable dropping HMS tables despite Iceberg metadata problems
[ https://issues.apache.org/jira/browse/HIVE-25564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-25564: -- Labels: pull-request-available (was: ) > Enable dropping HMS tables despite Iceberg metadata problems > > > Key: HIVE-25564 > URL: https://issues.apache.org/jira/browse/HIVE-25564 > Project: Hive > Issue Type: Improvement >Reporter: Ádám Szita >Assignee: Ádám Szita >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > This is a port of an original Iceberg change into Hive codebase: > https://github.com/apache/iceberg/pull/2583 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-25564) Enable dropping HMS tables despite Iceberg metadata problems
[ https://issues.apache.org/jira/browse/HIVE-25564?focusedWorklogId=659446=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-659446 ] ASF GitHub Bot logged work on HIVE-25564: - Author: ASF GitHub Bot Created on: 04/Oct/21 08:23 Start Date: 04/Oct/21 08:23 Worklog Time Spent: 10m Work Description: szlta merged pull request #2677: URL: https://github.com/apache/hive/pull/2677 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 659446) Remaining Estimate: 0h Time Spent: 10m > Enable dropping HMS tables despite Iceberg metadata problems > > > Key: HIVE-25564 > URL: https://issues.apache.org/jira/browse/HIVE-25564 > Project: Hive > Issue Type: Improvement >Reporter: Ádám Szita >Assignee: Ádám Szita >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > This is a port of an original Iceberg change into Hive codebase: > https://github.com/apache/iceberg/pull/2583 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25564) Enable dropping HMS tables despite Iceberg metadata problems
[ https://issues.apache.org/jira/browse/HIVE-25564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ádám Szita updated HIVE-25564: -- Fix Version/s: 4.0.0 Resolution: Fixed Status: Resolved (was: Patch Available) Pushed to master. Thanks for reviewing [~mbod]. > Enable dropping HMS tables despite Iceberg metadata problems > > > Key: HIVE-25564 > URL: https://issues.apache.org/jira/browse/HIVE-25564 > Project: Hive > Issue Type: Improvement >Reporter: Ádám Szita >Assignee: Ádám Szita >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Time Spent: 10m > Remaining Estimate: 0h > > This is a port of an original Iceberg change into Hive codebase: > https://github.com/apache/iceberg/pull/2583 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25563) Iceberg table operations hang a long time if metadata is missing/corrupted
[
https://issues.apache.org/jira/browse/HIVE-25563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ádám Szita updated HIVE-25563:
--
Fix Version/s: 4.0.0
Resolution: Fixed
Status: Resolved (was: Patch Available)
Pushed to master. Thanks for the review [~mbod].
> Iceberg table operations hang a long time if metadata is missing/corrupted
> --
>
> Key: HIVE-25563
> URL: https://issues.apache.org/jira/browse/HIVE-25563
> Project: Hive
> Issue Type: Bug
>Reporter: Ádám Szita
>Assignee: Ádám Szita
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> For all Iceberg table operations (select, insert, alter, etc..) Hive tries to
> load the Iceberg table by reading in its metadata files.
> If these metadata files are not present or are inaccessible for any reason,
> then operations on such a table will result in a long > 10 minutes hang for
> the user's session. This is because there's a retry logic with exponential
> intervals, and 21 retries by default in place:
> {code:java}
> java.lang.Thread.State: TIMED_WAITING (sleeping)
> at java.lang.Thread.sleep(Native Method)
> at java.lang.Thread.sleep(Thread.java:340)
> at java.util.concurrent.TimeUnit.sleep(TimeUnit.java:386)
> at
> org.apache.iceberg.util.Tasks$Builder.runTaskWithRetry(Tasks.java:453)
> at
> org.apache.iceberg.util.Tasks$Builder.runSingleThreaded(Tasks.java:214)
> at org.apache.iceberg.util.Tasks$Builder.run(Tasks.java:198)
> at org.apache.iceberg.util.Tasks$Builder.run(Tasks.java:190)
> at
> org.apache.iceberg.BaseMetastoreTableOperations.refreshFromMetadataLocation(BaseMetastoreTableOperations.java:178)
> at
> org.apache.iceberg.BaseMetastoreTableOperations.refreshFromMetadataLocation(BaseMetastoreTableOperations.java:160)
> at
> org.apache.iceberg.hive.HiveTableOperations.doRefresh(HiveTableOperations.java:183)
> at
> org.apache.iceberg.BaseMetastoreTableOperations.refresh(BaseMetastoreTableOperations.java:94)
> at
> org.apache.iceberg.BaseMetastoreTableOperations.current(BaseMetastoreTableOperations.java:77)
> at
> org.apache.iceberg.BaseMetastoreCatalog.loadTable(BaseMetastoreCatalog.java:93)
> at org.apache.iceberg.mr.Catalogs.loadTable(Catalogs.java:106)
> at org.apache.iceberg.mr.Catalogs.loadTable(Catalogs.java:96)
> at
> org.apache.iceberg.mr.hive.IcebergTableUtil.lambda$getTable$2(IcebergTableUtil.java:69)
> at
> org.apache.iceberg.mr.hive.IcebergTableUtil$$Lambda$284/1429147768.get(Unknown
> Source)
> at java.util.Optional.orElseGet(Optional.java:267)
> at
> org.apache.iceberg.mr.hive.IcebergTableUtil.getTable(IcebergTableUtil.java:66)
> at
> org.apache.iceberg.mr.hive.HiveIcebergSerDe.initialize(HiveIcebergSerDe.java:105)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreUtils.getDeserializer(HiveMetaStoreUtils.java:95)
> ...
> {code}
> We should make the retry count configurable, and with a lower default setting.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25563) Iceberg table operations hang a long time if metadata is missing/corrupted
[
https://issues.apache.org/jira/browse/HIVE-25563?focusedWorklogId=659445=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-659445
]
ASF GitHub Bot logged work on HIVE-25563:
-
Author: ASF GitHub Bot
Created on: 04/Oct/21 08:21
Start Date: 04/Oct/21 08:21
Worklog Time Spent: 10m
Work Description: szlta merged pull request #2676:
URL: https://github.com/apache/hive/pull/2676
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 659445)
Time Spent: 0.5h (was: 20m)
> Iceberg table operations hang a long time if metadata is missing/corrupted
> --
>
> Key: HIVE-25563
> URL: https://issues.apache.org/jira/browse/HIVE-25563
> Project: Hive
> Issue Type: Bug
>Reporter: Ádám Szita
>Assignee: Ádám Szita
>Priority: Major
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> For all Iceberg table operations (select, insert, alter, etc..) Hive tries to
> load the Iceberg table by reading in its metadata files.
> If these metadata files are not present or are inaccessible for any reason,
> then operations on such a table will result in a long > 10 minutes hang for
> the user's session. This is because there's a retry logic with exponential
> intervals, and 21 retries by default in place:
> {code:java}
> java.lang.Thread.State: TIMED_WAITING (sleeping)
> at java.lang.Thread.sleep(Native Method)
> at java.lang.Thread.sleep(Thread.java:340)
> at java.util.concurrent.TimeUnit.sleep(TimeUnit.java:386)
> at
> org.apache.iceberg.util.Tasks$Builder.runTaskWithRetry(Tasks.java:453)
> at
> org.apache.iceberg.util.Tasks$Builder.runSingleThreaded(Tasks.java:214)
> at org.apache.iceberg.util.Tasks$Builder.run(Tasks.java:198)
> at org.apache.iceberg.util.Tasks$Builder.run(Tasks.java:190)
> at
> org.apache.iceberg.BaseMetastoreTableOperations.refreshFromMetadataLocation(BaseMetastoreTableOperations.java:178)
> at
> org.apache.iceberg.BaseMetastoreTableOperations.refreshFromMetadataLocation(BaseMetastoreTableOperations.java:160)
> at
> org.apache.iceberg.hive.HiveTableOperations.doRefresh(HiveTableOperations.java:183)
> at
> org.apache.iceberg.BaseMetastoreTableOperations.refresh(BaseMetastoreTableOperations.java:94)
> at
> org.apache.iceberg.BaseMetastoreTableOperations.current(BaseMetastoreTableOperations.java:77)
> at
> org.apache.iceberg.BaseMetastoreCatalog.loadTable(BaseMetastoreCatalog.java:93)
> at org.apache.iceberg.mr.Catalogs.loadTable(Catalogs.java:106)
> at org.apache.iceberg.mr.Catalogs.loadTable(Catalogs.java:96)
> at
> org.apache.iceberg.mr.hive.IcebergTableUtil.lambda$getTable$2(IcebergTableUtil.java:69)
> at
> org.apache.iceberg.mr.hive.IcebergTableUtil$$Lambda$284/1429147768.get(Unknown
> Source)
> at java.util.Optional.orElseGet(Optional.java:267)
> at
> org.apache.iceberg.mr.hive.IcebergTableUtil.getTable(IcebergTableUtil.java:66)
> at
> org.apache.iceberg.mr.hive.HiveIcebergSerDe.initialize(HiveIcebergSerDe.java:105)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreUtils.getDeserializer(HiveMetaStoreUtils.java:95)
> ...
> {code}
> We should make the retry count configurable, and with a lower default setting.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25587) Disable Iceberg table migration for unsupported source file formats
[
https://issues.apache.org/jira/browse/HIVE-25587?focusedWorklogId=659436=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-659436
]
ASF GitHub Bot logged work on HIVE-25587:
-
Author: ASF GitHub Bot
Created on: 04/Oct/21 07:39
Start Date: 04/Oct/21 07:39
Worklog Time Spent: 10m
Work Description: pvary commented on a change in pull request #2696:
URL: https://github.com/apache/hive/pull/2696#discussion_r721110740
##
File path:
iceberg/iceberg-handler/src/test/java/org/apache/iceberg/mr/hive/TestHiveIcebergMigration.java
##
@@ -133,6 +135,39 @@ public void
testRollbackMigratePartitionedBucketedHiveTableToIceberg() throws TE
validateMigrationRollback(tableName);
}
+ @Test
+ public void testMigrationFailsForUnsupportedSourceFileFormat() {
+// enough to test once
+Assume.assumeTrue(fileFormat == FileFormat.ORC && isVectorized &&
+testTableType == TestTables.TestTableType.HIVE_CATALOG);
+String tableName = "tbl_unsupported";
+List formats = ImmutableList.of("TEXTFILE", "JSONFILE", "RCFILE",
"SEQUENCEFILE");
+formats.forEach(format -> {
+ shell.executeStatement("CREATE EXTERNAL TABLE " + tableName + " (a int)
STORED AS " + format + " " +
+ testTables.locationForCreateTableSQL(TableIdentifier.of("default",
tableName)));
+ shell.executeStatement("INSERT INTO " + tableName + " VALUES (1), (2),
(3)");
+ AssertHelpers.assertThrows("Migrating a " + format + " table to Iceberg
should have thrown an exception.",
+ IllegalArgumentException.class, "Cannot convert hive table to
iceberg with input format: ",
+ () -> shell.executeStatement("ALTER TABLE " + tableName + " SET
TBLPROPERTIES " +
+
"('storage_handler'='org.apache.iceberg.mr.hive.HiveIcebergStorageHandler')"));
+ shell.executeStatement("DROP TABLE " + tableName);
+});
+ }
+
+ @Test
+ public void testMigrationFailsForManagedTable() {
+// enough to test once
+Assume.assumeTrue(fileFormat == FileFormat.ORC && isVectorized &&
Review comment:
Nope - I did not know we have
`testMigrationFailsForUnsupportedSourceFileFormat`. Please disregard my comment
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 659436)
Time Spent: 1h (was: 50m)
> Disable Iceberg table migration for unsupported source file formats
> ---
>
> Key: HIVE-25587
> URL: https://issues.apache.org/jira/browse/HIVE-25587
> Project: Hive
> Issue Type: Improvement
>Reporter: Marton Bod
>Assignee: Marton Bod
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h
> Remaining Estimate: 0h
>
> Currently, we only support migrating ORC, Parquet and Avro tables to Iceberg.
> However, there is no check in the code to fail early for other formats (e.g.
> text, json, rcfile), which can lead to wasted effort at best, and leaving the
> source table unusable at worst. Therefore, we should check the source format
> early and shortcircuit for unsupported types.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25587) Disable Iceberg table migration for unsupported source file formats
[
https://issues.apache.org/jira/browse/HIVE-25587?focusedWorklogId=659435=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-659435
]
ASF GitHub Bot logged work on HIVE-25587:
-
Author: ASF GitHub Bot
Created on: 04/Oct/21 07:35
Start Date: 04/Oct/21 07:35
Worklog Time Spent: 10m
Work Description: marton-bod commented on a change in pull request #2696:
URL: https://github.com/apache/hive/pull/2696#discussion_r721108666
##
File path:
iceberg/iceberg-handler/src/test/java/org/apache/iceberg/mr/hive/TestHiveIcebergMigration.java
##
@@ -133,6 +135,39 @@ public void
testRollbackMigratePartitionedBucketedHiveTableToIceberg() throws TE
validateMigrationRollback(tableName);
}
+ @Test
+ public void testMigrationFailsForUnsupportedSourceFileFormat() {
+// enough to test once
+Assume.assumeTrue(fileFormat == FileFormat.ORC && isVectorized &&
+testTableType == TestTables.TestTableType.HIVE_CATALOG);
+String tableName = "tbl_unsupported";
+List formats = ImmutableList.of("TEXTFILE", "JSONFILE", "RCFILE",
"SEQUENCEFILE");
+formats.forEach(format -> {
+ shell.executeStatement("CREATE EXTERNAL TABLE " + tableName + " (a int)
STORED AS " + format + " " +
+ testTables.locationForCreateTableSQL(TableIdentifier.of("default",
tableName)));
+ shell.executeStatement("INSERT INTO " + tableName + " VALUES (1), (2),
(3)");
+ AssertHelpers.assertThrows("Migrating a " + format + " table to Iceberg
should have thrown an exception.",
+ IllegalArgumentException.class, "Cannot convert hive table to
iceberg with input format: ",
+ () -> shell.executeStatement("ALTER TABLE " + tableName + " SET
TBLPROPERTIES " +
+
"('storage_handler'='org.apache.iceberg.mr.hive.HiveIcebergStorageHandler')"));
+ shell.executeStatement("DROP TABLE " + tableName);
+});
+ }
+
+ @Test
+ public void testMigrationFailsForManagedTable() {
+// enough to test once
+Assume.assumeTrue(fileFormat == FileFormat.ORC && isVectorized &&
Review comment:
I think `testMigrationFailsForUnsupportedSourceFileFormat` already
provides tests for unsupported source table file formats. Or do you mean having
a test where the source table is managed AND its format is unsupported at the
same time?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 659435)
Time Spent: 50m (was: 40m)
> Disable Iceberg table migration for unsupported source file formats
> ---
>
> Key: HIVE-25587
> URL: https://issues.apache.org/jira/browse/HIVE-25587
> Project: Hive
> Issue Type: Improvement
>Reporter: Marton Bod
>Assignee: Marton Bod
>Priority: Major
> Labels: pull-request-available
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Currently, we only support migrating ORC, Parquet and Avro tables to Iceberg.
> However, there is no check in the code to fail early for other formats (e.g.
> text, json, rcfile), which can lead to wasted effort at best, and leaving the
> source table unusable at worst. Therefore, we should check the source format
> early and shortcircuit for unsupported types.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25553) Support Map data-type natively in Arrow format
[
https://issues.apache.org/jira/browse/HIVE-25553?focusedWorklogId=659417=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-659417
]
ASF GitHub Bot logged work on HIVE-25553:
-
Author: ASF GitHub Bot
Created on: 04/Oct/21 06:25
Start Date: 04/Oct/21 06:25
Worklog Time Spent: 10m
Work Description: sankarh commented on a change in pull request #2689:
URL: https://github.com/apache/hive/pull/2689#discussion_r721068047
##
File path:
ql/src/java/org/apache/hadoop/hive/ql/io/arrow/ArrowColumnarBatchSerDe.java
##
@@ -170,7 +171,7 @@ private static Field toField(String name, TypeInfo
typeInfo) {
for (int i = 0; i < structSize; i++) {
structFields.add(toField(fieldNames.get(i), fieldTypeInfos.get(i)));
}
-return new Field(name, FieldType.nullable(MinorType.STRUCT.getType()),
structFields);
+return new Field(name, new FieldType(false, new ArrowType.Struct(),
null), structFields);
Review comment:
I think, this point holds good only if Struct is a child element type of
Map column. If the column type is STRUCT, then it should be nullable. Can we
make this change only for first case?
##
File path:
ql/src/java/org/apache/hadoop/hive/llap/WritableByteChannelAdapter.java
##
@@ -93,7 +93,7 @@ public int write(ByteBuffer src) throws IOException {
int size = src.remaining();
//Down the semaphore or block until available
takeWriteResources(1);
-ByteBuf buf = allocator.buffer(size);
+ByteBuf buf = allocator.getAsByteBufAllocator().buffer(size);
Review comment:
Why this change relevant to this jira?
##
File path: ql/src/java/org/apache/hadoop/hive/ql/io/arrow/Serializer.java
##
@@ -226,7 +226,7 @@ public ArrowWrapperWritable
serializeBatch(VectorizedRowBatch vectorizedRowBatch
}
private static FieldType toFieldType(TypeInfo typeInfo) {
-return new FieldType(true, toArrowType(typeInfo), null);
+return new FieldType(false, toArrowType(typeInfo), null);
Review comment:
I think, even this change should come only if list or struct are
elements of other complex data types. We cannot make it global for all data
types.
##
File path:
ql/src/java/org/apache/hadoop/hive/ql/io/arrow/ArrowColumnarBatchSerDe.java
##
@@ -185,7 +186,7 @@ private static Field toField(String name, TypeInfo
typeInfo) {
final TypeInfo keyTypeInfo = mapTypeInfo.getMapKeyTypeInfo();
final TypeInfo valueTypeInfo = mapTypeInfo.getMapValueTypeInfo();
final StructTypeInfo mapStructTypeInfo = new StructTypeInfo();
-mapStructTypeInfo.setAllStructFieldNames(Lists.newArrayList("keys",
"values"));
+mapStructTypeInfo.setAllStructFieldNames(Lists.newArrayList("key",
"value"));
Review comment:
Make sense.
##
File path:
itests/hive-unit/src/test/java/org/apache/hive/jdbc/TestJdbcWithMiniLlapArrow.java
##
@@ -123,8 +123,8 @@ public static void afterTest() {
return new LlapArrowRowInputFormat(Long.MAX_VALUE);
}
- // Currently MAP type is not supported. Add it back when Arrow 1.0 is
released.
- // See: SPARK-21187
+ // Currently, loading from a text file gives errors with Map dataType.
+ // This needs to be fixed when adding support for non-ORC writes (text and
parquet) for the llap-ext-client.
Review comment:
This comment doesn't make sense here as this test is not specific to
llap-ext-client. Instead mention the Hive JIRA which would solve this issue.
##
File path:
itests/hive-unit/src/test/java/org/apache/hive/jdbc/TestJdbcWithMiniLlapArrow.java
##
@@ -123,8 +123,8 @@ public static void afterTest() {
return new LlapArrowRowInputFormat(Long.MAX_VALUE);
}
- // Currently MAP type is not supported. Add it back when Arrow 1.0 is
released.
- // See: SPARK-21187
+ // Currently, loading from a text file gives errors with Map dataType.
+ // This needs to be fixed when adding support for non-ORC writes (text and
parquet) for the llap-ext-client.
Review comment:
Ideally, anything that works for ORC should work for non-ORC as well.
Pls enable the tests on Map data type and see if it works after Arrow upgrade.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 659417)
Time Spent: 1h 10m (was: 1h)
> Support Map data-type natively in Arrow format
> --
>
> Key: HIVE-25553
> URL: https://issues.apache.org/jira/browse/HIVE-25553
>
