[jira] [Resolved] (HIVE-23931) Send ValidWriteIdList and tableId to get_*_constraints HMS APIs
[ https://issues.apache.org/jira/browse/HIVE-23931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan resolved HIVE-23931. - Fix Version/s: 4.0.0 Target Version/s: 4.0.0 Resolution: Fixed Merged to master! Thanks [~ashish-kumar-sharma] for the patch and [~kishendas], [~kgyrtkirk] for the reviews! > Send ValidWriteIdList and tableId to get_*_constraints HMS APIs > --- > > Key: HIVE-23931 > URL: https://issues.apache.org/jira/browse/HIVE-23931 > Project: Hive > Issue Type: Sub-task >Reporter: Kishen Das >Assignee: Ashish Sharma >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Time Spent: 2.5h > Remaining Estimate: 0h > > Send ValidWriteIdList and tableId to get_*_constraints HMS APIs. This would > be required in order to decide whether the response should be served from the > Cache or backing DB. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23931) Send ValidWriteIdList and tableId to get_*_constraints HMS APIs
[ https://issues.apache.org/jira/browse/HIVE-23931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-23931: Component/s: Standalone Metastore > Send ValidWriteIdList and tableId to get_*_constraints HMS APIs > --- > > Key: HIVE-23931 > URL: https://issues.apache.org/jira/browse/HIVE-23931 > Project: Hive > Issue Type: Sub-task > Components: Standalone Metastore >Reporter: Kishen Das >Assignee: Ashish Sharma >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Time Spent: 2.5h > Remaining Estimate: 0h > > Send ValidWriteIdList and tableId to get_*_constraints HMS APIs. This would > be required in order to decide whether the response should be served from the > Cache or backing DB. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-23931) Send ValidWriteIdList and tableId to get_*_constraints HMS APIs
[ https://issues.apache.org/jira/browse/HIVE-23931?focusedWorklogId=622378=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-622378 ] ASF GitHub Bot logged work on HIVE-23931: - Author: ASF GitHub Bot Created on: 14/Jul/21 04:57 Start Date: 14/Jul/21 04:57 Worklog Time Spent: 10m Work Description: sankarh merged pull request #2211: URL: https://github.com/apache/hive/pull/2211 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 622378) Time Spent: 2.5h (was: 2h 20m) > Send ValidWriteIdList and tableId to get_*_constraints HMS APIs > --- > > Key: HIVE-23931 > URL: https://issues.apache.org/jira/browse/HIVE-23931 > Project: Hive > Issue Type: Sub-task >Reporter: Kishen Das >Assignee: Ashish Sharma >Priority: Major > Labels: pull-request-available > Time Spent: 2.5h > Remaining Estimate: 0h > > Send ValidWriteIdList and tableId to get_*_constraints HMS APIs. This would > be required in order to decide whether the response should be served from the > Cache or backing DB. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-25246) Fix the clean up of open repl created transactions
[
https://issues.apache.org/jira/browse/HIVE-25246?focusedWorklogId=622377=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-622377
]
ASF GitHub Bot logged work on HIVE-25246:
-
Author: ASF GitHub Bot
Created on: 14/Jul/21 04:40
Start Date: 14/Jul/21 04:40
Worklog Time Spent: 10m
Work Description: hmangla98 commented on a change in pull request #2396:
URL: https://github.com/apache/hive/pull/2396#discussion_r669277084
##
File path:
itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/parse/TestReplicationScenariosAcrossInstances.java
##
@@ -692,8 +693,9 @@ public void testBootStrapDumpOfWarehouse() throws Throwable
{
.run("alter database default set dbproperties
('hive.repl.ckpt.key'='', 'repl.last.id'='')");
try {
replica.load("", "`*`");
-} catch (SemanticException e) {
- assertEquals("REPL LOAD * is not supported", e.getMessage());
+ assertTrue(false);
Review comment:
This particular error is outdated. Earlier, repl load did not raise any
exception and hence catch clause was of no use. In this change, Repl load will
throw an exception stating that dbname can't be null.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 622377)
Time Spent: 5h (was: 4h 50m)
> Fix the clean up of open repl created transactions
> --
>
> Key: HIVE-25246
> URL: https://issues.apache.org/jira/browse/HIVE-25246
> Project: Hive
> Issue Type: Improvement
>Reporter: Haymant Mangla
>Assignee: Haymant Mangla
>Priority: Major
> Labels: pull-request-available
> Time Spent: 5h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25246) Fix the clean up of open repl created transactions
[
https://issues.apache.org/jira/browse/HIVE-25246?focusedWorklogId=622374=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-622374
]
ASF GitHub Bot logged work on HIVE-25246:
-
Author: ASF GitHub Bot
Created on: 14/Jul/21 04:33
Start Date: 14/Jul/21 04:33
Worklog Time Spent: 10m
Work Description: pkumarsinha commented on a change in pull request #2396:
URL: https://github.com/apache/hive/pull/2396#discussion_r669272559
##
File path:
itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/parse/TestReplicationScenariosAcrossInstances.java
##
@@ -692,8 +693,9 @@ public void testBootStrapDumpOfWarehouse() throws Throwable
{
.run("alter database default set dbproperties
('hive.repl.ckpt.key'='', 'repl.last.id'='')");
try {
replica.load("", "`*`");
-} catch (SemanticException e) {
- assertEquals("REPL LOAD * is not supported", e.getMessage());
+ assertTrue(false);
Review comment:
This will always fail. Why do you need to have such assertion?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 622374)
Time Spent: 4h 40m (was: 4.5h)
> Fix the clean up of open repl created transactions
> --
>
> Key: HIVE-25246
> URL: https://issues.apache.org/jira/browse/HIVE-25246
> Project: Hive
> Issue Type: Improvement
>Reporter: Haymant Mangla
>Assignee: Haymant Mangla
>Priority: Major
> Labels: pull-request-available
> Time Spent: 4h 40m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25246) Fix the clean up of open repl created transactions
[
https://issues.apache.org/jira/browse/HIVE-25246?focusedWorklogId=622375=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-622375
]
ASF GitHub Bot logged work on HIVE-25246:
-
Author: ASF GitHub Bot
Created on: 14/Jul/21 04:33
Start Date: 14/Jul/21 04:33
Worklog Time Spent: 10m
Work Description: pkumarsinha commented on a change in pull request #2396:
URL: https://github.com/apache/hive/pull/2396#discussion_r669272781
##
File path:
itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/parse/TestReplicationScenariosAcrossInstances.java
##
@@ -692,8 +693,9 @@ public void testBootStrapDumpOfWarehouse() throws Throwable
{
.run("alter database default set dbproperties
('hive.repl.ckpt.key'='', 'repl.last.id'='')");
try {
replica.load("", "`*`");
-} catch (SemanticException e) {
- assertEquals("REPL LOAD * is not supported", e.getMessage());
+ assertTrue(false);
Review comment:
assertEquals("REPL LOAD * is not supported", e.getMessage()); -
Do we not throw this error anymore
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 622375)
Time Spent: 4h 50m (was: 4h 40m)
> Fix the clean up of open repl created transactions
> --
>
> Key: HIVE-25246
> URL: https://issues.apache.org/jira/browse/HIVE-25246
> Project: Hive
> Issue Type: Improvement
>Reporter: Haymant Mangla
>Assignee: Haymant Mangla
>Priority: Major
> Labels: pull-request-available
> Time Spent: 4h 50m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-22626) Fix TestStatsReplicationScenariosACIDNoAutogather
[
https://issues.apache.org/jira/browse/HIVE-22626?focusedWorklogId=622367=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-622367
]
ASF GitHub Bot logged work on HIVE-22626:
-
Author: ASF GitHub Bot
Created on: 14/Jul/21 04:20
Start Date: 14/Jul/21 04:20
Worklog Time Spent: 10m
Work Description: ayushtkn commented on pull request #2452:
URL: https://github.com/apache/hive/pull/2452#issuecomment-879573879
Enables 3 tests:
**TestStatsReplicationScenariosMMNoAutogather:**
http://ci.hive.apache.org/job/hive-flaky-check/308/testReport/org.apache.hadoop.hive.ql.parse/
**TestStatsReplicationScenariosACIDNoAutogather:**
http://ci.hive.apache.org/job/hive-flaky-check/304/testReport/org.apache.hadoop.hive.ql.parse/
**TestStatsReplicationScenariosACID:**
http://ci.hive.apache.org/job/hive-flaky-check/305/testReport/org.apache.hadoop.hive.ql.parse/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 622367)
Time Spent: 1h (was: 50m)
> Fix TestStatsReplicationScenariosACIDNoAutogather
> -
>
> Key: HIVE-22626
> URL: https://issues.apache.org/jira/browse/HIVE-22626
> Project: Hive
> Issue Type: Sub-task
> Components: Test
>Reporter: Zoltan Haindrich
>Assignee: Ayush Saxena
>Priority: Major
> Labels: pull-request-available
> Attachments: qalogs.tgz
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> this test is running "alone" because but still; it sometimes runs more than
> 40m which results in a timeout
> a jira search reveals that was pretty common:
> https://issues.apache.org/jira/issues/?jql=text%20~%20%22TestStatsReplicationScenariosACIDNoAutogather%22%20order%20by%20updated%20desc
> from the hive logs:
> * it seems like after a few minutes this test starts there is an exception:
> {code}
> 2019-12-10T22:43:19,594 DEBUG [Finalizer] metastore.HiveMetaStoreClient:
> Unable to shutdown metastore client. Will try closing transport directly.
> org.apache.thrift.transport.TTransportException: java.net.SocketException:
> Socket closed
> at
> org.apache.thrift.transport.TIOStreamTransport.flush(TIOStreamTransport.java:161)
> ~[libthrift-0.9.3-1.jar:0.9.3-1]
> at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:73)
> ~[libthrift-0.9.3-1.jar:0.9.3-1]
> at
> org.apache.thrift.TServiceClient.sendBaseOneway(TServiceClient.java:66)
> ~[libthrift-0.9.3-1.jar:0.9.3-1]
> at
> com.facebook.fb303.FacebookService$Client.send_shutdown(FacebookService.java:436)
> ~[libfb303-0.9.3.jar:?]
> at
> com.facebook.fb303.FacebookService$Client.shutdown(FacebookService.java:430)
> ~[libfb303-0.9.3.jar:?]
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.close(HiveMetaStoreClient.java:776)
> [hive-standalone-metastore-common-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ~[?:1.8.0_102]
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> ~[?:1.8.0_102]
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[?:1.8.0_102]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_102]
> at
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:212)
> [hive-standalone-metastore-common-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at com.sun.proxy.$Proxy62.close(Unknown Source) [?:?]
> at org.apache.hadoop.hive.ql.metadata.Hive.close(Hive.java:542)
> [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.metadata.Hive.finalize(Hive.java:514)
> [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at java.lang.System$2.invokeFinalize(System.java:1270) [?:1.8.0_102]
> at java.lang.ref.Finalizer.runFinalizer(Finalizer.java:98)
> [?:1.8.0_102]
> at java.lang.ref.Finalizer.access$100(Finalizer.java:34) [?:1.8.0_102]
> at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:210)
> [?:1.8.0_102]
> Caused by: java.net.SocketException: Socket closed
> at
> java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:116)
> ~[?:1.8.0_102]
> at java.net.SocketOutputStream.write(SocketOutputStream.java:153)
> ~[?:1.8.0_102]
> at
>
[jira] [Work logged] (HIVE-25137) getAllWriteEventInfo should go through the HMS client instead of using RawStore directly
[
https://issues.apache.org/jira/browse/HIVE-25137?focusedWorklogId=622351=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-622351
]
ASF GitHub Bot logged work on HIVE-25137:
-
Author: ASF GitHub Bot
Created on: 14/Jul/21 03:14
Start Date: 14/Jul/21 03:14
Worklog Time Spent: 10m
Work Description: pkumarsinha commented on a change in pull request #2457:
URL: https://github.com/apache/hive/pull/2457#discussion_r669248010
##
File path:
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HMSHandler.java
##
@@ -10447,4 +10447,22 @@ public void drop_package(DropPackageRequest request)
throws MetaException {
endFunction("drop_package", ex == null, ex);
}
}
+
+ @Override
+ public List
get_all_write_event_info(GetAllWriteEventInfoRequest request)
+ throws MetaException {
+startFunction("get_all_write_event_info");
+Exception ex = null;
+try {
+ List writeEventInfoList =
+ getMS().getAllWriteEventInfo(request.getTxnId(),
request.getDbName(), request.getTableName());
+ return writeEventInfoList == null ? Collections.emptyList() :
writeEventInfoList;
+} catch (Exception e) {
+ LOG.error("Caught exception", e);
+ ex = e;
Review comment:
While existing method also has it, I wonder what do you get by this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 622351)
Time Spent: 20m (was: 10m)
> getAllWriteEventInfo should go through the HMS client instead of using
> RawStore directly
>
>
> Key: HIVE-25137
> URL: https://issues.apache.org/jira/browse/HIVE-25137
> Project: Hive
> Issue Type: Improvement
>Reporter: Pratyush Madhukar
>Assignee: Yu-Wen Lai
>Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> {code:java}
> private List getAllWriteEventInfo(Context withinContext)
> throws Exception {
> String contextDbName =
> StringUtils.normalizeIdentifier(withinContext.replScope.getDbName());
> RawStore rawStore =
> HiveMetaStore.HMSHandler.getMSForConf(withinContext.hiveConf);
> List writeEventInfoList
> = rawStore.getAllWriteEventInfo(eventMessage.getTxnId(),
> contextDbName, null);
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-24802) Show operation log at webui
[ https://issues.apache.org/jira/browse/HIVE-24802?focusedWorklogId=622334=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-622334 ] ASF GitHub Bot logged work on HIVE-24802: - Author: ASF GitHub Bot Created on: 14/Jul/21 01:46 Start Date: 14/Jul/21 01:46 Worklog Time Spent: 10m Work Description: dengzhhu653 opened a new pull request #1998: URL: https://github.com/apache/hive/pull/1998 ### What changes were proposed in this pull request? ### Why are the changes needed? Currently we provide getQueryLog in HiveStatement to fetch the operation log, and the operation log would be deleted on operation closing(delay for the canceled operation). Sometimes it's would be not easy for the user(jdbc) or administrators to deep into the details of the finished(failed) operation, so we present the operation log on webui and keep the operation log for some time for latter analysis. ### Does this PR introduce _any_ user-facing change? default is disabled, should set hive.server2.historic.operation.log.enabled = true to enable it. ### How was this patch tested? unit test / local machine -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 622334) Time Spent: 6h 20m (was: 6h 10m) > Show operation log at webui > --- > > Key: HIVE-24802 > URL: https://issues.apache.org/jira/browse/HIVE-24802 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Reporter: Zhihua Deng >Assignee: Zhihua Deng >Priority: Minor > Labels: pull-request-available > Attachments: operationlog.png > > Time Spent: 6h 20m > Remaining Estimate: 0h > > Currently we provide getQueryLog in HiveStatement to fetch the operation log, > and the operation log would be deleted on operation closing(delay for the > canceled operation). Sometimes it's would be not easy for the user(jdbc) or > administrators to deep into the details of the finished(failed) operation, so > we present the operation log on webui and keep the operation log for some > time for latter analysis. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-24802) Show operation log at webui
[ https://issues.apache.org/jira/browse/HIVE-24802?focusedWorklogId=622333=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-622333 ] ASF GitHub Bot logged work on HIVE-24802: - Author: ASF GitHub Bot Created on: 14/Jul/21 01:40 Start Date: 14/Jul/21 01:40 Worklog Time Spent: 10m Work Description: dengzhhu653 closed pull request #1998: URL: https://github.com/apache/hive/pull/1998 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 622333) Time Spent: 6h 10m (was: 6h) > Show operation log at webui > --- > > Key: HIVE-24802 > URL: https://issues.apache.org/jira/browse/HIVE-24802 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Reporter: Zhihua Deng >Assignee: Zhihua Deng >Priority: Minor > Labels: pull-request-available > Attachments: operationlog.png > > Time Spent: 6h 10m > Remaining Estimate: 0h > > Currently we provide getQueryLog in HiveStatement to fetch the operation log, > and the operation log would be deleted on operation closing(delay for the > canceled operation). Sometimes it's would be not easy for the user(jdbc) or > administrators to deep into the details of the finished(failed) operation, so > we present the operation log on webui and keep the operation log for some > time for latter analysis. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-23756) Added more constraints to the package.jdo file
[
https://issues.apache.org/jira/browse/HIVE-23756?focusedWorklogId=622323=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-622323
]
ASF GitHub Bot logged work on HIVE-23756:
-
Author: ASF GitHub Bot
Created on: 14/Jul/21 00:46
Start Date: 14/Jul/21 00:46
Worklog Time Spent: 10m
Work Description: belugabehr commented on pull request #2254:
URL: https://github.com/apache/hive/pull/2254#issuecomment-879497420
@kgyrtkirk This PR was approved and then went stale. Do you want to take
another pass at it? Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 622323)
Time Spent: 1h 20m (was: 1h 10m)
> Added more constraints to the package.jdo file
> --
>
> Key: HIVE-23756
> URL: https://issues.apache.org/jira/browse/HIVE-23756
> Project: Hive
> Issue Type: Bug
>Reporter: Ganesha Shreedhara
>Assignee: Steve Carlin
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-23756.1.patch
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> Drop table command fails intermittently with the following exception.
> {code:java}
> Caused by: java.sql.BatchUpdateException: Cannot delete or update a parent
> row: a foreign key constraint fails ("metastore"."COLUMNS_V2", CONSTRAINT
> "COLUMNS_V2_FK1" FOREIGN KEY ("CD_ID") REFERENCES "CDS" ("CD_ID")) App > at
> com.mysql.jdbc.PreparedStatement.executeBatchSerially(PreparedStatement.java:1815)at
> com.mysql.jdbc.PreparedStatement.executeBatch(PreparedStatement.java:1277)
> Appat
> org.datanucleus.store.rdbms.ParamLoggingPreparedStatement.executeBatch(ParamLoggingPreparedStatement.java:372)
> at
> org.datanucleus.store.rdbms.SQLController.processConnectionStatement(SQLController.java:628)
> at
> org.datanucleus.store.rdbms.SQLController.getStatementForUpdate(SQLController.java:207)
> at
> org.datanucleus.store.rdbms.SQLController.getStatementForUpdate(SQLController.java:179)
> at
> org.datanucleus.store.rdbms.scostore.JoinMapStore.clearInternal(JoinMapStore.java:901)
> ... 36 more
> Caused by:
> com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException:
> Cannot delete or update a parent row: a foreign key constraint fails
> ("metastore"."COLUMNS_V2", CONSTRAINT "COLUMNS_V2_FK1" FOREIGN KEY ("CD_ID")
> REFERENCES "CDS" ("CD_ID"))
> at sun.reflect.GeneratedConstructorAccessor121.newInstance(Unknown Source)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at com.mysql.jdbc.Util.handleNewInstance(Util.java:377)
> at com.mysql.jdbc.Util.getInstance(Util.java:360)
> at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:971)
> at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3887)
> at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3823){code}
> Although HIVE-19994 resolves this issue, the FK constraint name of COLUMNS_V2
> table specified in package.jdo file is not same as the FK constraint name
> used while creating COLUMNS_V2 table ([Ref|#L60]]).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-23756) Added more constraints to the package.jdo file
[
https://issues.apache.org/jira/browse/HIVE-23756?focusedWorklogId=622304=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-622304
]
ASF GitHub Bot logged work on HIVE-23756:
-
Author: ASF GitHub Bot
Created on: 14/Jul/21 00:08
Start Date: 14/Jul/21 00:08
Worklog Time Spent: 10m
Work Description: github-actions[bot] commented on pull request #2254:
URL: https://github.com/apache/hive/pull/2254#issuecomment-879483836
This pull request has been automatically marked as stale because it has not
had recent activity. It will be closed if no further activity occurs.
Feel free to reach out on the d...@hive.apache.org list if the patch is in
need of reviews.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 622304)
Time Spent: 1h 10m (was: 1h)
> Added more constraints to the package.jdo file
> --
>
> Key: HIVE-23756
> URL: https://issues.apache.org/jira/browse/HIVE-23756
> Project: Hive
> Issue Type: Bug
>Reporter: Ganesha Shreedhara
>Assignee: Steve Carlin
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-23756.1.patch
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> Drop table command fails intermittently with the following exception.
> {code:java}
> Caused by: java.sql.BatchUpdateException: Cannot delete or update a parent
> row: a foreign key constraint fails ("metastore"."COLUMNS_V2", CONSTRAINT
> "COLUMNS_V2_FK1" FOREIGN KEY ("CD_ID") REFERENCES "CDS" ("CD_ID")) App > at
> com.mysql.jdbc.PreparedStatement.executeBatchSerially(PreparedStatement.java:1815)at
> com.mysql.jdbc.PreparedStatement.executeBatch(PreparedStatement.java:1277)
> Appat
> org.datanucleus.store.rdbms.ParamLoggingPreparedStatement.executeBatch(ParamLoggingPreparedStatement.java:372)
> at
> org.datanucleus.store.rdbms.SQLController.processConnectionStatement(SQLController.java:628)
> at
> org.datanucleus.store.rdbms.SQLController.getStatementForUpdate(SQLController.java:207)
> at
> org.datanucleus.store.rdbms.SQLController.getStatementForUpdate(SQLController.java:179)
> at
> org.datanucleus.store.rdbms.scostore.JoinMapStore.clearInternal(JoinMapStore.java:901)
> ... 36 more
> Caused by:
> com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException:
> Cannot delete or update a parent row: a foreign key constraint fails
> ("metastore"."COLUMNS_V2", CONSTRAINT "COLUMNS_V2_FK1" FOREIGN KEY ("CD_ID")
> REFERENCES "CDS" ("CD_ID"))
> at sun.reflect.GeneratedConstructorAccessor121.newInstance(Unknown Source)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at com.mysql.jdbc.Util.handleNewInstance(Util.java:377)
> at com.mysql.jdbc.Util.getInstance(Util.java:360)
> at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:971)
> at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3887)
> at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3823){code}
> Although HIVE-19994 resolves this issue, the FK constraint name of COLUMNS_V2
> table specified in package.jdo file is not same as the FK constraint name
> used while creating COLUMNS_V2 table ([Ref|#L60]]).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25324) Add option to disable PartitionManagementTask
[
https://issues.apache.org/jira/browse/HIVE-25324?focusedWorklogId=622267=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-622267
]
ASF GitHub Bot logged work on HIVE-25324:
-
Author: ASF GitHub Bot
Created on: 13/Jul/21 22:15
Start Date: 13/Jul/21 22:15
Worklog Time Spent: 10m
Work Description: rbalamohan commented on pull request #2470:
URL: https://github.com/apache/hive/pull/2470#issuecomment-879440550
Yes, that is an option as well.
Given https://issues.apache.org/jira/browse/HIVE-25039 (i.e disabling
discover.partitions by default), we can remove PartitionManagementTask from
"TASK_THREADS_REMOTE_ONLY" as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 622267)
Time Spent: 0.5h (was: 20m)
> Add option to disable PartitionManagementTask
> -
>
> Key: HIVE-25324
> URL: https://issues.apache.org/jira/browse/HIVE-25324
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
>Reporter: Rajesh Balamohan
>Priority: Major
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> When large number of tables (e.g 2000) and databases are present,
> PartitionManagementTask scans all tables and partitions causing pressure on
> HMS.
> Currently there is no way to disable PartitionManagementTask as well. Round
> about option is to provide pattern via
> "metastore.partition.management.database.pattern /
> metastore.partition.management.table.pattern".
> It will be good to provide an option to disable it completely.{color:#807d6e}
> {color}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25317) Relocate dependencies in shaded hive-exec module
[ https://issues.apache.org/jira/browse/HIVE-25317?focusedWorklogId=622256=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-622256 ] ASF GitHub Bot logged work on HIVE-25317: - Author: ASF GitHub Bot Created on: 13/Jul/21 21:33 Start Date: 13/Jul/21 21:33 Worklog Time Spent: 10m Work Description: sunchao commented on pull request #2459: URL: https://github.com/apache/hive/pull/2459#issuecomment-879417843 @viirya there are still more errors: ``` java.lang.NoClassDefFoundError: org/apache/hive/org/apache/commons/logging/LogFactory at org.apache.hadoop.hive.conf.valcoersion.JavaIOTmpdirVariableCoercion.(JavaIOTmpdirVariableCoercion.java:35) at org.apache.hadoop.hive.conf.SystemVariables.(SystemVariables.java:37) at org.apache.hadoop.hive.conf.HiveConf$ConfVars.(HiveConf.java:3445) at org.apache.hadoop.hive.conf.HiveConf$ConfVars.(HiveConf.java:3409) at org.apache.hadoop.hive.conf.HiveConf$ConfVars.(HiveConf.java:436) at org.apache.hadoop.hive.druid.serde.DruidSerDe.initialize(DruidSerDe.java:113) at org.apache.hadoop.hive.serde2.AbstractSerDe.initialize(AbstractSerDe.java:54) at org.apache.hadoop.hive.serde2.SerDeUtils.initializeSerDe(SerDeUtils.java:533) at org.apache.hadoop.hive.druid.TestDruidSerDe.testDruidDeserializer(TestDruidSerDe.java:510) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) ``` ``` Caused by: java.lang.NoClassDefFoundError: org/apache/hive/org/apache/commons/codec/language/Soundex at org.apache.hadoop.hive.ql.udf.generic.GenericUDFSoundex.(GenericUDFSoundex.java:49) ... 29 more ``` ``` Caused by: java.lang.NoClassDefFoundError: org/apache/hive/org/apache/commons/codec/binary/Base64 at org.apache.hadoop.hive.serde2.lazy.LazyBinary.decodeIfNeeded(LazyBinary.java:58) at org.apache.hadoop.hive.serde2.lazy.LazyBinary.init(LazyBinary.java:50) at org.apache.hadoop.hive.serde2.lazy.LazyStruct.uncheckedGetField(LazyStruct.java:226) at org.apache.hadoop.hive.serde2.lazy.LazyStruct.getField(LazyStruct.java:202) at org.apache.hadoop.hive.serde2.lazy.objectinspector.LazySimpleStructObjectInspector.getStructFieldData(LazySimpleStructObjectInspector.java:128) at org.apache.hadoop.hive.serde2.objectinspector.UnionStructObjectInspector.getStructFieldData(UnionStructObjectInspector.java:157) at org.apache.hadoop.hive.ql.exec.ExprNodeColumnEvaluator._evaluate(ExprNodeColumnEvaluator.java:95) at org.apache.hadoop.hive.ql.exec.ExprNodeEvaluator.evaluate(ExprNodeEvaluator.java:80) at org.apache.hadoop.hive.ql.exec.ExprNodeEvaluator.evaluate(ExprNodeEvaluator.java:68) at org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:88) at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:897) at org.apache.hadoop.hive.ql.exec.TableScanOperator.process(TableScanOperator.java:130) at org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:438) at org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:430) ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 622256) Time Spent: 50m (was: 40m) > Relocate dependencies in shaded hive-exec module > > > Key: HIVE-25317 > URL: https://issues.apache.org/jira/browse/HIVE-25317 > Project: Hive > Issue Type: Improvement >Affects Versions: 2.3.8 >Reporter: L. C. Hsieh >Priority: Major > Labels: pull-request-available > Time Spent: 50m > Remaining Estimate: 0h > > When we want to use shaded version of hive-exec (i.e., w/o classifier), more > dependencies conflict with Spark. We need to relocate these dependencies too. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-25317) Relocate dependencies in shaded hive-exec module
[ https://issues.apache.org/jira/browse/HIVE-25317?focusedWorklogId=622212=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-622212 ] ASF GitHub Bot logged work on HIVE-25317: - Author: ASF GitHub Bot Created on: 13/Jul/21 20:19 Start Date: 13/Jul/21 20:19 Worklog Time Spent: 10m Work Description: viirya commented on pull request #2459: URL: https://github.com/apache/hive/pull/2459#issuecomment-879374726 Thanks @sunchao. Seems caused by relocating avro, protobuf. Reverted the two relocations and see what's going on. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 622212) Time Spent: 40m (was: 0.5h) > Relocate dependencies in shaded hive-exec module > > > Key: HIVE-25317 > URL: https://issues.apache.org/jira/browse/HIVE-25317 > Project: Hive > Issue Type: Improvement >Affects Versions: 2.3.8 >Reporter: L. C. Hsieh >Priority: Major > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > When we want to use shaded version of hive-exec (i.e., w/o classifier), more > dependencies conflict with Spark. We need to relocate these dependencies too. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-25320) Purge hive.optimize.sort.dynamic.partition
[
https://issues.apache.org/jira/browse/HIVE-25320?focusedWorklogId=622203=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-622203
]
ASF GitHub Bot logged work on HIVE-25320:
-
Author: ASF GitHub Bot
Created on: 13/Jul/21 20:08
Start Date: 13/Jul/21 20:08
Worklog Time Spent: 10m
Work Description: sneakyburro commented on pull request #2466:
URL: https://github.com/apache/hive/pull/2466#issuecomment-879367271
> Can you please change PR (and JIRA) title to:
>
> > HIVE-25320: Purge hive.optimize.sort.dynamic.partition
>
> Please?
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 622203)
Time Spent: 1h 10m (was: 1h)
> Purge hive.optimize.sort.dynamic.partition
> --
>
> Key: HIVE-25320
> URL: https://issues.apache.org/jira/browse/HIVE-25320
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Reporter: David Mollitor
>Assignee: Alex Sun
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> {{hive.optimize.sort.dynamic.partition}} has been replace with
> {{hive.optimize.sort.dynamic.partition.threshold}} . It has been marked as
> "deprecated", but it's actually totally defunct in the current code base.
> Deprecation would allow an admin to continue to use it (maybe as an alias to
> {{threadshold}} = 0/-1), but that is not the case here.
>
> Remove all references to "hive.optimize.sort.dynamic.partition" in the q
> tests and remove {{HIVEOPTSORTDYNAMICPARTITION}} all together.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-25320) Purge hive.optimize.sort.dynamic.partition
[
https://issues.apache.org/jira/browse/HIVE-25320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Sun updated HIVE-25320:
Summary: Purge hive.optimize.sort.dynamic.partition (was: Remove
hive.optimize.sort.dynamic.partition)
> Purge hive.optimize.sort.dynamic.partition
> --
>
> Key: HIVE-25320
> URL: https://issues.apache.org/jira/browse/HIVE-25320
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Reporter: David Mollitor
>Assignee: Alex Sun
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 1h
> Remaining Estimate: 0h
>
> {{hive.optimize.sort.dynamic.partition}} has been replace with
> {{hive.optimize.sort.dynamic.partition.threshold}} . It has been marked as
> "deprecated", but it's actually totally defunct in the current code base.
> Deprecation would allow an admin to continue to use it (maybe as an alias to
> {{threadshold}} = 0/-1), but that is not the case here.
>
> Remove all references to "hive.optimize.sort.dynamic.partition" in the q
> tests and remove {{HIVEOPTSORTDYNAMICPARTITION}} all together.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HIVE-24522) Support DATE and VARCHAR column type in Kudu SerDe
[ https://issues.apache.org/jira/browse/HIVE-24522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Grant Henke reassigned HIVE-24522: -- Assignee: (was: Grant Henke) > Support DATE and VARCHAR column type in Kudu SerDe > -- > > Key: HIVE-24522 > URL: https://issues.apache.org/jira/browse/HIVE-24522 > Project: Hive > Issue Type: Sub-task > Components: Hive >Affects Versions: 2.0.0 >Reporter: Greg Solovyev >Priority: Major > Labels: pull-request-available > Time Spent: 1h > Remaining Estimate: 0h > > DATE type was added to Kudu in Kudu 1.12 (KUDU-2632) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-25320) Remove hive.optimize.sort.dynamic.partition
[
https://issues.apache.org/jira/browse/HIVE-25320?focusedWorklogId=622198=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-622198
]
ASF GitHub Bot logged work on HIVE-25320:
-
Author: ASF GitHub Bot
Created on: 13/Jul/21 20:01
Start Date: 13/Jul/21 20:01
Worklog Time Spent: 10m
Work Description: belugabehr edited a comment on pull request #2466:
URL: https://github.com/apache/hive/pull/2466#issuecomment-879362801
Can you please change PR (and JIRA) title to:
> HIVE-25320: Purge hive.optimize.sort.dynamic.partition
Please?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 622198)
Time Spent: 1h (was: 50m)
> Remove hive.optimize.sort.dynamic.partition
> ---
>
> Key: HIVE-25320
> URL: https://issues.apache.org/jira/browse/HIVE-25320
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Reporter: David Mollitor
>Assignee: Alex Sun
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 1h
> Remaining Estimate: 0h
>
> {{hive.optimize.sort.dynamic.partition}} has been replace with
> {{hive.optimize.sort.dynamic.partition.threshold}} . It has been marked as
> "deprecated", but it's actually totally defunct in the current code base.
> Deprecation would allow an admin to continue to use it (maybe as an alias to
> {{threadshold}} = 0/-1), but that is not the case here.
>
> Remove all references to "hive.optimize.sort.dynamic.partition" in the q
> tests and remove {{HIVEOPTSORTDYNAMICPARTITION}} all together.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25320) Remove hive.optimize.sort.dynamic.partition
[
https://issues.apache.org/jira/browse/HIVE-25320?focusedWorklogId=622197=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-622197
]
ASF GitHub Bot logged work on HIVE-25320:
-
Author: ASF GitHub Bot
Created on: 13/Jul/21 20:01
Start Date: 13/Jul/21 20:01
Worklog Time Spent: 10m
Work Description: belugabehr commented on pull request #2466:
URL: https://github.com/apache/hive/pull/2466#issuecomment-879362801
Can you please change PR (and JIRA) title to:
bq. HIVE-25320: Purge hive.optimize.sort.dynamic.partition
Please?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 622197)
Time Spent: 50m (was: 40m)
> Remove hive.optimize.sort.dynamic.partition
> ---
>
> Key: HIVE-25320
> URL: https://issues.apache.org/jira/browse/HIVE-25320
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Reporter: David Mollitor
>Assignee: Alex Sun
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 50m
> Remaining Estimate: 0h
>
> {{hive.optimize.sort.dynamic.partition}} has been replace with
> {{hive.optimize.sort.dynamic.partition.threshold}} . It has been marked as
> "deprecated", but it's actually totally defunct in the current code base.
> Deprecation would allow an admin to continue to use it (maybe as an alias to
> {{threadshold}} = 0/-1), but that is not the case here.
>
> Remove all references to "hive.optimize.sort.dynamic.partition" in the q
> tests and remove {{HIVEOPTSORTDYNAMICPARTITION}} all together.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-24546) Avoid unwanted cloud storage call during dynamic partition load
[
https://issues.apache.org/jira/browse/HIVE-24546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17380090#comment-17380090
]
Steve Loughran commented on HIVE-24546:
---
I'd recommend
* skip the dest path check
* call mkdirs() without any probe, and if it returns false, check that there's
a dir at the far end
if (!s.mkdirs(dpStagingPath) && !fs.isDir(dpStagingPath) {
throw new IOException("Failed to create dir " + dpStagingPath");
}
This relies on mkdir() returning false if the dir is already there
> Avoid unwanted cloud storage call during dynamic partition load
> ---
>
> Key: HIVE-24546
> URL: https://issues.apache.org/jira/browse/HIVE-24546
> Project: Hive
> Issue Type: Improvement
>Reporter: Rajesh Balamohan
>Priority: Major
> Attachments: simple_test.sql
>
>
> {code:java}
> private void createDpDirCheckSrc(final Path dpStagingPath, final Path
> dpFinalPath) throws IOException {
> if (!fs.exists(dpStagingPath) && !fs.exists(dpFinalPath)) {
> fs.mkdirs(dpStagingPath);
> // move task will create dp final path
> if (reporter != null) {
> reporter.incrCounter(counterGroup,
> Operator.HIVE_COUNTER_CREATED_DYNAMIC_PARTITIONS, 1);
> }
> }
> }
> {code}
>
>
> {noformat}
> at org.apache.hadoop.fs.s3a.Invoker.retryUntranslated(Invoker.java:370)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.listObjects(S3AFileSystem.java:1960)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.s3GetFileStatus(S3AFileSystem.java:3164)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.innerGetFileStatus(S3AFileSystem.java:3031)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.java:2899)
> at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1723)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.exists(S3AFileSystem.java:4157)
> at
> org.apache.hadoop.hive.ql.exec.FileSinkOperator.createDpDir(FileSinkOperator.java:948)
> at
> org.apache.hadoop.hive.ql.exec.FileSinkOperator.updateDPCounters(FileSinkOperator.java:916)
> at
> org.apache.hadoop.hive.ql.exec.FileSinkOperator.createBucketForFileIdx(FileSinkOperator.java:849)
> at
> org.apache.hadoop.hive.ql.exec.FileSinkOperator.createBucketFiles(FileSinkOperator.java:814)
> at
> org.apache.hadoop.hive.ql.exec.FileSinkOperator.createNewPaths(FileSinkOperator.java:1200)
> at
> org.apache.hadoop.hive.ql.exec.FileSinkOperator.getDynOutPaths(FileSinkOperator.java:1324)
> at
> org.apache.hadoop.hive.ql.exec.FileSinkOperator.process(FileSinkOperator.java:1036)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorFileSinkOperator.process(VectorFileSinkOperator.java:111)
> at
> org.apache.hadoop.hive.ql.exec.Operator.vectorForward(Operator.java:969)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25317) Relocate dependencies in shaded hive-exec module
[ https://issues.apache.org/jira/browse/HIVE-25317?focusedWorklogId=622117=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-622117 ] ASF GitHub Bot logged work on HIVE-25317: - Author: ASF GitHub Bot Created on: 13/Jul/21 17:28 Start Date: 13/Jul/21 17:28 Worklog Time Spent: 10m Work Description: sunchao commented on pull request #2459: URL: https://github.com/apache/hive/pull/2459#issuecomment-879269084 Thanks @viirya for the update! there are still some failures that are related, such as 1. ``` java.lang.NoSuchMethodError: org.apache.hadoop.hive.llap.daemon.rpc.LlapDaemonProtocolProtos$LlapDaemonProtocol.newReflectiveBlockingService(Lorg/apache/hadoop/hive/llap/daemon/rpc/LlapDaemonProtocolProtos$LlapDaemonProtocol$BlockingInterface;)Lcom/google/protobuf/BlockingService; at org.apache.hadoop.hive.llap.daemon.impl.LlapProtocolServerImpl.serviceStart(LlapProtocolServerImpl.java:142) at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193) at org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120) at org.apache.hadoop.hive.llap.daemon.impl.LlapDaemon.serviceStart(LlapDaemon.java:405) at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193) at org.apache.hadoop.hive.llap.daemon.MiniLlapCluster.serviceStart(MiniLlapCluster.java:177) at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193) at org.apache.hadoop.hive.llap.LlapItUtils.startAndGetMiniLlapCluster(LlapItUtils.java:78) at org.apache.hadoop.hive.ql.QTestUtil.setupMiniCluster(QTestUtil.java:656) at org.apache.hadoop.hive.ql.QTestUtil.(QTestUtil.java:584) at org.apache.hadoop.hive.cli.control.CoreCliDriver$1.invokeInternal(CoreCliDriver.java:65) at org.apache.hadoop.hive.cli.control.CoreCliDriver$1.invokeInternal(CoreCliDriver.java:60) at org.apache.hadoop.hive.util.ElapsedTimeLoggingWrapper.invoke(ElapsedTimeLoggingWrapper.java:33) at org.apache.hadoop.hive.cli.control.CoreCliDriver.beforeClass(CoreCliDriver.java:67) at org.apache.hadoop.hive.cli.control.CliAdapter$1$1.evaluate(CliAdapter.java:71) at org.junit.rules.RunRules.evaluate(RunRules.java:20) at org.junit.runners.ParentRunner.run(ParentRunner.java:309) at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365) at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273) at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238) at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:377) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:138) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:465) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:451) ``` 2. ``` java.lang.NoSuchMethodError: org.apache.hadoop.hive.serde2.avro.AvroSerDe.getSchemaFromCols(Ljava/util/Properties;Ljava/util/List;Ljava/util/List;Ljava/lang/String;)Lorg/apache/avro/Schema; at org.apache.hive.hcatalog.mapreduce.SpecialCases.addSpecialCasesParametersToOutputJobProperties(SpecialCases.java:125) at org.apache.hive.hcatalog.mapreduce.FosterStorageHandler.configureOutputJobProperties(FosterStorageHandler.java:217) at org.apache.hive.hcatalog.common.HCatUtil.configureOutputStorageHandler(HCatUtil.java:498) at org.apache.hive.hcatalog.mapreduce.HCatOutputFormat.setOutput(HCatOutputFormat.java:198) at org.apache.hive.hcatalog.mapreduce.HCatOutputFormat.setOutput(HCatOutputFormat.java:70) at org.apache.hive.hcatalog.pig.HCatStorer.setStoreLocation(HCatStorer.java:191) at org.apache.pig.newplan.logical.visitor.InputOutputFileValidatorVisitor.visit(InputOutputFileValidatorVisitor.java:57) at org.apache.pig.newplan.logical.relational.LOStore.accept(LOStore.java:66) at org.apache.pig.newplan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:64) at org.apache.pig.newplan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:66) at org.apache.pig.newplan.DepthFirstWalker.walk(DepthFirstWalker.java:53) at org.apache.pig.newplan.PlanVisitor.visit(PlanVisitor.java:52) at org.apache.pig.newplan.logical.relational.LogicalPlan.validate(LogicalPlan.java:212) at org.apache.pig.PigServer$Graph.compile(PigServer.java:1851) at org.apache.pig.PigServer$Graph.access$300(PigServer.java:1527) at org.apache.pig.PigServer.execute(PigServer.java:1440) at
[jira] [Commented] (HIVE-25323) Fix TestVectorCastStatement
[ https://issues.apache.org/jira/browse/HIVE-25323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17380025#comment-17380025 ] Adesh Kumar Rao commented on HIVE-25323: Sure [~klcopp], I will take a look at it. > Fix TestVectorCastStatement > --- > > Key: HIVE-25323 > URL: https://issues.apache.org/jira/browse/HIVE-25323 > Project: Hive > Issue Type: Task >Reporter: Karen Coppage >Assignee: Adesh Kumar Rao >Priority: Major > > org.apache.hadoop.hive.ql.exec.vector.expressions.TestVectorCastStatement > tests were timing out after 5 hours. > [http://ci.hive.apache.org/job/hive-flaky-check/307/] > First failure: > [http://ci.hive.apache.org/blue/organizations/jenkins/hive-precommit/detail/master/749/pipeline/242] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-25323) Fix TestVectorCastStatement
[ https://issues.apache.org/jira/browse/HIVE-25323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adesh Kumar Rao reassigned HIVE-25323: -- Assignee: Adesh Kumar Rao > Fix TestVectorCastStatement > --- > > Key: HIVE-25323 > URL: https://issues.apache.org/jira/browse/HIVE-25323 > Project: Hive > Issue Type: Task >Reporter: Karen Coppage >Assignee: Adesh Kumar Rao >Priority: Major > > org.apache.hadoop.hive.ql.exec.vector.expressions.TestVectorCastStatement > tests were timing out after 5 hours. > [http://ci.hive.apache.org/job/hive-flaky-check/307/] > First failure: > [http://ci.hive.apache.org/blue/organizations/jenkins/hive-precommit/detail/master/749/pipeline/242] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-25317) Relocate dependencies in shaded hive-exec module
[ https://issues.apache.org/jira/browse/HIVE-25317?focusedWorklogId=622097=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-622097 ] ASF GitHub Bot logged work on HIVE-25317: - Author: ASF GitHub Bot Created on: 13/Jul/21 16:29 Start Date: 13/Jul/21 16:29 Worklog Time Spent: 10m Work Description: viirya commented on pull request #2459: URL: https://github.com/apache/hive/pull/2459#issuecomment-879231076 cc @sunchao Most test failures are passed now. Please check if current CI is okay? Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 622097) Time Spent: 20m (was: 10m) > Relocate dependencies in shaded hive-exec module > > > Key: HIVE-25317 > URL: https://issues.apache.org/jira/browse/HIVE-25317 > Project: Hive > Issue Type: Improvement >Affects Versions: 2.3.8 >Reporter: L. C. Hsieh >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > When we want to use shaded version of hive-exec (i.e., w/o classifier), more > dependencies conflict with Spark. We need to relocate these dependencies too. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-25324) Add option to disable PartitionManagementTask
[
https://issues.apache.org/jira/browse/HIVE-25324?focusedWorklogId=622058=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-622058
]
ASF GitHub Bot logged work on HIVE-25324:
-
Author: ASF GitHub Bot
Created on: 13/Jul/21 15:34
Start Date: 13/Jul/21 15:34
Worklog Time Spent: 10m
Work Description: prasanthj commented on pull request #2470:
URL: https://github.com/apache/hive/pull/2470#issuecomment-879191568
Removing PartitionManagementTask class from metastore.task.threads.remote
config provides the same effect (disabling)?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 622058)
Time Spent: 20m (was: 10m)
> Add option to disable PartitionManagementTask
> -
>
> Key: HIVE-25324
> URL: https://issues.apache.org/jira/browse/HIVE-25324
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
>Reporter: Rajesh Balamohan
>Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> When large number of tables (e.g 2000) and databases are present,
> PartitionManagementTask scans all tables and partitions causing pressure on
> HMS.
> Currently there is no way to disable PartitionManagementTask as well. Round
> about option is to provide pattern via
> "metastore.partition.management.database.pattern /
> metastore.partition.management.table.pattern".
> It will be good to provide an option to disable it completely.{color:#807d6e}
> {color}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-21489) EXPLAIN command throws ClassCastException in Hive
[ https://issues.apache.org/jira/browse/HIVE-21489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17379963#comment-17379963 ] Panagiotis Garefalakis commented on HIVE-21489: --- Resolved via https://github.com/apache/hive/pull/2373 Thanks [~rameshkumar] for the patch! > EXPLAIN command throws ClassCastException in Hive > - > > Key: HIVE-21489 > URL: https://issues.apache.org/jira/browse/HIVE-21489 > Project: Hive > Issue Type: Bug >Affects Versions: 2.3.4, 3.1.2 >Reporter: Ping Lu >Assignee: Ramesh Kumar Thangarajan >Priority: Major > Labels: pull-request-available > Fix For: 3.1.3, 4.0.0 > > Attachments: HIVE-21489.1.patch, HIVE-21489.2.patch > > Time Spent: 1h 20m > Remaining Estimate: 0h > > I'm trying to run commands like explain select * from src in hive-2.3.4,but > it falls with the ClassCastException: > org.apache.hadoop.hive.ql.parse.ExplainSemanticAnalyzer cannot be cast to > org.apache.hadoop.hive.ql.parse.SemanticAnalyzer > Steps to reproduce: > 1）hive.execution.engine is the default value mr > 2）hive.security.authorization.enabled is set to true, and > hive.security.authorization.manager is set to > org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider > 3）start hivecli to run command：explain select * from src > I debug the code and find the issue HIVE-18778 causing the above > ClassCastException.If I set hive.in.test to true，the explain command can be > successfully executed。 > Now,I have one question,due to hive.in.test cann't be modified at runtime.how > to run explain command with using default authorization in hive-2.3.4, -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-21489) EXPLAIN command throws ClassCastException in Hive
[ https://issues.apache.org/jira/browse/HIVE-21489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Panagiotis Garefalakis updated HIVE-21489: -- Resolution: Fixed Status: Resolved (was: Patch Available) > EXPLAIN command throws ClassCastException in Hive > - > > Key: HIVE-21489 > URL: https://issues.apache.org/jira/browse/HIVE-21489 > Project: Hive > Issue Type: Bug >Affects Versions: 2.3.4, 3.1.2 >Reporter: Ping Lu >Assignee: Ramesh Kumar Thangarajan >Priority: Major > Labels: pull-request-available > Fix For: 3.1.3, 4.0.0 > > Attachments: HIVE-21489.1.patch, HIVE-21489.2.patch > > Time Spent: 1h 20m > Remaining Estimate: 0h > > I'm trying to run commands like explain select * from src in hive-2.3.4,but > it falls with the ClassCastException: > org.apache.hadoop.hive.ql.parse.ExplainSemanticAnalyzer cannot be cast to > org.apache.hadoop.hive.ql.parse.SemanticAnalyzer > Steps to reproduce: > 1）hive.execution.engine is the default value mr > 2）hive.security.authorization.enabled is set to true, and > hive.security.authorization.manager is set to > org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider > 3）start hivecli to run command：explain select * from src > I debug the code and find the issue HIVE-18778 causing the above > ClassCastException.If I set hive.in.test to true，the explain command can be > successfully executed。 > Now,I have one question,due to hive.in.test cann't be modified at runtime.how > to run explain command with using default authorization in hive-2.3.4, -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-21489) EXPLAIN command throws ClassCastException in Hive
[ https://issues.apache.org/jira/browse/HIVE-21489?focusedWorklogId=622048=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-622048 ] ASF GitHub Bot logged work on HIVE-21489: - Author: ASF GitHub Bot Created on: 13/Jul/21 15:19 Start Date: 13/Jul/21 15:19 Worklog Time Spent: 10m Work Description: pgaref merged pull request #2373: URL: https://github.com/apache/hive/pull/2373 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 622048) Time Spent: 1h 20m (was: 1h 10m) > EXPLAIN command throws ClassCastException in Hive > - > > Key: HIVE-21489 > URL: https://issues.apache.org/jira/browse/HIVE-21489 > Project: Hive > Issue Type: Bug >Affects Versions: 2.3.4, 3.1.2 >Reporter: Ping Lu >Assignee: Ramesh Kumar Thangarajan >Priority: Major > Labels: pull-request-available > Fix For: 3.1.3, 4.0.0 > > Attachments: HIVE-21489.1.patch, HIVE-21489.2.patch > > Time Spent: 1h 20m > Remaining Estimate: 0h > > I'm trying to run commands like explain select * from src in hive-2.3.4,but > it falls with the ClassCastException: > org.apache.hadoop.hive.ql.parse.ExplainSemanticAnalyzer cannot be cast to > org.apache.hadoop.hive.ql.parse.SemanticAnalyzer > Steps to reproduce: > 1）hive.execution.engine is the default value mr > 2）hive.security.authorization.enabled is set to true, and > hive.security.authorization.manager is set to > org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider > 3）start hivecli to run command：explain select * from src > I debug the code and find the issue HIVE-18778 causing the above > ClassCastException.If I set hive.in.test to true，the explain command can be > successfully executed。 > Now,I have one question,due to hive.in.test cann't be modified at runtime.how > to run explain command with using default authorization in hive-2.3.4, -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-21489) EXPLAIN command throws ClassCastException in Hive
[ https://issues.apache.org/jira/browse/HIVE-21489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Panagiotis Garefalakis updated HIVE-21489: -- Affects Version/s: 3.1.2 > EXPLAIN command throws ClassCastException in Hive > - > > Key: HIVE-21489 > URL: https://issues.apache.org/jira/browse/HIVE-21489 > Project: Hive > Issue Type: Bug >Affects Versions: 2.3.4, 3.1.2 >Reporter: Ping Lu >Assignee: Ramesh Kumar Thangarajan >Priority: Major > Labels: pull-request-available > Attachments: HIVE-21489.1.patch, HIVE-21489.2.patch > > Time Spent: 1h 10m > Remaining Estimate: 0h > > I'm trying to run commands like explain select * from src in hive-2.3.4,but > it falls with the ClassCastException: > org.apache.hadoop.hive.ql.parse.ExplainSemanticAnalyzer cannot be cast to > org.apache.hadoop.hive.ql.parse.SemanticAnalyzer > Steps to reproduce: > 1）hive.execution.engine is the default value mr > 2）hive.security.authorization.enabled is set to true, and > hive.security.authorization.manager is set to > org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider > 3）start hivecli to run command：explain select * from src > I debug the code and find the issue HIVE-18778 causing the above > ClassCastException.If I set hive.in.test to true，the explain command can be > successfully executed。 > Now,I have one question,due to hive.in.test cann't be modified at runtime.how > to run explain command with using default authorization in hive-2.3.4, -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-21489) EXPLAIN command throws ClassCastException in Hive
[ https://issues.apache.org/jira/browse/HIVE-21489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Panagiotis Garefalakis updated HIVE-21489: -- Fix Version/s: 4.0.0 3.1.3 > EXPLAIN command throws ClassCastException in Hive > - > > Key: HIVE-21489 > URL: https://issues.apache.org/jira/browse/HIVE-21489 > Project: Hive > Issue Type: Bug >Affects Versions: 2.3.4, 3.1.2 >Reporter: Ping Lu >Assignee: Ramesh Kumar Thangarajan >Priority: Major > Labels: pull-request-available > Fix For: 3.1.3, 4.0.0 > > Attachments: HIVE-21489.1.patch, HIVE-21489.2.patch > > Time Spent: 1h 10m > Remaining Estimate: 0h > > I'm trying to run commands like explain select * from src in hive-2.3.4,but > it falls with the ClassCastException: > org.apache.hadoop.hive.ql.parse.ExplainSemanticAnalyzer cannot be cast to > org.apache.hadoop.hive.ql.parse.SemanticAnalyzer > Steps to reproduce: > 1）hive.execution.engine is the default value mr > 2）hive.security.authorization.enabled is set to true, and > hive.security.authorization.manager is set to > org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider > 3）start hivecli to run command：explain select * from src > I debug the code and find the issue HIVE-18778 causing the above > ClassCastException.If I set hive.in.test to true，the explain command can be > successfully executed。 > Now,I have one question,due to hive.in.test cann't be modified at runtime.how > to run explain command with using default authorization in hive-2.3.4, -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-25323) Fix TestVectorCastStatement
[ https://issues.apache.org/jira/browse/HIVE-25323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17379948#comment-17379948 ] Karen Coppage commented on HIVE-25323: -- [~adeshrao] Not sure about this but you seem to have committed the only change related to (timestamp) casting just before this test started failing. Would you mind taking a look? [http://ci.hive.apache.org/job/hive-flaky-check/307/] > Fix TestVectorCastStatement > --- > > Key: HIVE-25323 > URL: https://issues.apache.org/jira/browse/HIVE-25323 > Project: Hive > Issue Type: Task >Reporter: Karen Coppage >Priority: Major > > org.apache.hadoop.hive.ql.exec.vector.expressions.TestVectorCastStatement > tests were timing out after 5 hours. > [http://ci.hive.apache.org/job/hive-flaky-check/307/] > First failure: > [http://ci.hive.apache.org/blue/organizations/jenkins/hive-precommit/detail/master/749/pipeline/242] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25323) Fix TestVectorCastStatement
[ https://issues.apache.org/jira/browse/HIVE-25323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karen Coppage updated HIVE-25323: - Description: org.apache.hadoop.hive.ql.exec.vector.expressions.TestVectorCastStatement tests were timing out after 5 hours. [http://ci.hive.apache.org/job/hive-flaky-check/307/] First failure: [http://ci.hive.apache.org/blue/organizations/jenkins/hive-precommit/detail/master/749/pipeline/242] was: org.apache.hadoop.hive.ql.exec.vector.expressions.TestVectorCastStatement tests were timing out after 5 hours. [http://ci.hive.apache.org/job/hive-flaky-check/307/] http://ci.hive.apache.org/blue/organizations/jenkins/hive-precommit/detail/master/749/pipeline/242 > Fix TestVectorCastStatement > --- > > Key: HIVE-25323 > URL: https://issues.apache.org/jira/browse/HIVE-25323 > Project: Hive > Issue Type: Task >Reporter: Karen Coppage >Priority: Major > > org.apache.hadoop.hive.ql.exec.vector.expressions.TestVectorCastStatement > tests were timing out after 5 hours. > [http://ci.hive.apache.org/job/hive-flaky-check/307/] > First failure: > [http://ci.hive.apache.org/blue/organizations/jenkins/hive-precommit/detail/master/749/pipeline/242] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-25320) Remove hive.optimize.sort.dynamic.partition
[
https://issues.apache.org/jira/browse/HIVE-25320?focusedWorklogId=622001=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-622001
]
ASF GitHub Bot logged work on HIVE-25320:
-
Author: ASF GitHub Bot
Created on: 13/Jul/21 13:39
Start Date: 13/Jul/21 13:39
Worklog Time Spent: 10m
Work Description: belugabehr commented on pull request #2466:
URL: https://github.com/apache/hive/pull/2466#issuecomment-879097041
+1 pending tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 622001)
Time Spent: 40m (was: 0.5h)
> Remove hive.optimize.sort.dynamic.partition
> ---
>
> Key: HIVE-25320
> URL: https://issues.apache.org/jira/browse/HIVE-25320
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Reporter: David Mollitor
>Assignee: Alex Sun
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 40m
> Remaining Estimate: 0h
>
> {{hive.optimize.sort.dynamic.partition}} has been replace with
> {{hive.optimize.sort.dynamic.partition.threshold}} . It has been marked as
> "deprecated", but it's actually totally defunct in the current code base.
> Deprecation would allow an admin to continue to use it (maybe as an alias to
> {{threadshold}} = 0/-1), but that is not the case here.
>
> Remove all references to "hive.optimize.sort.dynamic.partition" in the q
> tests and remove {{HIVEOPTSORTDYNAMICPARTITION}} all together.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HIVE-25288) Fix TestMmCompactorOnTez
[ https://issues.apache.org/jira/browse/HIVE-25288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Denys Kuzmenko reassigned HIVE-25288: - Assignee: Denys Kuzmenko > Fix TestMmCompactorOnTez > > > Key: HIVE-25288 > URL: https://issues.apache.org/jira/browse/HIVE-25288 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Denys Kuzmenko >Priority: Major > > http://ci.hive.apache.org/job/hive-flaky-check/240/ -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-25249) Fix TestWorker
[ https://issues.apache.org/jira/browse/HIVE-25249?focusedWorklogId=621959=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-621959 ] ASF GitHub Bot logged work on HIVE-25249: - Author: ASF GitHub Bot Created on: 13/Jul/21 12:07 Start Date: 13/Jul/21 12:07 Worklog Time Spent: 10m Work Description: klcopp commented on pull request #2474: URL: https://github.com/apache/hive/pull/2474#issuecomment-879031510 So we expect the visibility id to always be different? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 621959) Time Spent: 20m (was: 10m) > Fix TestWorker > -- > > Key: HIVE-25249 > URL: https://issues.apache.org/jira/browse/HIVE-25249 > Project: Hive > Issue Type: Bug >Reporter: Zoltan Haindrich >Assignee: Denys Kuzmenko >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > http://ci.hive.apache.org/job/hive-precommit/job/PR-2381/1/ > http://ci.hive.apache.org/job/hive-flaky-check/236/ -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-22626) Fix TestStatsReplicationScenariosACIDNoAutogather
[
https://issues.apache.org/jira/browse/HIVE-22626?focusedWorklogId=621935=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-621935
]
ASF GitHub Bot logged work on HIVE-22626:
-
Author: ASF GitHub Bot
Created on: 13/Jul/21 10:51
Start Date: 13/Jul/21 10:51
Worklog Time Spent: 10m
Work Description: ArkoSharma commented on a change in pull request #2452:
URL: https://github.com/apache/hive/pull/2452#discussion_r668649790
##
File path: ql/src/java/org/apache/hadoop/hive/ql/parse/repl/ReplLogger.java
##
@@ -1,4 +1,4 @@
-/*
+/**
Review comment:
typo
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 621935)
Time Spent: 50m (was: 40m)
> Fix TestStatsReplicationScenariosACIDNoAutogather
> -
>
> Key: HIVE-22626
> URL: https://issues.apache.org/jira/browse/HIVE-22626
> Project: Hive
> Issue Type: Sub-task
> Components: Test
>Reporter: Zoltan Haindrich
>Assignee: Ayush Saxena
>Priority: Major
> Labels: pull-request-available
> Attachments: qalogs.tgz
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> this test is running "alone" because but still; it sometimes runs more than
> 40m which results in a timeout
> a jira search reveals that was pretty common:
> https://issues.apache.org/jira/issues/?jql=text%20~%20%22TestStatsReplicationScenariosACIDNoAutogather%22%20order%20by%20updated%20desc
> from the hive logs:
> * it seems like after a few minutes this test starts there is an exception:
> {code}
> 2019-12-10T22:43:19,594 DEBUG [Finalizer] metastore.HiveMetaStoreClient:
> Unable to shutdown metastore client. Will try closing transport directly.
> org.apache.thrift.transport.TTransportException: java.net.SocketException:
> Socket closed
> at
> org.apache.thrift.transport.TIOStreamTransport.flush(TIOStreamTransport.java:161)
> ~[libthrift-0.9.3-1.jar:0.9.3-1]
> at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:73)
> ~[libthrift-0.9.3-1.jar:0.9.3-1]
> at
> org.apache.thrift.TServiceClient.sendBaseOneway(TServiceClient.java:66)
> ~[libthrift-0.9.3-1.jar:0.9.3-1]
> at
> com.facebook.fb303.FacebookService$Client.send_shutdown(FacebookService.java:436)
> ~[libfb303-0.9.3.jar:?]
> at
> com.facebook.fb303.FacebookService$Client.shutdown(FacebookService.java:430)
> ~[libfb303-0.9.3.jar:?]
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.close(HiveMetaStoreClient.java:776)
> [hive-standalone-metastore-common-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ~[?:1.8.0_102]
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> ~[?:1.8.0_102]
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[?:1.8.0_102]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_102]
> at
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:212)
> [hive-standalone-metastore-common-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at com.sun.proxy.$Proxy62.close(Unknown Source) [?:?]
> at org.apache.hadoop.hive.ql.metadata.Hive.close(Hive.java:542)
> [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.metadata.Hive.finalize(Hive.java:514)
> [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at java.lang.System$2.invokeFinalize(System.java:1270) [?:1.8.0_102]
> at java.lang.ref.Finalizer.runFinalizer(Finalizer.java:98)
> [?:1.8.0_102]
> at java.lang.ref.Finalizer.access$100(Finalizer.java:34) [?:1.8.0_102]
> at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:210)
> [?:1.8.0_102]
> Caused by: java.net.SocketException: Socket closed
> at
> java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:116)
> ~[?:1.8.0_102]
> at java.net.SocketOutputStream.write(SocketOutputStream.java:153)
> ~[?:1.8.0_102]
> at
> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
> ~[?:1.8.0_102]
> at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
> ~[?:1.8.0_102]
> at
> org.apache.thrift.transport.TIOStreamTransport.flush(TIOStreamTransport.java:159)
> ~[libthrift-0.9.3-1.jar:0.9.3-1]
> {code}
> * after that some
[jira] [Work logged] (HIVE-25249) Fix TestWorker
[ https://issues.apache.org/jira/browse/HIVE-25249?focusedWorklogId=621931=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-621931 ] ASF GitHub Bot logged work on HIVE-25249: - Author: ASF GitHub Bot Created on: 13/Jul/21 10:48 Start Date: 13/Jul/21 10:48 Worklog Time Spent: 10m Work Description: deniskuzZ opened a new pull request #2474: URL: https://github.com/apache/hive/pull/2474 ### What changes were proposed in this pull request? ### Why are the changes needed? Flaky test ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? http://ci.hive.apache.org/job/hive-flaky-check/309 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 621931) Remaining Estimate: 0h Time Spent: 10m > Fix TestWorker > -- > > Key: HIVE-25249 > URL: https://issues.apache.org/jira/browse/HIVE-25249 > Project: Hive > Issue Type: Bug >Reporter: Zoltan Haindrich >Assignee: Denys Kuzmenko >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > http://ci.hive.apache.org/job/hive-precommit/job/PR-2381/1/ > http://ci.hive.apache.org/job/hive-flaky-check/236/ -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25249) Fix TestWorker
[ https://issues.apache.org/jira/browse/HIVE-25249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-25249: -- Labels: pull-request-available (was: ) > Fix TestWorker > -- > > Key: HIVE-25249 > URL: https://issues.apache.org/jira/browse/HIVE-25249 > Project: Hive > Issue Type: Bug >Reporter: Zoltan Haindrich >Assignee: Denys Kuzmenko >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > http://ci.hive.apache.org/job/hive-precommit/job/PR-2381/1/ > http://ci.hive.apache.org/job/hive-flaky-check/236/ -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-25316) Map partition key columns when pushing TNK op through select
[
https://issues.apache.org/jira/browse/HIVE-25316?focusedWorklogId=621922=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-621922
]
ASF GitHub Bot logged work on HIVE-25316:
-
Author: ASF GitHub Bot
Created on: 13/Jul/21 10:35
Start Date: 13/Jul/21 10:35
Worklog Time Spent: 10m
Work Description: kasakrisz merged pull request #2469:
URL: https://github.com/apache/hive/pull/2469
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 621922)
Time Spent: 0.5h (was: 20m)
> Map partition key columns when pushing TNK op through select
>
>
> Key: HIVE-25316
> URL: https://issues.apache.org/jira/browse/HIVE-25316
> Project: Hive
> Issue Type: Bug
> Components: JDBC storage handler, Physical Optimizer, Query Processor
>Affects Versions: 4.0.0
>Reporter: Stamatis Zampetakis
>Assignee: Krisztian Kasa
>Priority: Major
> Labels: pull-request-available
> Attachments: external_jdbc_table_perf2.q
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> The following TPC-DS query fails at runtime when the table {{store_sales}} is
> an external JDBC table.
> {code:sql}
> SELECT ranking
> FROM
> (SELECT rank() OVER (PARTITION BY ss_store_sk
> ORDER BY sum(ss_net_profit)) AS ranking
> FROM store_sales
> GROUP BY ss_store_sk) tmp1
> WHERE ranking <= 5
> {code}
> The stacktrace below shows that problem occurs while trying to initialize the
> {{TopNKeyOperator}}.
> {noformat}
> 2021-07-08T09:04:37,444 ERROR [TezTR-270335_1_3_0_0_0] tez.TezProcessor:
> Failed initializeAndRunProcessor
> java.lang.RuntimeException: Map operator initialization failed
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:351)
> ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:310)
> [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:277)
> [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:381)
> [tez-runtime-internals-0.10.0.jar:0.10.0]
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:75)
> [tez-runtime-internals-0.10.0.jar:0.10.0]
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:62)
> [tez-runtime-internals-0.10.0.jar:0.10.0]
> at java.security.AccessController.doPrivileged(Native Method)
> ~[?:1.8.0_261]
> at javax.security.auth.Subject.doAs(Subject.java:422) [?:1.8.0_261]
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1682)
> [hadoop-common-3.1.0.jar:?]
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:62)
> [tez-runtime-internals-0.10.0.jar:0.10.0]
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:38)
> [tez-runtime-internals-0.10.0.jar:0.10.0]
> at
> org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> [tez-common-0.10.0.jar:0.10.0]
> at
> org.apache.hadoop.hive.llap.daemon.impl.StatsRecordingThreadPool$WrappedCallable.call(StatsRecordingThreadPool.java:118)
> [hive-llap-server-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> [?:1.8.0_261]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> [?:1.8.0_261]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> [?:1.8.0_261]
> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_261]
> Caused by: java.lang.RuntimeException: cannot find field _col0 from
> [0:ss_store_sk, 1:$f1]
> at
> org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.getStandardStructFieldRef(ObjectInspectorUtils.java:550)
> ~[hive-serde-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.getStructFieldRef(StandardStructObjectInspector.java:153)
> ~[hive-serde-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at
>
[jira] [Updated] (HIVE-25316) Map partition key columns when pushing TNK op through select
[
https://issues.apache.org/jira/browse/HIVE-25316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-25316:
--
Resolution: Fixed
Status: Resolved (was: Patch Available)
Pushed to master. Thanks [~kgyrtkirk] for review.
> Map partition key columns when pushing TNK op through select
>
>
> Key: HIVE-25316
> URL: https://issues.apache.org/jira/browse/HIVE-25316
> Project: Hive
> Issue Type: Bug
> Components: JDBC storage handler, Physical Optimizer, Query Processor
>Affects Versions: 4.0.0
>Reporter: Stamatis Zampetakis
>Assignee: Krisztian Kasa
>Priority: Major
> Labels: pull-request-available
> Attachments: external_jdbc_table_perf2.q
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> The following TPC-DS query fails at runtime when the table {{store_sales}} is
> an external JDBC table.
> {code:sql}
> SELECT ranking
> FROM
> (SELECT rank() OVER (PARTITION BY ss_store_sk
> ORDER BY sum(ss_net_profit)) AS ranking
> FROM store_sales
> GROUP BY ss_store_sk) tmp1
> WHERE ranking <= 5
> {code}
> The stacktrace below shows that problem occurs while trying to initialize the
> {{TopNKeyOperator}}.
> {noformat}
> 2021-07-08T09:04:37,444 ERROR [TezTR-270335_1_3_0_0_0] tez.TezProcessor:
> Failed initializeAndRunProcessor
> java.lang.RuntimeException: Map operator initialization failed
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:351)
> ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:310)
> [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:277)
> [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:381)
> [tez-runtime-internals-0.10.0.jar:0.10.0]
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:75)
> [tez-runtime-internals-0.10.0.jar:0.10.0]
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:62)
> [tez-runtime-internals-0.10.0.jar:0.10.0]
> at java.security.AccessController.doPrivileged(Native Method)
> ~[?:1.8.0_261]
> at javax.security.auth.Subject.doAs(Subject.java:422) [?:1.8.0_261]
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1682)
> [hadoop-common-3.1.0.jar:?]
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:62)
> [tez-runtime-internals-0.10.0.jar:0.10.0]
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:38)
> [tez-runtime-internals-0.10.0.jar:0.10.0]
> at
> org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> [tez-common-0.10.0.jar:0.10.0]
> at
> org.apache.hadoop.hive.llap.daemon.impl.StatsRecordingThreadPool$WrappedCallable.call(StatsRecordingThreadPool.java:118)
> [hive-llap-server-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> [?:1.8.0_261]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> [?:1.8.0_261]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> [?:1.8.0_261]
> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_261]
> Caused by: java.lang.RuntimeException: cannot find field _col0 from
> [0:ss_store_sk, 1:$f1]
> at
> org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.getStandardStructFieldRef(ObjectInspectorUtils.java:550)
> ~[hive-serde-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.getStructFieldRef(StandardStructObjectInspector.java:153)
> ~[hive-serde-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.ExprNodeColumnEvaluator.initialize(ExprNodeColumnEvaluator.java:56)
> ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.TopNKeyOperator.initObjectInspectors(TopNKeyOperator.java:101)
> ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.TopNKeyOperator.initializeOp(TopNKeyOperator.java:82)
> ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:360)
> ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:549)
>
[jira] [Assigned] (HIVE-25249) Fix TestWorker
[ https://issues.apache.org/jira/browse/HIVE-25249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Denys Kuzmenko reassigned HIVE-25249: - Assignee: Denys Kuzmenko > Fix TestWorker > -- > > Key: HIVE-25249 > URL: https://issues.apache.org/jira/browse/HIVE-25249 > Project: Hive > Issue Type: Bug >Reporter: Zoltan Haindrich >Assignee: Denys Kuzmenko >Priority: Major > > http://ci.hive.apache.org/job/hive-precommit/job/PR-2381/1/ > http://ci.hive.apache.org/job/hive-flaky-check/236/ -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25327) Mapjoins in HiveServer2 fail when jmxremote is used
[
https://issues.apache.org/jira/browse/HIVE-25327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
louiechen updated HIVE-25327:
-
Description:
I also encountered the same problem, although this problem has been closed in
the previous version, tracking the source code of the current version, there
are also corrections, but the problem remains
The following is the main content of the previous issue[HIVE-11369]:
having hive.auto.convert.join set to true works in the CLI with no issue, but
fails in HiveServer2 when jmx options are passed to the service on startup.
This (in hive-env.sh) is enough to make it fail:
{noformat}
-Dcom.sun.management.jmxremote
-Dcom.sun.management.jmxremote.authenticate=false
-Dcom.sun.management.jmxremote.ssl=false
-Dcom.sun.management.jmxremote.port=8009
{noformat}
As soon as I remove the line, it works properly. I have *no*idea...
Here's the log from the service:
{noformat}
2015-07-24 17:19:27,457 INFO [HiveServer2-Handler-Pool: Thread-22]: ql.Driver
(SessionState.java:printInfo(912)) - Query ID =
hive_20150724171919_aaa88a89-dc6d-490b-821c-4eec6d4c0421
2015-07-24 17:19:27,457 INFO [HiveServer2-Handler-Pool: Thread-22]: ql.Driver
(SessionState.java:printInfo(912)) - Total jobs = 1
2015-07-24 17:19:27,465 INFO [HiveServer2-Handler-Pool: Thread-22]: ql.Driver
(Driver.java:launchTask(1638)) - Starting task [Stage-4:MAPREDLOCAL] in serial
mode
2015-07-24 17:19:27,467 INFO [HiveServer2-Handler-Pool: Thread-22]:
mr.MapredLocalTask (MapredLocalTask.java:executeInChildVM(159)) - Generating
plan file
file:/tmp/hive/8932c206-5420-4b6f-9f1f-5f1706f30df8/hive_2015-07-24_17-19-26_552_5082133674120283907-1/-local-10005/plan.xml
2015-07-24 17:19:27,625 WARN [HiveServer2-Handler-Pool: Thread-22]:
conf.HiveConf (HiveConf.java:initialize(2620)) - HiveConf of name
hive.files.umask.value does not exist
2015-07-24 17:19:27,708 INFO [HiveServer2-Handler-Pool: Thread-22]:
mr.MapredLocalTask (MapredLocalTask.java:executeInChildVM(288)) - Executing:
/usr/lib/hadoop/bin/hadoop jar /usr/lib/hive/lib/hive-common-1.1.0-cdh5.4.3.jar
org.apache.hadoop.hive.ql.exec.mr.ExecDriver -localtask -plan
file:/tmp/hive/8932c206-5420-4b6f-9f1f-5f1706f30df8/hive_2015-07-24_17-19-26_552_5082133674120283907-1/-local-10005/plan.xml
-jobconffile
file:/tmp/hive/8932c206-5420-4b6f-9f1f-5f1706f30df8/hive_2015-07-24_17-19-26_552_5082133674120283907-1/-local-10006/jobconf.xml
2015-07-24 17:19:28,499 ERROR [HiveServer2-Handler-Pool: Thread-22]: exec.Task
(SessionState.java:printError(921)) - Execution failed with exit status: 1
2015-07-24 17:19:28,500 ERROR [HiveServer2-Handler-Pool: Thread-22]: exec.Task
(SessionState.java:printError(921)) - Obtaining error information
2015-07-24 17:19:28,500 ERROR [HiveServer2-Handler-Pool: Thread-22]: exec.Task
(SessionState.java:printError(921)) -
Task failed!
Task ID:
Stage-4
Logs:
2015-07-24 17:19:28,501 ERROR [HiveServer2-Handler-Pool: Thread-22]: exec.Task
(SessionState.java:printError(921)) - /tmp/hiveserver2_manual/hive-server2.log
2015-07-24 17:19:28,501 ERROR [HiveServer2-Handler-Pool: Thread-22]:
mr.MapredLocalTask (MapredLocalTask.java:executeInChildVM(308)) - Execution
failed with exit status: 1
2015-07-24 17:19:28,518 ERROR [HiveServer2-Handler-Pool: Thread-22]: ql.Driver
(SessionState.java:printError(921)) - FAILED: Execution Error, return code 1
from org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask
2015-07-24 17:19:28,599 WARN [HiveServer2-Handler-Pool: Thread-22]:
security.UserGroupInformation (UserGroupInformation.java:doAs(1674)) -
PriviledgedActionException as:hive (auth:SIMPLE)
cause:org.apache.hive.service.cli.HiveSQLException: Error while processing
statement: FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask
2015-07-24 17:19:28,600 WARN [HiveServer2-Handler-Pool: Thread-22]:
thrift.ThriftCLIService (ThriftCLIService.java:ExecuteStatement(496)) - Error
executing statement:
org.apache.hive.service.cli.HiveSQLException: Error while processing statement:
FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask
at
org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:315)
at
org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:146)
at
org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:173)
at
org.apache.hive.service.cli.operation.Operation.run(Operation.java:257)
at
org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:398)
at
org.apache.hive.service.cli.session.HiveSessionImpl.executeStatement(HiveSessionImpl.java:379)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
[jira] [Updated] (HIVE-25327) Mapjoins in HiveServer2 fail when jmxremote is used
[
https://issues.apache.org/jira/browse/HIVE-25327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
louiechen updated HIVE-25327:
-
Description:
I also encountered the same problem, although this problem has been closed in
the previous version, tracking the source code of the current version, there
are also corrections, but the problem remains
The following is the main content of the previous issue:
having hive.auto.convert.join set to true works in the CLI with no issue, but
fails in HiveServer2 when jmx options are passed to the service on startup.
This (in hive-env.sh) is enough to make it fail:
{noformat}
-Dcom.sun.management.jmxremote
-Dcom.sun.management.jmxremote.authenticate=false
-Dcom.sun.management.jmxremote.ssl=false
-Dcom.sun.management.jmxremote.port=8009
{noformat}
As soon as I remove the line, it works properly. I have *no*idea...
Here's the log from the service:
{noformat}
2015-07-24 17:19:27,457 INFO [HiveServer2-Handler-Pool: Thread-22]: ql.Driver
(SessionState.java:printInfo(912)) - Query ID =
hive_20150724171919_aaa88a89-dc6d-490b-821c-4eec6d4c0421
2015-07-24 17:19:27,457 INFO [HiveServer2-Handler-Pool: Thread-22]: ql.Driver
(SessionState.java:printInfo(912)) - Total jobs = 1
2015-07-24 17:19:27,465 INFO [HiveServer2-Handler-Pool: Thread-22]: ql.Driver
(Driver.java:launchTask(1638)) - Starting task [Stage-4:MAPREDLOCAL] in serial
mode
2015-07-24 17:19:27,467 INFO [HiveServer2-Handler-Pool: Thread-22]:
mr.MapredLocalTask (MapredLocalTask.java:executeInChildVM(159)) - Generating
plan file
file:/tmp/hive/8932c206-5420-4b6f-9f1f-5f1706f30df8/hive_2015-07-24_17-19-26_552_5082133674120283907-1/-local-10005/plan.xml
2015-07-24 17:19:27,625 WARN [HiveServer2-Handler-Pool: Thread-22]:
conf.HiveConf (HiveConf.java:initialize(2620)) - HiveConf of name
hive.files.umask.value does not exist
2015-07-24 17:19:27,708 INFO [HiveServer2-Handler-Pool: Thread-22]:
mr.MapredLocalTask (MapredLocalTask.java:executeInChildVM(288)) - Executing:
/usr/lib/hadoop/bin/hadoop jar /usr/lib/hive/lib/hive-common-1.1.0-cdh5.4.3.jar
org.apache.hadoop.hive.ql.exec.mr.ExecDriver -localtask -plan
file:/tmp/hive/8932c206-5420-4b6f-9f1f-5f1706f30df8/hive_2015-07-24_17-19-26_552_5082133674120283907-1/-local-10005/plan.xml
-jobconffile
file:/tmp/hive/8932c206-5420-4b6f-9f1f-5f1706f30df8/hive_2015-07-24_17-19-26_552_5082133674120283907-1/-local-10006/jobconf.xml
2015-07-24 17:19:28,499 ERROR [HiveServer2-Handler-Pool: Thread-22]: exec.Task
(SessionState.java:printError(921)) - Execution failed with exit status: 1
2015-07-24 17:19:28,500 ERROR [HiveServer2-Handler-Pool: Thread-22]: exec.Task
(SessionState.java:printError(921)) - Obtaining error information
2015-07-24 17:19:28,500 ERROR [HiveServer2-Handler-Pool: Thread-22]: exec.Task
(SessionState.java:printError(921)) -
Task failed!
Task ID:
Stage-4
Logs:
2015-07-24 17:19:28,501 ERROR [HiveServer2-Handler-Pool: Thread-22]: exec.Task
(SessionState.java:printError(921)) - /tmp/hiveserver2_manual/hive-server2.log
2015-07-24 17:19:28,501 ERROR [HiveServer2-Handler-Pool: Thread-22]:
mr.MapredLocalTask (MapredLocalTask.java:executeInChildVM(308)) - Execution
failed with exit status: 1
2015-07-24 17:19:28,518 ERROR [HiveServer2-Handler-Pool: Thread-22]: ql.Driver
(SessionState.java:printError(921)) - FAILED: Execution Error, return code 1
from org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask
2015-07-24 17:19:28,599 WARN [HiveServer2-Handler-Pool: Thread-22]:
security.UserGroupInformation (UserGroupInformation.java:doAs(1674)) -
PriviledgedActionException as:hive (auth:SIMPLE)
cause:org.apache.hive.service.cli.HiveSQLException: Error while processing
statement: FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask
2015-07-24 17:19:28,600 WARN [HiveServer2-Handler-Pool: Thread-22]:
thrift.ThriftCLIService (ThriftCLIService.java:ExecuteStatement(496)) - Error
executing statement:
org.apache.hive.service.cli.HiveSQLException: Error while processing statement:
FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask
at
org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:315)
at
org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:146)
at
org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:173)
at
org.apache.hive.service.cli.operation.Operation.run(Operation.java:257)
at
org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:398)
at
org.apache.hive.service.cli.session.HiveSessionImpl.executeStatement(HiveSessionImpl.java:379)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
[jira] [Updated] (HIVE-25327) Mapjoins in HiveServer2 fail when jmxremote is used
[
https://issues.apache.org/jira/browse/HIVE-25327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
louiechen updated HIVE-25327:
-
Environment:
apache hadoop 3.1.3
apache hive 3.1.2
java version 1.8.0_282
OS:RedHat 8.2
was:CDH 5.4.3, Centos 6.5, java version "1.7.0_67"
> Mapjoins in HiveServer2 fail when jmxremote is used
> ---
>
> Key: HIVE-25327
> URL: https://issues.apache.org/jira/browse/HIVE-25327
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Affects Versions: 3.1.2
> Environment: apache hadoop 3.1.3
> apache hive 3.1.2
> java version 1.8.0_282
> OS:RedHat 8.2
>
>Reporter: louiechen
>Assignee: loushang
>Priority: Major
>
> having hive.auto.convert.join set to true works in the CLI with no issue, but
> fails in HiveServer2 when jmx options are passed to the service on startup.
> This (in hive-env.sh) is enough to make it fail:
> {noformat}
> -Dcom.sun.management.jmxremote
> -Dcom.sun.management.jmxremote.authenticate=false
> -Dcom.sun.management.jmxremote.ssl=false
> -Dcom.sun.management.jmxremote.port=8009
> {noformat}
> As soon as I remove the line, it works properly. I have *no*idea...
> Here's the log from the service:
> {noformat}
> 2015-07-24 17:19:27,457 INFO [HiveServer2-Handler-Pool: Thread-22]:
> ql.Driver (SessionState.java:printInfo(912)) - Query ID =
> hive_20150724171919_aaa88a89-dc6d-490b-821c-4eec6d4c0421
> 2015-07-24 17:19:27,457 INFO [HiveServer2-Handler-Pool: Thread-22]:
> ql.Driver (SessionState.java:printInfo(912)) - Total jobs = 1
> 2015-07-24 17:19:27,465 INFO [HiveServer2-Handler-Pool: Thread-22]:
> ql.Driver (Driver.java:launchTask(1638)) - Starting task
> [Stage-4:MAPREDLOCAL] in serial mode
> 2015-07-24 17:19:27,467 INFO [HiveServer2-Handler-Pool: Thread-22]:
> mr.MapredLocalTask (MapredLocalTask.java:executeInChildVM(159)) - Generating
> plan file
> file:/tmp/hive/8932c206-5420-4b6f-9f1f-5f1706f30df8/hive_2015-07-24_17-19-26_552_5082133674120283907-1/-local-10005/plan.xml
> 2015-07-24 17:19:27,625 WARN [HiveServer2-Handler-Pool: Thread-22]:
> conf.HiveConf (HiveConf.java:initialize(2620)) - HiveConf of name
> hive.files.umask.value does not exist
> 2015-07-24 17:19:27,708 INFO [HiveServer2-Handler-Pool: Thread-22]:
> mr.MapredLocalTask (MapredLocalTask.java:executeInChildVM(288)) - Executing:
> /usr/lib/hadoop/bin/hadoop jar
> /usr/lib/hive/lib/hive-common-1.1.0-cdh5.4.3.jar
> org.apache.hadoop.hive.ql.exec.mr.ExecDriver -localtask -plan
> file:/tmp/hive/8932c206-5420-4b6f-9f1f-5f1706f30df8/hive_2015-07-24_17-19-26_552_5082133674120283907-1/-local-10005/plan.xml
>-jobconffile
> file:/tmp/hive/8932c206-5420-4b6f-9f1f-5f1706f30df8/hive_2015-07-24_17-19-26_552_5082133674120283907-1/-local-10006/jobconf.xml
> 2015-07-24 17:19:28,499 ERROR [HiveServer2-Handler-Pool: Thread-22]:
> exec.Task (SessionState.java:printError(921)) - Execution failed with exit
> status: 1
> 2015-07-24 17:19:28,500 ERROR [HiveServer2-Handler-Pool: Thread-22]:
> exec.Task (SessionState.java:printError(921)) - Obtaining error information
> 2015-07-24 17:19:28,500 ERROR [HiveServer2-Handler-Pool: Thread-22]:
> exec.Task (SessionState.java:printError(921)) -
> Task failed!
> Task ID:
> Stage-4
> Logs:
> 2015-07-24 17:19:28,501 ERROR [HiveServer2-Handler-Pool: Thread-22]:
> exec.Task (SessionState.java:printError(921)) -
> /tmp/hiveserver2_manual/hive-server2.log
> 2015-07-24 17:19:28,501 ERROR [HiveServer2-Handler-Pool: Thread-22]:
> mr.MapredLocalTask (MapredLocalTask.java:executeInChildVM(308)) - Execution
> failed with exit status: 1
> 2015-07-24 17:19:28,518 ERROR [HiveServer2-Handler-Pool: Thread-22]:
> ql.Driver (SessionState.java:printError(921)) - FAILED: Execution Error,
> return code 1 from org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask
> 2015-07-24 17:19:28,599 WARN [HiveServer2-Handler-Pool: Thread-22]:
> security.UserGroupInformation (UserGroupInformation.java:doAs(1674)) -
> PriviledgedActionException as:hive (auth:SIMPLE)
> cause:org.apache.hive.service.cli.HiveSQLException: Error while processing
> statement: FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask
> 2015-07-24 17:19:28,600 WARN [HiveServer2-Handler-Pool: Thread-22]:
> thrift.ThriftCLIService (ThriftCLIService.java:ExecuteStatement(496)) - Error
> executing statement:
> org.apache.hive.service.cli.HiveSQLException: Error while processing
> statement: FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask
> at
> org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:315)
> at
> org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:146)
> at
>
[jira] [Updated] (HIVE-25327) Mapjoins in HiveServer2 fail when jmxremote is used
[
https://issues.apache.org/jira/browse/HIVE-25327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
louiechen updated HIVE-25327:
-
Affects Version/s: (was: 1.1.0)
3.1.2
> Mapjoins in HiveServer2 fail when jmxremote is used
> ---
>
> Key: HIVE-25327
> URL: https://issues.apache.org/jira/browse/HIVE-25327
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Affects Versions: 3.1.2
> Environment: CDH 5.4.3, Centos 6.5, java version "1.7.0_67"
>Reporter: louiechen
>Assignee: loushang
>Priority: Major
>
> having hive.auto.convert.join set to true works in the CLI with no issue, but
> fails in HiveServer2 when jmx options are passed to the service on startup.
> This (in hive-env.sh) is enough to make it fail:
> {noformat}
> -Dcom.sun.management.jmxremote
> -Dcom.sun.management.jmxremote.authenticate=false
> -Dcom.sun.management.jmxremote.ssl=false
> -Dcom.sun.management.jmxremote.port=8009
> {noformat}
> As soon as I remove the line, it works properly. I have *no*idea...
> Here's the log from the service:
> {noformat}
> 2015-07-24 17:19:27,457 INFO [HiveServer2-Handler-Pool: Thread-22]:
> ql.Driver (SessionState.java:printInfo(912)) - Query ID =
> hive_20150724171919_aaa88a89-dc6d-490b-821c-4eec6d4c0421
> 2015-07-24 17:19:27,457 INFO [HiveServer2-Handler-Pool: Thread-22]:
> ql.Driver (SessionState.java:printInfo(912)) - Total jobs = 1
> 2015-07-24 17:19:27,465 INFO [HiveServer2-Handler-Pool: Thread-22]:
> ql.Driver (Driver.java:launchTask(1638)) - Starting task
> [Stage-4:MAPREDLOCAL] in serial mode
> 2015-07-24 17:19:27,467 INFO [HiveServer2-Handler-Pool: Thread-22]:
> mr.MapredLocalTask (MapredLocalTask.java:executeInChildVM(159)) - Generating
> plan file
> file:/tmp/hive/8932c206-5420-4b6f-9f1f-5f1706f30df8/hive_2015-07-24_17-19-26_552_5082133674120283907-1/-local-10005/plan.xml
> 2015-07-24 17:19:27,625 WARN [HiveServer2-Handler-Pool: Thread-22]:
> conf.HiveConf (HiveConf.java:initialize(2620)) - HiveConf of name
> hive.files.umask.value does not exist
> 2015-07-24 17:19:27,708 INFO [HiveServer2-Handler-Pool: Thread-22]:
> mr.MapredLocalTask (MapredLocalTask.java:executeInChildVM(288)) - Executing:
> /usr/lib/hadoop/bin/hadoop jar
> /usr/lib/hive/lib/hive-common-1.1.0-cdh5.4.3.jar
> org.apache.hadoop.hive.ql.exec.mr.ExecDriver -localtask -plan
> file:/tmp/hive/8932c206-5420-4b6f-9f1f-5f1706f30df8/hive_2015-07-24_17-19-26_552_5082133674120283907-1/-local-10005/plan.xml
>-jobconffile
> file:/tmp/hive/8932c206-5420-4b6f-9f1f-5f1706f30df8/hive_2015-07-24_17-19-26_552_5082133674120283907-1/-local-10006/jobconf.xml
> 2015-07-24 17:19:28,499 ERROR [HiveServer2-Handler-Pool: Thread-22]:
> exec.Task (SessionState.java:printError(921)) - Execution failed with exit
> status: 1
> 2015-07-24 17:19:28,500 ERROR [HiveServer2-Handler-Pool: Thread-22]:
> exec.Task (SessionState.java:printError(921)) - Obtaining error information
> 2015-07-24 17:19:28,500 ERROR [HiveServer2-Handler-Pool: Thread-22]:
> exec.Task (SessionState.java:printError(921)) -
> Task failed!
> Task ID:
> Stage-4
> Logs:
> 2015-07-24 17:19:28,501 ERROR [HiveServer2-Handler-Pool: Thread-22]:
> exec.Task (SessionState.java:printError(921)) -
> /tmp/hiveserver2_manual/hive-server2.log
> 2015-07-24 17:19:28,501 ERROR [HiveServer2-Handler-Pool: Thread-22]:
> mr.MapredLocalTask (MapredLocalTask.java:executeInChildVM(308)) - Execution
> failed with exit status: 1
> 2015-07-24 17:19:28,518 ERROR [HiveServer2-Handler-Pool: Thread-22]:
> ql.Driver (SessionState.java:printError(921)) - FAILED: Execution Error,
> return code 1 from org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask
> 2015-07-24 17:19:28,599 WARN [HiveServer2-Handler-Pool: Thread-22]:
> security.UserGroupInformation (UserGroupInformation.java:doAs(1674)) -
> PriviledgedActionException as:hive (auth:SIMPLE)
> cause:org.apache.hive.service.cli.HiveSQLException: Error while processing
> statement: FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask
> 2015-07-24 17:19:28,600 WARN [HiveServer2-Handler-Pool: Thread-22]:
> thrift.ThriftCLIService (ThriftCLIService.java:ExecuteStatement(496)) - Error
> executing statement:
> org.apache.hive.service.cli.HiveSQLException: Error while processing
> statement: FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask
> at
> org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:315)
> at
> org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:146)
> at
> org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:173)
> at
>
[jira] [Work logged] (HIVE-25326) Include partitioning info in DESCRIBE TABLE command on Iceberg tables
[
https://issues.apache.org/jira/browse/HIVE-25326?focusedWorklogId=621897=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-621897
]
ASF GitHub Bot logged work on HIVE-25326:
-
Author: ASF GitHub Bot
Created on: 13/Jul/21 09:46
Start Date: 13/Jul/21 09:46
Worklog Time Spent: 10m
Work Description: marton-bod commented on a change in pull request #2472:
URL: https://github.com/apache/hive/pull/2472#discussion_r668604534
##
File path:
ql/src/java/org/apache/hadoop/hive/ql/ddl/table/info/desc/formatter/TextDescTableFormatter.java
##
@@ -100,6 +102,32 @@ public void describeTable(HiveConf conf, DataOutputStream
out, String columnPath
}
}
+ private void addPartitionTransformData(DataOutputStream out, Table table,
boolean isOutputPadded) throws IOException {
+String partitionTransformData = "";
Review comment:
nit: can we name this partitionTransformOutput instead, or something
similar?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 621897)
Time Spent: 0.5h (was: 20m)
> Include partitioning info in DESCRIBE TABLE command on Iceberg tables
> -
>
> Key: HIVE-25326
> URL: https://issues.apache.org/jira/browse/HIVE-25326
> Project: Hive
> Issue Type: Improvement
>Reporter: László Pintér
>Assignee: László Pintér
>Priority: Major
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25326) Include partitioning info in DESCRIBE TABLE command on Iceberg tables
[
https://issues.apache.org/jira/browse/HIVE-25326?focusedWorklogId=621898=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-621898
]
ASF GitHub Bot logged work on HIVE-25326:
-
Author: ASF GitHub Bot
Created on: 13/Jul/21 09:46
Start Date: 13/Jul/21 09:46
Worklog Time Spent: 10m
Work Description: marton-bod commented on a change in pull request #2472:
URL: https://github.com/apache/hive/pull/2472#discussion_r668604935
##
File path:
ql/src/java/org/apache/hadoop/hive/ql/ddl/table/info/desc/formatter/TextDescTableFormatter.java
##
@@ -100,6 +102,32 @@ public void describeTable(HiveConf conf, DataOutputStream
out, String columnPath
}
}
+ private void addPartitionTransformData(DataOutputStream out, Table table,
boolean isOutputPadded) throws IOException {
+String partitionTransformData = "";
+if (table.isNonNative() && table.getStorageHandler() != null &&
+table.getStorageHandler().supportsPartitionTransform()) {
+
+ List partSpecDatas =
table.getStorageHandler().getPartitionTransformSpec(table);
Review comment:
nit: can we name this partSpecs, or something similar?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 621898)
Time Spent: 40m (was: 0.5h)
> Include partitioning info in DESCRIBE TABLE command on Iceberg tables
> -
>
> Key: HIVE-25326
> URL: https://issues.apache.org/jira/browse/HIVE-25326
> Project: Hive
> Issue Type: Improvement
>Reporter: László Pintér
>Assignee: László Pintér
>Priority: Major
> Labels: pull-request-available
> Time Spent: 40m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-25327) Mapjoins in HiveServer2 fail when jmxremote is used
[
https://issues.apache.org/jira/browse/HIVE-25327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
louiechen updated HIVE-25327:
-
Fix Version/s: (was: 2.0.0)
> Mapjoins in HiveServer2 fail when jmxremote is used
> ---
>
> Key: HIVE-25327
> URL: https://issues.apache.org/jira/browse/HIVE-25327
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Affects Versions: 1.1.0
> Environment: CDH 5.4.3, Centos 6.5, java version "1.7.0_67"
>Reporter: louiechen
>Assignee: loushang
>Priority: Major
>
> having hive.auto.convert.join set to true works in the CLI with no issue, but
> fails in HiveServer2 when jmx options are passed to the service on startup.
> This (in hive-env.sh) is enough to make it fail:
> {noformat}
> -Dcom.sun.management.jmxremote
> -Dcom.sun.management.jmxremote.authenticate=false
> -Dcom.sun.management.jmxremote.ssl=false
> -Dcom.sun.management.jmxremote.port=8009
> {noformat}
> As soon as I remove the line, it works properly. I have *no*idea...
> Here's the log from the service:
> {noformat}
> 2015-07-24 17:19:27,457 INFO [HiveServer2-Handler-Pool: Thread-22]:
> ql.Driver (SessionState.java:printInfo(912)) - Query ID =
> hive_20150724171919_aaa88a89-dc6d-490b-821c-4eec6d4c0421
> 2015-07-24 17:19:27,457 INFO [HiveServer2-Handler-Pool: Thread-22]:
> ql.Driver (SessionState.java:printInfo(912)) - Total jobs = 1
> 2015-07-24 17:19:27,465 INFO [HiveServer2-Handler-Pool: Thread-22]:
> ql.Driver (Driver.java:launchTask(1638)) - Starting task
> [Stage-4:MAPREDLOCAL] in serial mode
> 2015-07-24 17:19:27,467 INFO [HiveServer2-Handler-Pool: Thread-22]:
> mr.MapredLocalTask (MapredLocalTask.java:executeInChildVM(159)) - Generating
> plan file
> file:/tmp/hive/8932c206-5420-4b6f-9f1f-5f1706f30df8/hive_2015-07-24_17-19-26_552_5082133674120283907-1/-local-10005/plan.xml
> 2015-07-24 17:19:27,625 WARN [HiveServer2-Handler-Pool: Thread-22]:
> conf.HiveConf (HiveConf.java:initialize(2620)) - HiveConf of name
> hive.files.umask.value does not exist
> 2015-07-24 17:19:27,708 INFO [HiveServer2-Handler-Pool: Thread-22]:
> mr.MapredLocalTask (MapredLocalTask.java:executeInChildVM(288)) - Executing:
> /usr/lib/hadoop/bin/hadoop jar
> /usr/lib/hive/lib/hive-common-1.1.0-cdh5.4.3.jar
> org.apache.hadoop.hive.ql.exec.mr.ExecDriver -localtask -plan
> file:/tmp/hive/8932c206-5420-4b6f-9f1f-5f1706f30df8/hive_2015-07-24_17-19-26_552_5082133674120283907-1/-local-10005/plan.xml
>-jobconffile
> file:/tmp/hive/8932c206-5420-4b6f-9f1f-5f1706f30df8/hive_2015-07-24_17-19-26_552_5082133674120283907-1/-local-10006/jobconf.xml
> 2015-07-24 17:19:28,499 ERROR [HiveServer2-Handler-Pool: Thread-22]:
> exec.Task (SessionState.java:printError(921)) - Execution failed with exit
> status: 1
> 2015-07-24 17:19:28,500 ERROR [HiveServer2-Handler-Pool: Thread-22]:
> exec.Task (SessionState.java:printError(921)) - Obtaining error information
> 2015-07-24 17:19:28,500 ERROR [HiveServer2-Handler-Pool: Thread-22]:
> exec.Task (SessionState.java:printError(921)) -
> Task failed!
> Task ID:
> Stage-4
> Logs:
> 2015-07-24 17:19:28,501 ERROR [HiveServer2-Handler-Pool: Thread-22]:
> exec.Task (SessionState.java:printError(921)) -
> /tmp/hiveserver2_manual/hive-server2.log
> 2015-07-24 17:19:28,501 ERROR [HiveServer2-Handler-Pool: Thread-22]:
> mr.MapredLocalTask (MapredLocalTask.java:executeInChildVM(308)) - Execution
> failed with exit status: 1
> 2015-07-24 17:19:28,518 ERROR [HiveServer2-Handler-Pool: Thread-22]:
> ql.Driver (SessionState.java:printError(921)) - FAILED: Execution Error,
> return code 1 from org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask
> 2015-07-24 17:19:28,599 WARN [HiveServer2-Handler-Pool: Thread-22]:
> security.UserGroupInformation (UserGroupInformation.java:doAs(1674)) -
> PriviledgedActionException as:hive (auth:SIMPLE)
> cause:org.apache.hive.service.cli.HiveSQLException: Error while processing
> statement: FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask
> 2015-07-24 17:19:28,600 WARN [HiveServer2-Handler-Pool: Thread-22]:
> thrift.ThriftCLIService (ThriftCLIService.java:ExecuteStatement(496)) - Error
> executing statement:
> org.apache.hive.service.cli.HiveSQLException: Error while processing
> statement: FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask
> at
> org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:315)
> at
> org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:146)
> at
> org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:173)
> at
> org.apache.hive.service.cli.operation.Operation.run(Operation.java:257)
> at
>
[jira] [Updated] (HIVE-25327) Mapjoins in HiveServer2 fail when jmxremote is used
[
https://issues.apache.org/jira/browse/HIVE-25327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
louiechen updated HIVE-25327:
-
Summary: Mapjoins in HiveServer2 fail when jmxremote is used (was: CLONE -
Mapjoins in HiveServer2 fail when jmxremote is used)
> Mapjoins in HiveServer2 fail when jmxremote is used
> ---
>
> Key: HIVE-25327
> URL: https://issues.apache.org/jira/browse/HIVE-25327
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Affects Versions: 1.1.0
> Environment: CDH 5.4.3, Centos 6.5, java version "1.7.0_67"
>Reporter: louiechen
>Assignee: loushang
>Priority: Major
> Fix For: 2.0.0
>
>
> having hive.auto.convert.join set to true works in the CLI with no issue, but
> fails in HiveServer2 when jmx options are passed to the service on startup.
> This (in hive-env.sh) is enough to make it fail:
> {noformat}
> -Dcom.sun.management.jmxremote
> -Dcom.sun.management.jmxremote.authenticate=false
> -Dcom.sun.management.jmxremote.ssl=false
> -Dcom.sun.management.jmxremote.port=8009
> {noformat}
> As soon as I remove the line, it works properly. I have *no*idea...
> Here's the log from the service:
> {noformat}
> 2015-07-24 17:19:27,457 INFO [HiveServer2-Handler-Pool: Thread-22]:
> ql.Driver (SessionState.java:printInfo(912)) - Query ID =
> hive_20150724171919_aaa88a89-dc6d-490b-821c-4eec6d4c0421
> 2015-07-24 17:19:27,457 INFO [HiveServer2-Handler-Pool: Thread-22]:
> ql.Driver (SessionState.java:printInfo(912)) - Total jobs = 1
> 2015-07-24 17:19:27,465 INFO [HiveServer2-Handler-Pool: Thread-22]:
> ql.Driver (Driver.java:launchTask(1638)) - Starting task
> [Stage-4:MAPREDLOCAL] in serial mode
> 2015-07-24 17:19:27,467 INFO [HiveServer2-Handler-Pool: Thread-22]:
> mr.MapredLocalTask (MapredLocalTask.java:executeInChildVM(159)) - Generating
> plan file
> file:/tmp/hive/8932c206-5420-4b6f-9f1f-5f1706f30df8/hive_2015-07-24_17-19-26_552_5082133674120283907-1/-local-10005/plan.xml
> 2015-07-24 17:19:27,625 WARN [HiveServer2-Handler-Pool: Thread-22]:
> conf.HiveConf (HiveConf.java:initialize(2620)) - HiveConf of name
> hive.files.umask.value does not exist
> 2015-07-24 17:19:27,708 INFO [HiveServer2-Handler-Pool: Thread-22]:
> mr.MapredLocalTask (MapredLocalTask.java:executeInChildVM(288)) - Executing:
> /usr/lib/hadoop/bin/hadoop jar
> /usr/lib/hive/lib/hive-common-1.1.0-cdh5.4.3.jar
> org.apache.hadoop.hive.ql.exec.mr.ExecDriver -localtask -plan
> file:/tmp/hive/8932c206-5420-4b6f-9f1f-5f1706f30df8/hive_2015-07-24_17-19-26_552_5082133674120283907-1/-local-10005/plan.xml
>-jobconffile
> file:/tmp/hive/8932c206-5420-4b6f-9f1f-5f1706f30df8/hive_2015-07-24_17-19-26_552_5082133674120283907-1/-local-10006/jobconf.xml
> 2015-07-24 17:19:28,499 ERROR [HiveServer2-Handler-Pool: Thread-22]:
> exec.Task (SessionState.java:printError(921)) - Execution failed with exit
> status: 1
> 2015-07-24 17:19:28,500 ERROR [HiveServer2-Handler-Pool: Thread-22]:
> exec.Task (SessionState.java:printError(921)) - Obtaining error information
> 2015-07-24 17:19:28,500 ERROR [HiveServer2-Handler-Pool: Thread-22]:
> exec.Task (SessionState.java:printError(921)) -
> Task failed!
> Task ID:
> Stage-4
> Logs:
> 2015-07-24 17:19:28,501 ERROR [HiveServer2-Handler-Pool: Thread-22]:
> exec.Task (SessionState.java:printError(921)) -
> /tmp/hiveserver2_manual/hive-server2.log
> 2015-07-24 17:19:28,501 ERROR [HiveServer2-Handler-Pool: Thread-22]:
> mr.MapredLocalTask (MapredLocalTask.java:executeInChildVM(308)) - Execution
> failed with exit status: 1
> 2015-07-24 17:19:28,518 ERROR [HiveServer2-Handler-Pool: Thread-22]:
> ql.Driver (SessionState.java:printError(921)) - FAILED: Execution Error,
> return code 1 from org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask
> 2015-07-24 17:19:28,599 WARN [HiveServer2-Handler-Pool: Thread-22]:
> security.UserGroupInformation (UserGroupInformation.java:doAs(1674)) -
> PriviledgedActionException as:hive (auth:SIMPLE)
> cause:org.apache.hive.service.cli.HiveSQLException: Error while processing
> statement: FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask
> 2015-07-24 17:19:28,600 WARN [HiveServer2-Handler-Pool: Thread-22]:
> thrift.ThriftCLIService (ThriftCLIService.java:ExecuteStatement(496)) - Error
> executing statement:
> org.apache.hive.service.cli.HiveSQLException: Error while processing
> statement: FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask
> at
> org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:315)
> at
> org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:146)
> at
>
[jira] [Assigned] (HIVE-25327) CLONE - Mapjoins in HiveServer2 fail when jmxremote is used
[
https://issues.apache.org/jira/browse/HIVE-25327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
louiechen reassigned HIVE-25327:
> CLONE - Mapjoins in HiveServer2 fail when jmxremote is used
> ---
>
> Key: HIVE-25327
> URL: https://issues.apache.org/jira/browse/HIVE-25327
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Affects Versions: 1.1.0
> Environment: CDH 5.4.3, Centos 6.5, java version "1.7.0_67"
>Reporter: louiechen
>Assignee: loushang
>Priority: Major
> Fix For: 2.0.0
>
>
> having hive.auto.convert.join set to true works in the CLI with no issue, but
> fails in HiveServer2 when jmx options are passed to the service on startup.
> This (in hive-env.sh) is enough to make it fail:
> {noformat}
> -Dcom.sun.management.jmxremote
> -Dcom.sun.management.jmxremote.authenticate=false
> -Dcom.sun.management.jmxremote.ssl=false
> -Dcom.sun.management.jmxremote.port=8009
> {noformat}
> As soon as I remove the line, it works properly. I have *no*idea...
> Here's the log from the service:
> {noformat}
> 2015-07-24 17:19:27,457 INFO [HiveServer2-Handler-Pool: Thread-22]:
> ql.Driver (SessionState.java:printInfo(912)) - Query ID =
> hive_20150724171919_aaa88a89-dc6d-490b-821c-4eec6d4c0421
> 2015-07-24 17:19:27,457 INFO [HiveServer2-Handler-Pool: Thread-22]:
> ql.Driver (SessionState.java:printInfo(912)) - Total jobs = 1
> 2015-07-24 17:19:27,465 INFO [HiveServer2-Handler-Pool: Thread-22]:
> ql.Driver (Driver.java:launchTask(1638)) - Starting task
> [Stage-4:MAPREDLOCAL] in serial mode
> 2015-07-24 17:19:27,467 INFO [HiveServer2-Handler-Pool: Thread-22]:
> mr.MapredLocalTask (MapredLocalTask.java:executeInChildVM(159)) - Generating
> plan file
> file:/tmp/hive/8932c206-5420-4b6f-9f1f-5f1706f30df8/hive_2015-07-24_17-19-26_552_5082133674120283907-1/-local-10005/plan.xml
> 2015-07-24 17:19:27,625 WARN [HiveServer2-Handler-Pool: Thread-22]:
> conf.HiveConf (HiveConf.java:initialize(2620)) - HiveConf of name
> hive.files.umask.value does not exist
> 2015-07-24 17:19:27,708 INFO [HiveServer2-Handler-Pool: Thread-22]:
> mr.MapredLocalTask (MapredLocalTask.java:executeInChildVM(288)) - Executing:
> /usr/lib/hadoop/bin/hadoop jar
> /usr/lib/hive/lib/hive-common-1.1.0-cdh5.4.3.jar
> org.apache.hadoop.hive.ql.exec.mr.ExecDriver -localtask -plan
> file:/tmp/hive/8932c206-5420-4b6f-9f1f-5f1706f30df8/hive_2015-07-24_17-19-26_552_5082133674120283907-1/-local-10005/plan.xml
>-jobconffile
> file:/tmp/hive/8932c206-5420-4b6f-9f1f-5f1706f30df8/hive_2015-07-24_17-19-26_552_5082133674120283907-1/-local-10006/jobconf.xml
> 2015-07-24 17:19:28,499 ERROR [HiveServer2-Handler-Pool: Thread-22]:
> exec.Task (SessionState.java:printError(921)) - Execution failed with exit
> status: 1
> 2015-07-24 17:19:28,500 ERROR [HiveServer2-Handler-Pool: Thread-22]:
> exec.Task (SessionState.java:printError(921)) - Obtaining error information
> 2015-07-24 17:19:28,500 ERROR [HiveServer2-Handler-Pool: Thread-22]:
> exec.Task (SessionState.java:printError(921)) -
> Task failed!
> Task ID:
> Stage-4
> Logs:
> 2015-07-24 17:19:28,501 ERROR [HiveServer2-Handler-Pool: Thread-22]:
> exec.Task (SessionState.java:printError(921)) -
> /tmp/hiveserver2_manual/hive-server2.log
> 2015-07-24 17:19:28,501 ERROR [HiveServer2-Handler-Pool: Thread-22]:
> mr.MapredLocalTask (MapredLocalTask.java:executeInChildVM(308)) - Execution
> failed with exit status: 1
> 2015-07-24 17:19:28,518 ERROR [HiveServer2-Handler-Pool: Thread-22]:
> ql.Driver (SessionState.java:printError(921)) - FAILED: Execution Error,
> return code 1 from org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask
> 2015-07-24 17:19:28,599 WARN [HiveServer2-Handler-Pool: Thread-22]:
> security.UserGroupInformation (UserGroupInformation.java:doAs(1674)) -
> PriviledgedActionException as:hive (auth:SIMPLE)
> cause:org.apache.hive.service.cli.HiveSQLException: Error while processing
> statement: FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask
> 2015-07-24 17:19:28,600 WARN [HiveServer2-Handler-Pool: Thread-22]:
> thrift.ThriftCLIService (ThriftCLIService.java:ExecuteStatement(496)) - Error
> executing statement:
> org.apache.hive.service.cli.HiveSQLException: Error while processing
> statement: FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask
> at
> org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:315)
> at
> org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:146)
> at
> org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:173)
> at
> org.apache.hive.service.cli.operation.Operation.run(Operation.java:257)
>
[jira] [Work logged] (HIVE-25326) Include partitioning info in DESCRIBE TABLE command on Iceberg tables
[
https://issues.apache.org/jira/browse/HIVE-25326?focusedWorklogId=621892=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-621892
]
ASF GitHub Bot logged work on HIVE-25326:
-
Author: ASF GitHub Bot
Created on: 13/Jul/21 09:37
Start Date: 13/Jul/21 09:37
Worklog Time Spent: 10m
Work Description: marton-bod commented on a change in pull request #2472:
URL: https://github.com/apache/hive/pull/2472#discussion_r668597490
##
File path:
iceberg/iceberg-handler/src/test/queries/positive/describe_iceberg_table.q
##
@@ -0,0 +1,22 @@
+DROP TABLE IF EXISTS ice_t;
+CREATE EXTERNAL TABLE ice_t (i int, s string, ts timestamp, d date) STORED BY
ICEBERG;
+
+DROP TABLE IF EXISTS ice_t_transform;
+CREATE EXTERNAL TABLE ice_t_transform (year_field date, month_field date,
day_field date, hour_field timestamp, truncate_field string, bucket_field int,
identity_field int) PARTITIONED BY SPEC (year(year_field), month(month_field),
day(day_field), hour(hour_field), truncate(2, truncate_field), bucket(2,
bucket_field), identity_field) STORED BY ICEBERG;
+
+DROP TABLE IF EXISTS ice_t_transform_prop;
+CREATE EXTERNAL TABLE ice_t_transform_prop (id int, year_field date,
month_field date, day_field date, hour_field timestamp, truncate_field string,
bucket_field int, identity_field int) STORED BY ICEBERG TBLPROPERTIES
('iceberg.mr.table.partition.spec'='{"spec-id":0,"fields":[{"name":"year_field_year","transform":"year","source-id":1,"field-id":1000},{"name":"month_field_month","transform":"month","source-id":2,"field-id":1001},{"name":"day_field_day","transform":"day","source-id":3,"field-id":1002},{"name":"hour_field_hour","transform":"hour","source-id":4,"field-id":1003},{"name":"truncate_field_trunc","transform":"truncate[2]","source-id":5,"field-id":1004},{"name":"bucket_field_bucket","transform":"bucket[2]","source-id":6,"field-id":1005},{"name":"identity_field","transform":"identity","source-id":7,"field-id":1006}]}');
+
+DROP TABLE IF EXISTS ice_t_identity_part;
+CREATE EXTERNAL TABLE ice_t_identity_part (a int) PARTITIONED BY (b string)
STORED BY ICEBERG;
+
+DESCRIBE FORMATTED ice_t;
+DESCRIBE FORMATTED ice_t_transform;
+DESCRIBE FORMATTED ice_t_transform_prop;
+DESCRIBE FORMATTED ice_t_identity_part;
+
+SET hive.ddl.output.format=json;
+DESCRIBE EXTENDED ice_t;
Review comment:
The json output of these are not supposed to show up in the q.out file?
Is it because `EXTENDED` is used here or because the output format is json?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 621892)
Time Spent: 20m (was: 10m)
> Include partitioning info in DESCRIBE TABLE command on Iceberg tables
> -
>
> Key: HIVE-25326
> URL: https://issues.apache.org/jira/browse/HIVE-25326
> Project: Hive
> Issue Type: Improvement
>Reporter: László Pintér
>Assignee: László Pintér
>Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-11369) Mapjoins in HiveServer2 fail when jmxremote is used
[
https://issues.apache.org/jira/browse/HIVE-11369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17379776#comment-17379776
]
louiechen commented on HIVE-11369:
--
Although this problem has been corrected, I still encounter the same problem in
future versions. What kind of information can I provide? Please help
hadoop 3.1.3 (https://hadoop.apache.org/release/3.1.3.html)
hive 3.1.2
([https://downloads.apache.org/hive/hive-3.1.2/apache-hive-3.1.2-bin.tar.gz)]
and my env enable kerberos and ranger hive plugin.
> Mapjoins in HiveServer2 fail when jmxremote is used
> ---
>
> Key: HIVE-11369
> URL: https://issues.apache.org/jira/browse/HIVE-11369
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Affects Versions: 1.1.0
> Environment: CDH 5.4.3, Centos 6.5, java version "1.7.0_67"
>Reporter: David Morel
>Assignee: loushang
>Priority: Major
> Fix For: 2.0.0
>
> Attachments: HIVE-11369.1.patch, HIVE-11369.2.patch
>
>
> having hive.auto.convert.join set to true works in the CLI with no issue, but
> fails in HiveServer2 when jmx options are passed to the service on startup.
> This (in hive-env.sh) is enough to make it fail:
> {noformat}
> -Dcom.sun.management.jmxremote
> -Dcom.sun.management.jmxremote.authenticate=false
> -Dcom.sun.management.jmxremote.ssl=false
> -Dcom.sun.management.jmxremote.port=8009
> {noformat}
> As soon as I remove the line, it works properly. I have *no*idea...
> Here's the log from the service:
> {noformat}
> 2015-07-24 17:19:27,457 INFO [HiveServer2-Handler-Pool: Thread-22]:
> ql.Driver (SessionState.java:printInfo(912)) - Query ID =
> hive_20150724171919_aaa88a89-dc6d-490b-821c-4eec6d4c0421
> 2015-07-24 17:19:27,457 INFO [HiveServer2-Handler-Pool: Thread-22]:
> ql.Driver (SessionState.java:printInfo(912)) - Total jobs = 1
> 2015-07-24 17:19:27,465 INFO [HiveServer2-Handler-Pool: Thread-22]:
> ql.Driver (Driver.java:launchTask(1638)) - Starting task
> [Stage-4:MAPREDLOCAL] in serial mode
> 2015-07-24 17:19:27,467 INFO [HiveServer2-Handler-Pool: Thread-22]:
> mr.MapredLocalTask (MapredLocalTask.java:executeInChildVM(159)) - Generating
> plan file
> file:/tmp/hive/8932c206-5420-4b6f-9f1f-5f1706f30df8/hive_2015-07-24_17-19-26_552_5082133674120283907-1/-local-10005/plan.xml
> 2015-07-24 17:19:27,625 WARN [HiveServer2-Handler-Pool: Thread-22]:
> conf.HiveConf (HiveConf.java:initialize(2620)) - HiveConf of name
> hive.files.umask.value does not exist
> 2015-07-24 17:19:27,708 INFO [HiveServer2-Handler-Pool: Thread-22]:
> mr.MapredLocalTask (MapredLocalTask.java:executeInChildVM(288)) - Executing:
> /usr/lib/hadoop/bin/hadoop jar
> /usr/lib/hive/lib/hive-common-1.1.0-cdh5.4.3.jar
> org.apache.hadoop.hive.ql.exec.mr.ExecDriver -localtask -plan
> file:/tmp/hive/8932c206-5420-4b6f-9f1f-5f1706f30df8/hive_2015-07-24_17-19-26_552_5082133674120283907-1/-local-10005/plan.xml
>-jobconffile
> file:/tmp/hive/8932c206-5420-4b6f-9f1f-5f1706f30df8/hive_2015-07-24_17-19-26_552_5082133674120283907-1/-local-10006/jobconf.xml
> 2015-07-24 17:19:28,499 ERROR [HiveServer2-Handler-Pool: Thread-22]:
> exec.Task (SessionState.java:printError(921)) - Execution failed with exit
> status: 1
> 2015-07-24 17:19:28,500 ERROR [HiveServer2-Handler-Pool: Thread-22]:
> exec.Task (SessionState.java:printError(921)) - Obtaining error information
> 2015-07-24 17:19:28,500 ERROR [HiveServer2-Handler-Pool: Thread-22]:
> exec.Task (SessionState.java:printError(921)) -
> Task failed!
> Task ID:
> Stage-4
> Logs:
> 2015-07-24 17:19:28,501 ERROR [HiveServer2-Handler-Pool: Thread-22]:
> exec.Task (SessionState.java:printError(921)) -
> /tmp/hiveserver2_manual/hive-server2.log
> 2015-07-24 17:19:28,501 ERROR [HiveServer2-Handler-Pool: Thread-22]:
> mr.MapredLocalTask (MapredLocalTask.java:executeInChildVM(308)) - Execution
> failed with exit status: 1
> 2015-07-24 17:19:28,518 ERROR [HiveServer2-Handler-Pool: Thread-22]:
> ql.Driver (SessionState.java:printError(921)) - FAILED: Execution Error,
> return code 1 from org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask
> 2015-07-24 17:19:28,599 WARN [HiveServer2-Handler-Pool: Thread-22]:
> security.UserGroupInformation (UserGroupInformation.java:doAs(1674)) -
> PriviledgedActionException as:hive (auth:SIMPLE)
> cause:org.apache.hive.service.cli.HiveSQLException: Error while processing
> statement: FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask
> 2015-07-24 17:19:28,600 WARN [HiveServer2-Handler-Pool: Thread-22]:
> thrift.ThriftCLIService (ThriftCLIService.java:ExecuteStatement(496)) - Error
> executing statement:
> org.apache.hive.service.cli.HiveSQLException: Error while processing
> statement: FAILED: Execution Error, return code 1 from
[jira] [Updated] (HIVE-25307) Hive Server 2 crashes when Thrift library encounters particular security protocol issue
[ https://issues.apache.org/jira/browse/HIVE-25307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-25307: Attachment: hive-thrift-fix2-01-3_1.patch > Hive Server 2 crashes when Thrift library encounters particular security > protocol issue > --- > > Key: HIVE-25307 > URL: https://issues.apache.org/jira/browse/HIVE-25307 > Project: Hive > Issue Type: Bug >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Labels: pull-request-available > Attachments: hive-thrift-fix2-01-3_1.patch > > Time Spent: 10m > Remaining Estimate: 0h > > A RuntimeException is thrown by the Thrift library that causes Hive Server 2 > to crash on our customer's machine. If you Google this the exception has been > reported a couple of times over the years but not fixed. A blog (see > references below) says it is an occasional security protocol issue between > Hive Server 2 and a proxy like a Gateway. > One challenge in the older 0.9.3 Thrift version was the Thrift > TTransportFactory getTransport method declaration had throws no Exceptions. > Hence the likely choice of RuntimeException. But that Exception is fatal to > Hive Server 2. > The proposed fix is a work around is we catch RuntimeException in the inner > class TUGIAssumingTransportFactory of the HadoopThriftAuthBridge class in > Hive Server 2. And throw a throw the RuntimeException's (inner) cause (e.g. > TSaslTransportException) as a TTransportException. > Once the Thrift library stops throwing RuntimeException or we catch fatal > Throwable exceptions in the Thrift library's TThreadPoolServer's inner class > WorkerProcess run method and display them, the RuntimeException try/catch > clause can be removed. > ExceptionClassName: > java.lang.RuntimeException > ExceptionStackTrace: > java.lang.RuntimeException: > org.apache.thrift.transport.TSaslTransportException: No data or no sasl data > in the stream > at > org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:219) > at > org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge.java:694) > at > org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge.java:691) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:360) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1710) > at > org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory.getTransport(HadoopThriftAuthBridge.java:691) > at > org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:269) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > Caused by: org.apache.thrift.transport.TSaslTransportException: No data or no > sasl data in the stream > at > org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:326) > at > org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:41) > at > org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:216) > ... 10 more > > References: > [Hive server 2 thrift error - Cloudera Community - > 34293|https://community.cloudera.com/t5/Support-Questions/Hive-server-2-thrift-error/td-p/34293] > Eric Lin blog "“NO DATA OR NO SASL DATA IN THE STREAM” ERROR IN HIVESERVER2 > LOG" > HIVE-12754 AuthTypes.NONE cause exception after HS2 start - ASF JIRA > (apache.org) > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25307) Hive Server 2 crashes when Thrift library encounters particular security protocol issue
[ https://issues.apache.org/jira/browse/HIVE-25307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-25307: Attachment: (was: hive-thrift-fix-02-3_1.patch) > Hive Server 2 crashes when Thrift library encounters particular security > protocol issue > --- > > Key: HIVE-25307 > URL: https://issues.apache.org/jira/browse/HIVE-25307 > Project: Hive > Issue Type: Bug >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Labels: pull-request-available > Attachments: hive-thrift-fix2-01-3_1.patch > > Time Spent: 10m > Remaining Estimate: 0h > > A RuntimeException is thrown by the Thrift library that causes Hive Server 2 > to crash on our customer's machine. If you Google this the exception has been > reported a couple of times over the years but not fixed. A blog (see > references below) says it is an occasional security protocol issue between > Hive Server 2 and a proxy like a Gateway. > One challenge in the older 0.9.3 Thrift version was the Thrift > TTransportFactory getTransport method declaration had throws no Exceptions. > Hence the likely choice of RuntimeException. But that Exception is fatal to > Hive Server 2. > The proposed fix is a work around is we catch RuntimeException in the inner > class TUGIAssumingTransportFactory of the HadoopThriftAuthBridge class in > Hive Server 2. And throw a throw the RuntimeException's (inner) cause (e.g. > TSaslTransportException) as a TTransportException. > Once the Thrift library stops throwing RuntimeException or we catch fatal > Throwable exceptions in the Thrift library's TThreadPoolServer's inner class > WorkerProcess run method and display them, the RuntimeException try/catch > clause can be removed. > ExceptionClassName: > java.lang.RuntimeException > ExceptionStackTrace: > java.lang.RuntimeException: > org.apache.thrift.transport.TSaslTransportException: No data or no sasl data > in the stream > at > org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:219) > at > org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge.java:694) > at > org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge.java:691) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:360) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1710) > at > org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory.getTransport(HadoopThriftAuthBridge.java:691) > at > org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:269) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > Caused by: org.apache.thrift.transport.TSaslTransportException: No data or no > sasl data in the stream > at > org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:326) > at > org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:41) > at > org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:216) > ... 10 more > > References: > [Hive server 2 thrift error - Cloudera Community - > 34293|https://community.cloudera.com/t5/Support-Questions/Hive-server-2-thrift-error/td-p/34293] > Eric Lin blog "“NO DATA OR NO SASL DATA IN THE STREAM” ERROR IN HIVESERVER2 > LOG" > HIVE-12754 AuthTypes.NONE cause exception after HS2 start - ASF JIRA > (apache.org) > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-25192) Nullify summary for non-native tables
[ https://issues.apache.org/jira/browse/HIVE-25192?focusedWorklogId=621861=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-621861 ] ASF GitHub Bot logged work on HIVE-25192: - Author: ASF GitHub Bot Created on: 13/Jul/21 07:40 Start Date: 13/Jul/21 07:40 Worklog Time Spent: 10m Work Description: dengzhhu653 opened a new pull request #2473: URL: https://github.com/apache/hive/pull/2473 ### What changes were proposed in this pull request? ### Why are the changes needed? ### Does this PR introduce _any_ user-facing change? ### How was this patch tested? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 621861) Time Spent: 50m (was: 40m) > Nullify summary for non-native tables > - > > Key: HIVE-25192 > URL: https://issues.apache.org/jira/browse/HIVE-25192 > Project: Hive > Issue Type: Bug >Reporter: Zhihua Deng >Assignee: Zhihua Deng >Priority: Major > Labels: pull-request-available > Time Spent: 50m > Remaining Estimate: 0h > > When creating non-native tables like kudu, hbase and so on, we create a > warehouse location for these tables, though these tables may not use the > location to store data or for job plan, we should skip getting the input > summary of non-native tables when optimising joins, as which may cause oom > problem when the non-native table is on the build side. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Reopened] (HIVE-25192) Nullify summary for non-native tables
[ https://issues.apache.org/jira/browse/HIVE-25192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihua Deng reopened HIVE-25192: > Nullify summary for non-native tables > - > > Key: HIVE-25192 > URL: https://issues.apache.org/jira/browse/HIVE-25192 > Project: Hive > Issue Type: Bug >Reporter: Zhihua Deng >Assignee: Zhihua Deng >Priority: Major > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > When creating non-native tables like kudu, hbase and so on, we create a > warehouse location for these tables, though these tables may not use the > location to store data or for job plan, we should skip getting the input > summary of non-native tables when optimising joins, as which may cause oom > problem when the non-native table is on the build side. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25192) Nullify summary for non-native tables
[ https://issues.apache.org/jira/browse/HIVE-25192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihua Deng updated HIVE-25192: --- Summary: Nullify summary for non-native tables (was: No need to create table directory for the non-native table) > Nullify summary for non-native tables > - > > Key: HIVE-25192 > URL: https://issues.apache.org/jira/browse/HIVE-25192 > Project: Hive > Issue Type: Bug >Reporter: Zhihua Deng >Assignee: Zhihua Deng >Priority: Major > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > When creating non-native tables like kudu, hbase and so on, we create a > warehouse location for these tables, though these tables may not use the > location to store data or for job plan, we should skip getting the input > summary of non-native tables when optimising joins, as which may cause oom > problem when the non-native table is on the build side. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-25315) SQL executed hiveserver is killed by the HiveServer2-Handler-Pool thread, very occasionally
[
https://issues.apache.org/jira/browse/HIVE-25315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17379664#comment-17379664
]
Zhihua Deng commented on HIVE-25315:
The instance is stopped due to some reasons, maybe caused by
[oom|https://github.com/apache/hive/blob/rel/release-3.1.0/service/src/java/org/apache/hive/service/cli/thrift/ThreadPoolExecutorWithOomHook.java#L51-L53]
, you can add some logs like HIVE-22037 to identify the problem.
> SQL executed hiveserver is killed by the HiveServer2-Handler-Pool thread,
> very occasionally
> ---
>
> Key: HIVE-25315
> URL: https://issues.apache.org/jira/browse/HIVE-25315
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Affects Versions: 3.1.0
>Reporter: GuangMing Lu
>Priority: Major
>
> {code:java}
> 2021-07-05 15:23:17,376 | INFO | HiveServer2-Handler-Pool: Thread-226765745
> | Shutting down HiveServer2 |
> org.apache.hive.service.server.HiveServer2.stop(HiveServer2.java:1090)
> 2021-07-05 15:23:17,376 | INFO | HiveServer2-Handler-Pool: Thread-226765745
> | Thrift server has stopped |
> org.apache.hive.service.cli.thrift.ThriftBinaryCLIService.stopServer(ThriftBinaryCLIService.java:225)
> 2021-07-05 15:23:17,376 | INFO | HiveServer2-Handler-Pool: Thread-226765745
> | Service:ThriftBinaryCLIService is stopped. |
> org.apache.hive.service.AbstractService.stop(AbstractService.java:130)
> 2021-07-05 15:23:17,376 | INFO | HiveServer2-Handler-Pool: Thread-226765745
> | Service:OperationManager is stopped. |
> org.apache.hive.service.AbstractService.stop(AbstractService.java:130)
> 2021-07-05 15:23:17,376 | INFO | HiveServer2-Handler-Pool: Thread-226765745
> | Service:SessionManager is stopped. |
> org.apache.hive.service.AbstractService.stop(AbstractService.java:130)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25326) Include partitioning info in DESCRIBE TABLE command on Iceberg tables
[ https://issues.apache.org/jira/browse/HIVE-25326?focusedWorklogId=621848=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-621848 ] ASF GitHub Bot logged work on HIVE-25326: - Author: ASF GitHub Bot Created on: 13/Jul/21 07:10 Start Date: 13/Jul/21 07:10 Worklog Time Spent: 10m Work Description: lcspinter opened a new pull request #2472: URL: https://github.com/apache/hive/pull/2472 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 621848) Remaining Estimate: 0h Time Spent: 10m > Include partitioning info in DESCRIBE TABLE command on Iceberg tables > - > > Key: HIVE-25326 > URL: https://issues.apache.org/jira/browse/HIVE-25326 > Project: Hive > Issue Type: Improvement >Reporter: László Pintér >Assignee: László Pintér >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25326) Include partitioning info in DESCRIBE TABLE command on Iceberg tables
[ https://issues.apache.org/jira/browse/HIVE-25326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-25326: -- Labels: pull-request-available (was: ) > Include partitioning info in DESCRIBE TABLE command on Iceberg tables > - > > Key: HIVE-25326 > URL: https://issues.apache.org/jira/browse/HIVE-25326 > Project: Hive > Issue Type: Improvement >Reporter: László Pintér >Assignee: László Pintér >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25192) No need to create table directory for the non-native table
[ https://issues.apache.org/jira/browse/HIVE-25192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihua Deng updated HIVE-25192: --- Description: When creating non-native tables like kudu, hbase and so on, we create a warehouse location for these tables, though these tables may not use the location to store data or for job plan, we should skip getting the input summary of non-native tables when optimising joins, as which may cause oom problem when the non-native table is on the build side. (was: When creating non-native tables like kudu, hbase and so on, we always create a warehouse location for these tables, though these tables may not use the location to store data or for job plan, so there is no need to create such location. We also should skip getting the input summary of non-native tables in some cases, this will avoid oom problem of building the hash table when the non-native table is on build side.) > No need to create table directory for the non-native table > -- > > Key: HIVE-25192 > URL: https://issues.apache.org/jira/browse/HIVE-25192 > Project: Hive > Issue Type: Bug >Reporter: Zhihua Deng >Assignee: Zhihua Deng >Priority: Major > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > When creating non-native tables like kudu, hbase and so on, we create a > warehouse location for these tables, though these tables may not use the > location to store data or for job plan, we should skip getting the input > summary of non-native tables when optimising joins, as which may cause oom > problem when the non-native table is on the build side. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-25326) Include partitioning info in DESCRIBE TABLE command on Iceberg tables
[ https://issues.apache.org/jira/browse/HIVE-25326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Pintér reassigned HIVE-25326: > Include partitioning info in DESCRIBE TABLE command on Iceberg tables > - > > Key: HIVE-25326 > URL: https://issues.apache.org/jira/browse/HIVE-25326 > Project: Hive > Issue Type: Improvement >Reporter: László Pintér >Assignee: László Pintér >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HIVE-25314) Implement SHOW CREATE TABLE command for Iceberg tables
[ https://issues.apache.org/jira/browse/HIVE-25314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Pintér resolved HIVE-25314. -- Resolution: Fixed > Implement SHOW CREATE TABLE command for Iceberg tables > -- > > Key: HIVE-25314 > URL: https://issues.apache.org/jira/browse/HIVE-25314 > Project: Hive > Issue Type: Improvement >Reporter: László Pintér >Assignee: László Pintér >Priority: Major > Labels: pull-request-available > Time Spent: 1h 50m > Remaining Estimate: 0h > > Implement SHOW CREATE TABLE, including the partitioning info as well in the > output (even though the HMS table is unpartitioned). That would probably > require loading the Iceberg table to acquire the partitioning information. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-25314) Implement SHOW CREATE TABLE command for Iceberg tables
[ https://issues.apache.org/jira/browse/HIVE-25314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17379659#comment-17379659 ] László Pintér commented on HIVE-25314: -- Merged into master. Thanks, [~mbod] for the review. > Implement SHOW CREATE TABLE command for Iceberg tables > -- > > Key: HIVE-25314 > URL: https://issues.apache.org/jira/browse/HIVE-25314 > Project: Hive > Issue Type: Improvement >Reporter: László Pintér >Assignee: László Pintér >Priority: Major > Labels: pull-request-available > Time Spent: 1h 50m > Remaining Estimate: 0h > > Implement SHOW CREATE TABLE, including the partitioning info as well in the > output (even though the HMS table is unpartitioned). That would probably > require loading the Iceberg table to acquire the partitioning information. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-25314) Implement SHOW CREATE TABLE command for Iceberg tables
[ https://issues.apache.org/jira/browse/HIVE-25314?focusedWorklogId=621845=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-621845 ] ASF GitHub Bot logged work on HIVE-25314: - Author: ASF GitHub Bot Created on: 13/Jul/21 06:50 Start Date: 13/Jul/21 06:50 Worklog Time Spent: 10m Work Description: lcspinter merged pull request #2458: URL: https://github.com/apache/hive/pull/2458 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 621845) Time Spent: 1h 50m (was: 1h 40m) > Implement SHOW CREATE TABLE command for Iceberg tables > -- > > Key: HIVE-25314 > URL: https://issues.apache.org/jira/browse/HIVE-25314 > Project: Hive > Issue Type: Improvement >Reporter: László Pintér >Assignee: László Pintér >Priority: Major > Labels: pull-request-available > Time Spent: 1h 50m > Remaining Estimate: 0h > > Implement SHOW CREATE TABLE, including the partitioning info as well in the > output (even though the HMS table is unpartitioned). That would probably > require loading the Iceberg table to acquire the partitioning information. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-25246) Fix the clean up of open repl created transactions
[
https://issues.apache.org/jira/browse/HIVE-25246?focusedWorklogId=621836=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-621836
]
ASF GitHub Bot logged work on HIVE-25246:
-
Author: ASF GitHub Bot
Created on: 13/Jul/21 06:10
Start Date: 13/Jul/21 06:10
Worklog Time Spent: 10m
Work Description: hmangla98 commented on a change in pull request #2396:
URL: https://github.com/apache/hive/pull/2396#discussion_r668454856
##
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/repl/ReplLoadTask.java
##
@@ -124,6 +124,13 @@ public int execute() {
try {
long loadTaskStartTime = System.currentTimeMillis();
SecurityUtils.reloginExpiringKeytabUser();
+ //Don't proceed if target db is replication incompatible.
+ if (work.dbNameToLoadIn != null) {
Review comment:
we have test cases to test REPL LOAD * scenarios.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 621836)
Time Spent: 4.5h (was: 4h 20m)
> Fix the clean up of open repl created transactions
> --
>
> Key: HIVE-25246
> URL: https://issues.apache.org/jira/browse/HIVE-25246
> Project: Hive
> Issue Type: Improvement
>Reporter: Haymant Mangla
>Assignee: Haymant Mangla
>Priority: Major
> Labels: pull-request-available
> Time Spent: 4.5h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25246) Fix the clean up of open repl created transactions
[
https://issues.apache.org/jira/browse/HIVE-25246?focusedWorklogId=621835=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-621835
]
ASF GitHub Bot logged work on HIVE-25246:
-
Author: ASF GitHub Bot
Created on: 13/Jul/21 06:09
Start Date: 13/Jul/21 06:09
Worklog Time Spent: 10m
Work Description: hmangla98 commented on a change in pull request #2396:
URL: https://github.com/apache/hive/pull/2396#discussion_r668454856
##
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/repl/ReplLoadTask.java
##
@@ -124,6 +124,13 @@ public int execute() {
try {
long loadTaskStartTime = System.currentTimeMillis();
SecurityUtils.reloginExpiringKeytabUser();
+ //Don't proceed if target db is replication incompatible.
+ if (work.dbNameToLoadIn != null) {
Review comment:
we have some cases which to test REPL LOAD * scenarios.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 621835)
Time Spent: 4h 20m (was: 4h 10m)
> Fix the clean up of open repl created transactions
> --
>
> Key: HIVE-25246
> URL: https://issues.apache.org/jira/browse/HIVE-25246
> Project: Hive
> Issue Type: Improvement
>Reporter: Haymant Mangla
>Assignee: Haymant Mangla
>Priority: Major
> Labels: pull-request-available
> Time Spent: 4h 20m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25246) Fix the clean up of open repl created transactions
[
https://issues.apache.org/jira/browse/HIVE-25246?focusedWorklogId=621834=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-621834
]
ASF GitHub Bot logged work on HIVE-25246:
-
Author: ASF GitHub Bot
Created on: 13/Jul/21 05:59
Start Date: 13/Jul/21 05:59
Worklog Time Spent: 10m
Work Description: pkumarsinha commented on a change in pull request #2396:
URL: https://github.com/apache/hive/pull/2396#discussion_r668449689
##
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/repl/ReplLoadTask.java
##
@@ -124,6 +124,13 @@ public int execute() {
try {
long loadTaskStartTime = System.currentTimeMillis();
SecurityUtils.reloginExpiringKeytabUser();
+ //Don't proceed if target db is replication incompatible.
+ if (work.dbNameToLoadIn != null) {
Review comment:
Do we know when could that be?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 621834)
Time Spent: 4h 10m (was: 4h)
> Fix the clean up of open repl created transactions
> --
>
> Key: HIVE-25246
> URL: https://issues.apache.org/jira/browse/HIVE-25246
> Project: Hive
> Issue Type: Improvement
>Reporter: Haymant Mangla
>Assignee: Haymant Mangla
>Priority: Major
> Labels: pull-request-available
> Time Spent: 4h 10m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
