[jira] [Assigned] (HIVE-25865) ALTER RENAME suppresses commitTransaction failure and reports operation success
[ https://issues.apache.org/jira/browse/HIVE-25865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline reassigned HIVE-25865: --- > ALTER RENAME suppresses commitTransaction failure and reports operation > success > --- > > Key: HIVE-25865 > URL: https://issues.apache.org/jira/browse/HIVE-25865 > Project: Hive > Issue Type: Bug > Components: Metastore >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > > If the Commit Tx fails, HiveAlterHandler,alterTable does not report an error. > It suppresses the issue and returns successfully. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Assigned] (HIVE-25493) TBLPROPERTIES upper- vs. lower-case confusion
[ https://issues.apache.org/jira/browse/HIVE-25493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline reassigned HIVE-25493: --- > TBLPROPERTIES upper- vs. lower-case confusion > - > > Key: HIVE-25493 > URL: https://issues.apache.org/jira/browse/HIVE-25493 > Project: Hive > Issue Type: Bug >Affects Versions: 3.1.2 >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > > User confused by ALTER TABLE SET PROPERTIES difference between > 'EXTERNAL'='FALSE' (ignored adds 2 properties EXTERNAL and FALSE) and > 'external'='false' (transaction error). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-25478) Temp file left over after ANALYZE TABLE .. COMPUTE STATISTICS FOR COLUMNS
[
https://issues.apache.org/jira/browse/HIVE-25478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt McCline reassigned HIVE-25478:
---
> Temp file left over after ANALYZE TABLE .. COMPUTE STATISTICS FOR COLUMNS
> -
>
> Key: HIVE-25478
> URL: https://issues.apache.org/jira/browse/HIVE-25478
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.0
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Major
>
> The dot staging file (".hive-staging") file is not removed at the end of the
> ANALYZE TABLE .. COMPUTE STATISTICS FOR COLUMNS operation as it is for say an
> INSERT that does automatic statistics collection. I expected it would be
> deleted after the Stats Work stage.
> Any ideas where in the code to add automatic deletion (hook)?
> hdfs dfs -ls /hive/warehouse/managed/table_orc
> Found 2 items
> drwxr-xr-x - hive supergroup 0 2021-08-24 17:19
> /hive/warehouse/managed/table_orc/.hive-staging_hive_2021-08-24_17-19-17_228_4856027533912221506-7
> drwxr-xr-x - hive supergroup 0 2021-08-24 07:17
> /hive/warehouse/managed/table_orc/delta_001_001_
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-25446) VectorMapJoinFastHashTable.validateCapacity AssertionError: Capacity must be a power of two
[ https://issues.apache.org/jira/browse/HIVE-25446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-25446: Description: Encountered this in a very large query: Caused by: java.lang.AssertionError: Capacity must be a power of two at org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.VectorMapJoinFastHashTable.validateCapacity(VectorMapJoinFastHashTable.java:60) at org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.VectorMapJoinFastHashTable.(VectorMapJoinFastHashTable.java:77) at org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.VectorMapJoinFastBytesHashTable.(VectorMapJoinFastBytesHashTable.java:132) at org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.VectorMapJoinFastBytesHashMap.(VectorMapJoinFastBytesHashMap.java:166) at org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.VectorMapJoinFastStringHashMap.(VectorMapJoinFastStringHashMap.java:43) at org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.VectorMapJoinFastTableContainer.createHashTable(VectorMapJoinFastTableContainer.java:137) at org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.VectorMapJoinFastTableContainer.(VectorMapJoinFastTableContainer.java:86) at org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.VectorMapJoinFastHashTableLoader.load(VectorMapJoinFastHashTableLoader.java:122) at org.apache.hadoop.hive.ql.exec.MapJoinOperator.loadHashTableInternal(MapJoinOperator.java:344) at org.apache.hadoop.hive.ql.exec.MapJoinOperator.loadHashTable(MapJoinOperator.java:413) at org.apache.hadoop.hive.ql.exec.MapJoinOperator.lambda$initializeOp$0(MapJoinOperator.java:215) at org.apache.hadoop.hive.ql.exec.tez.ObjectCache.retrieve(ObjectCache.java:96) at org.apache.hadoop.hive.ql.exec.tez.ObjectCache$1.call(ObjectCache.java:113) at java.util.concurrent.FutureTask.run(FutureTask.java:266) > VectorMapJoinFastHashTable.validateCapacity AssertionError: Capacity must be > a power of two > --- > > Key: HIVE-25446 > URL: https://issues.apache.org/jira/browse/HIVE-25446 > Project: Hive > Issue Type: Bug >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > Fix For: 4.0.0 > > > Encountered this in a very large query: > Caused by: java.lang.AssertionError: Capacity must be a power of two > at > org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.VectorMapJoinFastHashTable.validateCapacity(VectorMapJoinFastHashTable.java:60) > at > org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.VectorMapJoinFastHashTable.(VectorMapJoinFastHashTable.java:77) > at > org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.VectorMapJoinFastBytesHashTable.(VectorMapJoinFastBytesHashTable.java:132) > at > org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.VectorMapJoinFastBytesHashMap.(VectorMapJoinFastBytesHashMap.java:166) > at > org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.VectorMapJoinFastStringHashMap.(VectorMapJoinFastStringHashMap.java:43) > at > org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.VectorMapJoinFastTableContainer.createHashTable(VectorMapJoinFastTableContainer.java:137) > at > org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.VectorMapJoinFastTableContainer.(VectorMapJoinFastTableContainer.java:86) > at > org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.VectorMapJoinFastHashTableLoader.load(VectorMapJoinFastHashTableLoader.java:122) > at > org.apache.hadoop.hive.ql.exec.MapJoinOperator.loadHashTableInternal(MapJoinOperator.java:344) > at > org.apache.hadoop.hive.ql.exec.MapJoinOperator.loadHashTable(MapJoinOperator.java:413) > at > org.apache.hadoop.hive.ql.exec.MapJoinOperator.lambda$initializeOp$0(MapJoinOperator.java:215) > at > org.apache.hadoop.hive.ql.exec.tez.ObjectCache.retrieve(ObjectCache.java:96) > at > org.apache.hadoop.hive.ql.exec.tez.ObjectCache$1.call(ObjectCache.java:113) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25446) VectorMapJoinFastHashTable.validateCapacity AssertionError: Capacity must be a power of two
[ https://issues.apache.org/jira/browse/HIVE-25446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-25446: Environment: (was: Encountered this in a very large query: Caused by: java.lang.AssertionError: Capacity must be a power of two at org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.VectorMapJoinFastHashTable.validateCapacity(VectorMapJoinFastHashTable.java:60) at org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.VectorMapJoinFastHashTable.(VectorMapJoinFastHashTable.java:77) at org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.VectorMapJoinFastBytesHashTable.(VectorMapJoinFastBytesHashTable.java:132) at org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.VectorMapJoinFastBytesHashMap.(VectorMapJoinFastBytesHashMap.java:166) at org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.VectorMapJoinFastStringHashMap.(VectorMapJoinFastStringHashMap.java:43) at org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.VectorMapJoinFastTableContainer.createHashTable(VectorMapJoinFastTableContainer.java:137) at org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.VectorMapJoinFastTableContainer.(VectorMapJoinFastTableContainer.java:86) at org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.VectorMapJoinFastHashTableLoader.load(VectorMapJoinFastHashTableLoader.java:122) at org.apache.hadoop.hive.ql.exec.MapJoinOperator.loadHashTableInternal(MapJoinOperator.java:344) at org.apache.hadoop.hive.ql.exec.MapJoinOperator.loadHashTable(MapJoinOperator.java:413) at org.apache.hadoop.hive.ql.exec.MapJoinOperator.lambda$initializeOp$0(MapJoinOperator.java:215) at org.apache.hadoop.hive.ql.exec.tez.ObjectCache.retrieve(ObjectCache.java:96) at org.apache.hadoop.hive.ql.exec.tez.ObjectCache$1.call(ObjectCache.java:113) at java.util.concurrent.FutureTask.run(FutureTask.java:266)) > VectorMapJoinFastHashTable.validateCapacity AssertionError: Capacity must be > a power of two > --- > > Key: HIVE-25446 > URL: https://issues.apache.org/jira/browse/HIVE-25446 > Project: Hive > Issue Type: Bug >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > Fix For: 4.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-25446) VectorMapJoinFastHashTable.validateCapacity AssertionError: Capacity must be a power of two
[ https://issues.apache.org/jira/browse/HIVE-25446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline reassigned HIVE-25446: --- > VectorMapJoinFastHashTable.validateCapacity AssertionError: Capacity must be > a power of two > --- > > Key: HIVE-25446 > URL: https://issues.apache.org/jira/browse/HIVE-25446 > Project: Hive > Issue Type: Bug > Environment: Encountered this in a very large query: > Caused by: java.lang.AssertionError: Capacity must be a power of two > at > org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.VectorMapJoinFastHashTable.validateCapacity(VectorMapJoinFastHashTable.java:60) > at > org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.VectorMapJoinFastHashTable.(VectorMapJoinFastHashTable.java:77) > at > org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.VectorMapJoinFastBytesHashTable.(VectorMapJoinFastBytesHashTable.java:132) > at > org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.VectorMapJoinFastBytesHashMap.(VectorMapJoinFastBytesHashMap.java:166) > at > org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.VectorMapJoinFastStringHashMap.(VectorMapJoinFastStringHashMap.java:43) > at > org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.VectorMapJoinFastTableContainer.createHashTable(VectorMapJoinFastTableContainer.java:137) > at > org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.VectorMapJoinFastTableContainer.(VectorMapJoinFastTableContainer.java:86) > at > org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.VectorMapJoinFastHashTableLoader.load(VectorMapJoinFastHashTableLoader.java:122) > at > org.apache.hadoop.hive.ql.exec.MapJoinOperator.loadHashTableInternal(MapJoinOperator.java:344) > at > org.apache.hadoop.hive.ql.exec.MapJoinOperator.loadHashTable(MapJoinOperator.java:413) > at > org.apache.hadoop.hive.ql.exec.MapJoinOperator.lambda$initializeOp$0(MapJoinOperator.java:215) > at > org.apache.hadoop.hive.ql.exec.tez.ObjectCache.retrieve(ObjectCache.java:96) > at > org.apache.hadoop.hive.ql.exec.tez.ObjectCache$1.call(ObjectCache.java:113) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > Fix For: 4.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25396) Improve Exception handling in Hive Server 2
[ https://issues.apache.org/jira/browse/HIVE-25396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-25396: Attachment: (was: hive-improve-exception-handling-03-master.patch) > Improve Exception handling in Hive Server 2 > --- > > Key: HIVE-25396 > URL: https://issues.apache.org/jira/browse/HIVE-25396 > Project: Hive > Issue Type: Bug >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > Labels: pull-request-available > Attachments: hive-improve-exception-handling-04-master.patch > > Time Spent: 1h 20m > Remaining Estimate: 0h > > Hive's org.apache.hive.service.thrift.ThriftHttpServlet.doPost method does > not handle all Exception kinds. This leaves uncaught Exception handling > choices to the Jetty HTTP library. We fix that. > Also, a Thread.UncaughtExceptionHandler is added to Hive Server 2 so uncaught > Exception are handled uniformly, including making them logged and not just > printed to stderr. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25396) Improve Exception handling in Hive Server 2
[ https://issues.apache.org/jira/browse/HIVE-25396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-25396: Attachment: hive-improve-exception-handling-04-master.patch > Improve Exception handling in Hive Server 2 > --- > > Key: HIVE-25396 > URL: https://issues.apache.org/jira/browse/HIVE-25396 > Project: Hive > Issue Type: Bug >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > Labels: pull-request-available > Attachments: hive-improve-exception-handling-03-master.patch, > hive-improve-exception-handling-04-master.patch > > Time Spent: 1h 20m > Remaining Estimate: 0h > > Hive's org.apache.hive.service.thrift.ThriftHttpServlet.doPost method does > not handle all Exception kinds. This leaves uncaught Exception handling > choices to the Jetty HTTP library. We fix that. > Also, a Thread.UncaughtExceptionHandler is added to Hive Server 2 so uncaught > Exception are handled uniformly, including making them logged and not just > printed to stderr. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25396) Improve Exception handling in Hive Server 2
[ https://issues.apache.org/jira/browse/HIVE-25396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-25396: Attachment: hive-improve-exception-handling-03-master.patch > Improve Exception handling in Hive Server 2 > --- > > Key: HIVE-25396 > URL: https://issues.apache.org/jira/browse/HIVE-25396 > Project: Hive > Issue Type: Bug >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > Labels: pull-request-available > Attachments: hive-improve-exception-handling-03-master.patch > > Time Spent: 1h 10m > Remaining Estimate: 0h > > Hive's org.apache.hive.service.thrift.ThriftHttpServlet.doPost method does > not handle all Exception kinds. This leaves uncaught Exception handling > choices to the Jetty HTTP library. We fix that. > Also, a Thread.UncaughtExceptionHandler is added to Hive Server 2 so uncaught > Exception are handled uniformly, including making them logged and not just > printed to stderr. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25396) Improve Exception handling in Hive Server 2
[ https://issues.apache.org/jira/browse/HIVE-25396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-25396: Attachment: (was: hive-improve-exception-handling-03-master.patch) > Improve Exception handling in Hive Server 2 > --- > > Key: HIVE-25396 > URL: https://issues.apache.org/jira/browse/HIVE-25396 > Project: Hive > Issue Type: Bug >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > Labels: pull-request-available > Attachments: hive-improve-exception-handling-03-master.patch > > Time Spent: 1h 10m > Remaining Estimate: 0h > > Hive's org.apache.hive.service.thrift.ThriftHttpServlet.doPost method does > not handle all Exception kinds. This leaves uncaught Exception handling > choices to the Jetty HTTP library. We fix that. > Also, a Thread.UncaughtExceptionHandler is added to Hive Server 2 so uncaught > Exception are handled uniformly, including making them logged and not just > printed to stderr. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25396) Improve Exception handling in Hive Server 2
[ https://issues.apache.org/jira/browse/HIVE-25396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-25396: Attachment: hive-improve-exception-handling-03-master.patch > Improve Exception handling in Hive Server 2 > --- > > Key: HIVE-25396 > URL: https://issues.apache.org/jira/browse/HIVE-25396 > Project: Hive > Issue Type: Bug >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > Labels: pull-request-available > Attachments: hive-improve-exception-handling-03-master.patch > > Time Spent: 1h 10m > Remaining Estimate: 0h > > Hive's org.apache.hive.service.thrift.ThriftHttpServlet.doPost method does > not handle all Exception kinds. This leaves uncaught Exception handling > choices to the Jetty HTTP library. We fix that. > Also, a Thread.UncaughtExceptionHandler is added to Hive Server 2 so uncaught > Exception are handled uniformly, including making them logged and not just > printed to stderr. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25396) Improve Exception handling in Hive Server 2
[ https://issues.apache.org/jira/browse/HIVE-25396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-25396: Attachment: (was: hive-improve-exception-handling-03-master.patch) > Improve Exception handling in Hive Server 2 > --- > > Key: HIVE-25396 > URL: https://issues.apache.org/jira/browse/HIVE-25396 > Project: Hive > Issue Type: Bug >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > Labels: pull-request-available > Attachments: hive-improve-exception-handling-03-master.patch > > Time Spent: 1h > Remaining Estimate: 0h > > Hive's org.apache.hive.service.thrift.ThriftHttpServlet.doPost method does > not handle all Exception kinds. This leaves uncaught Exception handling > choices to the Jetty HTTP library. We fix that. > Also, a Thread.UncaughtExceptionHandler is added to Hive Server 2 so uncaught > Exception are handled uniformly, including making them logged and not just > printed to stderr. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25396) Improve Exception handling in Hive Server 2
[ https://issues.apache.org/jira/browse/HIVE-25396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-25396: Attachment: (was: hive-improve-exception-handling-02-master.patch) > Improve Exception handling in Hive Server 2 > --- > > Key: HIVE-25396 > URL: https://issues.apache.org/jira/browse/HIVE-25396 > Project: Hive > Issue Type: Bug >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > Labels: pull-request-available > Attachments: hive-improve-exception-handling-03-master.patch > > Time Spent: 1h > Remaining Estimate: 0h > > Hive's org.apache.hive.service.thrift.ThriftHttpServlet.doPost method does > not handle all Exception kinds. This leaves uncaught Exception handling > choices to the Jetty HTTP library. We fix that. > Also, a Thread.UncaughtExceptionHandler is added to Hive Server 2 so uncaught > Exception are handled uniformly, including making them logged and not just > printed to stderr. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25396) Improve Exception handling in Hive Server 2
[ https://issues.apache.org/jira/browse/HIVE-25396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-25396: Attachment: (was: hive-uncaught-http-servlet-01-3_1.patch) > Improve Exception handling in Hive Server 2 > --- > > Key: HIVE-25396 > URL: https://issues.apache.org/jira/browse/HIVE-25396 > Project: Hive > Issue Type: Bug >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > Labels: pull-request-available > Attachments: hive-improve-exception-handling-03-master.patch > > Time Spent: 1h > Remaining Estimate: 0h > > Hive's org.apache.hive.service.thrift.ThriftHttpServlet.doPost method does > not handle all Exception kinds. This leaves uncaught Exception handling > choices to the Jetty HTTP library. We fix that. > Also, a Thread.UncaughtExceptionHandler is added to Hive Server 2 so uncaught > Exception are handled uniformly, including making them logged and not just > printed to stderr. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25396) Improve Exception handling in Hive Server 2
[ https://issues.apache.org/jira/browse/HIVE-25396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-25396: Attachment: (was: hive-improve-exception-handling-03-master.patch) > Improve Exception handling in Hive Server 2 > --- > > Key: HIVE-25396 > URL: https://issues.apache.org/jira/browse/HIVE-25396 > Project: Hive > Issue Type: Bug >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > Labels: pull-request-available > Attachments: hive-improve-exception-handling-02-master.patch, > hive-improve-exception-handling-03-master.patch, > hive-uncaught-http-servlet-01-3_1.patch > > Time Spent: 40m > Remaining Estimate: 0h > > Hive's org.apache.hive.service.thrift.ThriftHttpServlet.doPost method does > not handle all Exception kinds. This leaves uncaught Exception handling > choices to the Jetty HTTP library. We fix that. > Also, a Thread.UncaughtExceptionHandler is added to Hive Server 2 so uncaught > Exception are handled uniformly, including making them logged and not just > printed to stderr. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25396) Improve Exception handling in Hive Server 2
[ https://issues.apache.org/jira/browse/HIVE-25396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-25396: Attachment: hive-improve-exception-handling-03-master.patch > Improve Exception handling in Hive Server 2 > --- > > Key: HIVE-25396 > URL: https://issues.apache.org/jira/browse/HIVE-25396 > Project: Hive > Issue Type: Bug >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > Labels: pull-request-available > Attachments: hive-improve-exception-handling-02-master.patch, > hive-improve-exception-handling-03-master.patch, > hive-uncaught-http-servlet-01-3_1.patch > > Time Spent: 40m > Remaining Estimate: 0h > > Hive's org.apache.hive.service.thrift.ThriftHttpServlet.doPost method does > not handle all Exception kinds. This leaves uncaught Exception handling > choices to the Jetty HTTP library. We fix that. > Also, a Thread.UncaughtExceptionHandler is added to Hive Server 2 so uncaught > Exception are handled uniformly, including making them logged and not just > printed to stderr. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25396) Improve Exception handling in Hive Server 2
[ https://issues.apache.org/jira/browse/HIVE-25396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-25396: Attachment: hive-improve-exception-handling-03-master.patch > Improve Exception handling in Hive Server 2 > --- > > Key: HIVE-25396 > URL: https://issues.apache.org/jira/browse/HIVE-25396 > Project: Hive > Issue Type: Bug >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > Labels: pull-request-available > Attachments: hive-improve-exception-handling-02-master.patch, > hive-improve-exception-handling-03-master.patch, > hive-uncaught-http-servlet-01-3_1.patch > > Time Spent: 40m > Remaining Estimate: 0h > > Hive's org.apache.hive.service.thrift.ThriftHttpServlet.doPost method does > not handle all Exception kinds. This leaves uncaught Exception handling > choices to the Jetty HTTP library. We fix that. > Also, a Thread.UncaughtExceptionHandler is added to Hive Server 2 so uncaught > Exception are handled uniformly, including making them logged and not just > printed to stderr. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25396) Improve Exception handling in Hive Server 2
[ https://issues.apache.org/jira/browse/HIVE-25396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-25396: Attachment: (was: hive-improve-exception-handling-03-master.patch) > Improve Exception handling in Hive Server 2 > --- > > Key: HIVE-25396 > URL: https://issues.apache.org/jira/browse/HIVE-25396 > Project: Hive > Issue Type: Bug >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > Labels: pull-request-available > Attachments: hive-improve-exception-handling-02-master.patch, > hive-improve-exception-handling-03-master.patch, > hive-uncaught-http-servlet-01-3_1.patch > > Time Spent: 40m > Remaining Estimate: 0h > > Hive's org.apache.hive.service.thrift.ThriftHttpServlet.doPost method does > not handle all Exception kinds. This leaves uncaught Exception handling > choices to the Jetty HTTP library. We fix that. > Also, a Thread.UncaughtExceptionHandler is added to Hive Server 2 so uncaught > Exception are handled uniformly, including making them logged and not just > printed to stderr. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25396) Improve Exception handling in Hive Server 2
[ https://issues.apache.org/jira/browse/HIVE-25396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-25396: Attachment: (was: hive-improve-exception-handling-03-master.patch) > Improve Exception handling in Hive Server 2 > --- > > Key: HIVE-25396 > URL: https://issues.apache.org/jira/browse/HIVE-25396 > Project: Hive > Issue Type: Bug >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > Labels: pull-request-available > Attachments: hive-improve-exception-handling-02-master.patch, > hive-improve-exception-handling-03-master.patch, > hive-uncaught-http-servlet-01-3_1.patch > > Time Spent: 0.5h > Remaining Estimate: 0h > > Hive's org.apache.hive.service.thrift.ThriftHttpServlet.doPost method does > not handle all Exception kinds. This leaves uncaught Exception handling > choices to the Jetty HTTP library. We fix that. > Also, a Thread.UncaughtExceptionHandler is added to Hive Server 2 so uncaught > Exception are handled uniformly, including making them logged and not just > printed to stderr. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25396) Improve Exception handling in Hive Server 2
[ https://issues.apache.org/jira/browse/HIVE-25396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-25396: Attachment: hive-improve-exception-handling-03-master.patch > Improve Exception handling in Hive Server 2 > --- > > Key: HIVE-25396 > URL: https://issues.apache.org/jira/browse/HIVE-25396 > Project: Hive > Issue Type: Bug >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > Labels: pull-request-available > Attachments: hive-improve-exception-handling-02-master.patch, > hive-improve-exception-handling-03-master.patch, > hive-uncaught-http-servlet-01-3_1.patch > > Time Spent: 0.5h > Remaining Estimate: 0h > > Hive's org.apache.hive.service.thrift.ThriftHttpServlet.doPost method does > not handle all Exception kinds. This leaves uncaught Exception handling > choices to the Jetty HTTP library. We fix that. > Also, a Thread.UncaughtExceptionHandler is added to Hive Server 2 so uncaught > Exception are handled uniformly, including making them logged and not just > printed to stderr. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25396) Improve Exception handling in Hive Server 2
[ https://issues.apache.org/jira/browse/HIVE-25396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-25396: Attachment: hive-improve-exception-handling-03-master.patch > Improve Exception handling in Hive Server 2 > --- > > Key: HIVE-25396 > URL: https://issues.apache.org/jira/browse/HIVE-25396 > Project: Hive > Issue Type: Bug >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > Labels: pull-request-available > Attachments: hive-improve-exception-handling-02-master.patch, > hive-improve-exception-handling-03-master.patch, > hive-uncaught-http-servlet-01-3_1.patch > > Time Spent: 20m > Remaining Estimate: 0h > > Hive's org.apache.hive.service.thrift.ThriftHttpServlet.doPost method does > not handle all Exception kinds. This leaves uncaught Exception handling > choices to the Jetty HTTP library. We fix that. > Also, a Thread.UncaughtExceptionHandler is added to Hive Server 2 so uncaught > Exception are handled uniformly, including making them logged and not just > printed to stderr. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25396) Improve Exception handling in Hive Server 2
[ https://issues.apache.org/jira/browse/HIVE-25396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-25396: Attachment: (was: hive-improve-exception-handling-03-master.patch) > Improve Exception handling in Hive Server 2 > --- > > Key: HIVE-25396 > URL: https://issues.apache.org/jira/browse/HIVE-25396 > Project: Hive > Issue Type: Bug >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > Labels: pull-request-available > Attachments: hive-improve-exception-handling-02-master.patch, > hive-uncaught-http-servlet-01-3_1.patch > > Time Spent: 20m > Remaining Estimate: 0h > > Hive's org.apache.hive.service.thrift.ThriftHttpServlet.doPost method does > not handle all Exception kinds. This leaves uncaught Exception handling > choices to the Jetty HTTP library. We fix that. > Also, a Thread.UncaughtExceptionHandler is added to Hive Server 2 so uncaught > Exception are handled uniformly, including making them logged and not just > printed to stderr. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25396) Improve Exception handling in Hive Server 2
[ https://issues.apache.org/jira/browse/HIVE-25396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-25396: Attachment: hive-improve-exception-handling-03-master.patch > Improve Exception handling in Hive Server 2 > --- > > Key: HIVE-25396 > URL: https://issues.apache.org/jira/browse/HIVE-25396 > Project: Hive > Issue Type: Bug >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > Labels: pull-request-available > Attachments: hive-improve-exception-handling-02-master.patch, > hive-improve-exception-handling-03-master.patch, > hive-uncaught-http-servlet-01-3_1.patch > > Time Spent: 20m > Remaining Estimate: 0h > > Hive's org.apache.hive.service.thrift.ThriftHttpServlet.doPost method does > not handle all Exception kinds. This leaves uncaught Exception handling > choices to the Jetty HTTP library. We fix that. > Also, a Thread.UncaughtExceptionHandler is added to Hive Server 2 so uncaught > Exception are handled uniformly, including making them logged and not just > printed to stderr. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25396) Improve Exception handling in Hive Server 2
[ https://issues.apache.org/jira/browse/HIVE-25396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-25396: Attachment: hive-improve-exception-handling-02-master.patch > Improve Exception handling in Hive Server 2 > --- > > Key: HIVE-25396 > URL: https://issues.apache.org/jira/browse/HIVE-25396 > Project: Hive > Issue Type: Bug >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > Labels: pull-request-available > Attachments: hive-improve-exception-handling-02-master.patch, > hive-uncaught-http-servlet-01-3_1.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Hive's org.apache.hive.service.thrift.ThriftHttpServlet.doPost method does > not handle all Exception kinds. This leaves uncaught Exception handling > choices to the Jetty HTTP library. We fix that. > Also, a Thread.UncaughtExceptionHandler is added to Hive Server 2 so uncaught > Exception are handled uniformly, including making them logged and not just > printed to stderr. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25396) Improve Exception handling in Hive Server 2
[ https://issues.apache.org/jira/browse/HIVE-25396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-25396: Attachment: (was: hive-improve-exception-handling-02-master.patch) > Improve Exception handling in Hive Server 2 > --- > > Key: HIVE-25396 > URL: https://issues.apache.org/jira/browse/HIVE-25396 > Project: Hive > Issue Type: Bug >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > Labels: pull-request-available > Attachments: hive-improve-exception-handling-02-master.patch, > hive-uncaught-http-servlet-01-3_1.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Hive's org.apache.hive.service.thrift.ThriftHttpServlet.doPost method does > not handle all Exception kinds. This leaves uncaught Exception handling > choices to the Jetty HTTP library. We fix that. > Also, a Thread.UncaughtExceptionHandler is added to Hive Server 2 so uncaught > Exception are handled uniformly, including making them logged and not just > printed to stderr. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25396) Improve Exception handling in Hive Server 2
[ https://issues.apache.org/jira/browse/HIVE-25396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-25396: Attachment: hive-improve-exception-handling-02-master.patch > Improve Exception handling in Hive Server 2 > --- > > Key: HIVE-25396 > URL: https://issues.apache.org/jira/browse/HIVE-25396 > Project: Hive > Issue Type: Bug >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > Labels: pull-request-available > Attachments: hive-improve-exception-handling-02-master.patch, > hive-uncaught-http-servlet-01-3_1.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Hive's org.apache.hive.service.thrift.ThriftHttpServlet.doPost method does > not handle all Exception kinds. This leaves uncaught Exception handling > choices to the Jetty HTTP library. We fix that. > Also, a Thread.UncaughtExceptionHandler is added to Hive Server 2 so uncaught > Exception are handled uniformly, including making them logged and not just > printed to stderr. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25396) Improve Exception handling in Hive Server 2
[ https://issues.apache.org/jira/browse/HIVE-25396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-25396: Attachment: (was: hive-improve-exception-handling-02-master.patch) > Improve Exception handling in Hive Server 2 > --- > > Key: HIVE-25396 > URL: https://issues.apache.org/jira/browse/HIVE-25396 > Project: Hive > Issue Type: Bug >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > Labels: pull-request-available > Attachments: hive-uncaught-http-servlet-01-3_1.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Hive's org.apache.hive.service.thrift.ThriftHttpServlet.doPost method does > not handle all Exception kinds. This leaves uncaught Exception handling > choices to the Jetty HTTP library. We fix that. > Also, a Thread.UncaughtExceptionHandler is added to Hive Server 2 so uncaught > Exception are handled uniformly, including making them logged and not just > printed to stderr. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25396) Improve Exception handling in Hive Server 2
[ https://issues.apache.org/jira/browse/HIVE-25396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-25396: Attachment: hive-improve-exception-handling-02-master.patch > Improve Exception handling in Hive Server 2 > --- > > Key: HIVE-25396 > URL: https://issues.apache.org/jira/browse/HIVE-25396 > Project: Hive > Issue Type: Bug >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > Labels: pull-request-available > Attachments: hive-improve-exception-handling-02-master.patch, > hive-uncaught-http-servlet-01-3_1.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Hive's org.apache.hive.service.thrift.ThriftHttpServlet.doPost method does > not handle all Exception kinds. This leaves uncaught Exception handling > choices to the Jetty HTTP library. We fix that. > Also, a Thread.UncaughtExceptionHandler is added to Hive Server 2 so uncaught > Exception are handled uniformly, including making them logged and not just > printed to stderr. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25396) Improve Exception handling in Hive Server 2
[ https://issues.apache.org/jira/browse/HIVE-25396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-25396: Summary: Improve Exception handling in Hive Server 2 (was: Improve uncaught Thread Exception handling in Hive Server 2) > Improve Exception handling in Hive Server 2 > --- > > Key: HIVE-25396 > URL: https://issues.apache.org/jira/browse/HIVE-25396 > Project: Hive > Issue Type: Bug >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > Labels: pull-request-available > Attachments: hive-uncaught-http-servlet-01-3_1.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Hive's org.apache.hive.service.thrift.ThriftHttpServlet.doPost method does > not handle all Exception kinds. This leaves uncaught Exception handling > choices to the Jetty HTTP library. We fix that. > Also, a Thread.UncaughtExceptionHandler is added to Hive Server 2 so uncaught > Exception are handled uniformly, including making them logged and not just > printed to stderr. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (HIVE-25069) Hive Distributed Tracing
[ https://issues.apache.org/jira/browse/HIVE-25069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17390901#comment-17390901 ] Matt McCline edited comment on HIVE-25069 at 7/31/21, 11:55 PM: Rethink – closing. was (Author: mattmccline): Rethink. > Hive Distributed Tracing > > > Key: HIVE-25069 > URL: https://issues.apache.org/jira/browse/HIVE-25069 > Project: Hive > Issue Type: New Feature >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > > Instrument Hive code to gather distributed traces and export trace data to a > configurable collector. > Distributed tracing is a revolutionary tool for debugging issues. > We will use the new OpenTelemetry open-source standard that our industry has > aligned on. OpenTelemetry is the merger of two earlier distributed tracing > projects OpenTracing and OpenCensus. > Next step: Add design document that goes into more detail on the benefits of > distributed tracing and describes how Hive will enhanced. > Also see: > HBASE-22120 Replace HTrace with OpenTelemetry -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Issue Comment Deleted] (HIVE-25069) Hive Distributed Tracing
[ https://issues.apache.org/jira/browse/HIVE-25069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-25069: Comment: was deleted (was: FYI: [~ashutoshc] [~thejas] [~jcamachorodriguez] [~gopalv] [~abstractdog] [~kgyrtkirk] [~sershe] [~zhangduo] [~weichiu] [~Shkuro] [~zghao]) > Hive Distributed Tracing > > > Key: HIVE-25069 > URL: https://issues.apache.org/jira/browse/HIVE-25069 > Project: Hive > Issue Type: New Feature >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > > Instrument Hive code to gather distributed traces and export trace data to a > configurable collector. > Distributed tracing is a revolutionary tool for debugging issues. > We will use the new OpenTelemetry open-source standard that our industry has > aligned on. OpenTelemetry is the merger of two earlier distributed tracing > projects OpenTracing and OpenCensus. > Next step: Add design document that goes into more detail on the benefits of > distributed tracing and describes how Hive will enhanced. > Also see: > HBASE-22120 Replace HTrace with OpenTelemetry -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HIVE-25069) Hive Distributed Tracing
[ https://issues.apache.org/jira/browse/HIVE-25069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline resolved HIVE-25069. - Resolution: Won't Fix > Hive Distributed Tracing > > > Key: HIVE-25069 > URL: https://issues.apache.org/jira/browse/HIVE-25069 > Project: Hive > Issue Type: New Feature >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > > Instrument Hive code to gather distributed traces and export trace data to a > configurable collector. > Distributed tracing is a revolutionary tool for debugging issues. > We will use the new OpenTelemetry open-source standard that our industry has > aligned on. OpenTelemetry is the merger of two earlier distributed tracing > projects OpenTracing and OpenCensus. > Next step: Add design document that goes into more detail on the benefits of > distributed tracing and describes how Hive will enhanced. > Also see: > HBASE-22120 Replace HTrace with OpenTelemetry -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-25069) Hive Distributed Tracing
[ https://issues.apache.org/jira/browse/HIVE-25069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17390901#comment-17390901 ] Matt McCline commented on HIVE-25069: - Rethink. > Hive Distributed Tracing > > > Key: HIVE-25069 > URL: https://issues.apache.org/jira/browse/HIVE-25069 > Project: Hive > Issue Type: New Feature >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > > Instrument Hive code to gather distributed traces and export trace data to a > configurable collector. > Distributed tracing is a revolutionary tool for debugging issues. > We will use the new OpenTelemetry open-source standard that our industry has > aligned on. OpenTelemetry is the merger of two earlier distributed tracing > projects OpenTracing and OpenCensus. > Next step: Add design document that goes into more detail on the benefits of > distributed tracing and describes how Hive will enhanced. > Also see: > HBASE-22120 Replace HTrace with OpenTelemetry -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Issue Comment Deleted] (HIVE-25069) Hive Distributed Tracing
[ https://issues.apache.org/jira/browse/HIVE-25069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-25069: Comment: was deleted (was: !image-2021-05-10-19-06-02-679.png!) > Hive Distributed Tracing > > > Key: HIVE-25069 > URL: https://issues.apache.org/jira/browse/HIVE-25069 > Project: Hive > Issue Type: New Feature >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > > Instrument Hive code to gather distributed traces and export trace data to a > configurable collector. > Distributed tracing is a revolutionary tool for debugging issues. > We will use the new OpenTelemetry open-source standard that our industry has > aligned on. OpenTelemetry is the merger of two earlier distributed tracing > projects OpenTracing and OpenCensus. > Next step: Add design document that goes into more detail on the benefits of > distributed tracing and describes how Hive will enhanced. > Also see: > HBASE-22120 Replace HTrace with OpenTelemetry -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (HIVE-25069) Hive Distributed Tracing
[ https://issues.apache.org/jira/browse/HIVE-25069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17341993#comment-17341993 ] Matt McCline edited comment on HIVE-25069 at 7/31/21, 6:18 AM: --- [~abstractdog] Thank you for your comments. I am actively working on the design document and preparing a first patch. I have Thrift propagation almost done. Other areas in progress. One general issue I have run into is version conflicts with libthrift (e.g. 0.9.3 vs 0.13.0). I am looking at doing *shade-renames* of dependent JAR used for tracing. Otherwise, I am very concerned about JAR conflicts and execution errors including Throwable MethodNotFoundException. Without *shade-renames* I think it will be harder to use the tracing in Tez, HWC, and other components which I want to address next after the first part is checked in. I too think Tez state-changes would be great to trace. LLAP, too. Note that Tez needs tracing propagation with its protobuf msgs (it is not Thrift). It has protobuf dependencies version problems (2 vs. 3), also. was (Author: mattmccline): [~abstractdog] Thank you for your comments. I am actively working on the design document and preparing a first patch. I have Thrift propagation almost done. Other areas in progress. One general issue I have run into is version conflicts with libthrift (e.g. 0.9.3 vs 0.13.0). I am looking at doing *shade-renames* of dependent JAR used for tracing. Otherwise, I am very concerned about JAR conflicts and execution errors including Throwable MethodNotFoundException. Without *shade-renames* I think it will be harder to use the tracing in Tez, HWC, and other components which I want to address next after the first part is checked in. I too think Tez state-changes would be great to trace. LLAP, too. Note that Tez needs tracing propagation with its protobuf msgs (it is not Thrift). It has protobuf dependencies version problems (2 vs. 3), also. Thrift Propagation: !image-2021-05-10-09-20-54-688.png! High level Tez related Span in Hive Server 2 so far: !image-2021-05-10-09-30-44-570.png! > Hive Distributed Tracing > > > Key: HIVE-25069 > URL: https://issues.apache.org/jira/browse/HIVE-25069 > Project: Hive > Issue Type: New Feature >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > > Instrument Hive code to gather distributed traces and export trace data to a > configurable collector. > Distributed tracing is a revolutionary tool for debugging issues. > We will use the new OpenTelemetry open-source standard that our industry has > aligned on. OpenTelemetry is the merger of two earlier distributed tracing > projects OpenTracing and OpenCensus. > Next step: Add design document that goes into more detail on the benefits of > distributed tracing and describes how Hive will enhanced. > Also see: > HBASE-22120 Replace HTrace with OpenTelemetry -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25069) Hive Distributed Tracing
[ https://issues.apache.org/jira/browse/HIVE-25069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-25069: Attachment: (was: image-2021-05-10-09-20-54-688.png) > Hive Distributed Tracing > > > Key: HIVE-25069 > URL: https://issues.apache.org/jira/browse/HIVE-25069 > Project: Hive > Issue Type: New Feature >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > > Instrument Hive code to gather distributed traces and export trace data to a > configurable collector. > Distributed tracing is a revolutionary tool for debugging issues. > We will use the new OpenTelemetry open-source standard that our industry has > aligned on. OpenTelemetry is the merger of two earlier distributed tracing > projects OpenTracing and OpenCensus. > Next step: Add design document that goes into more detail on the benefits of > distributed tracing and describes how Hive will enhanced. > Also see: > HBASE-22120 Replace HTrace with OpenTelemetry -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25069) Hive Distributed Tracing
[ https://issues.apache.org/jira/browse/HIVE-25069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-25069: Attachment: (was: image-2021-05-10-09-30-44-570.png) > Hive Distributed Tracing > > > Key: HIVE-25069 > URL: https://issues.apache.org/jira/browse/HIVE-25069 > Project: Hive > Issue Type: New Feature >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > > Instrument Hive code to gather distributed traces and export trace data to a > configurable collector. > Distributed tracing is a revolutionary tool for debugging issues. > We will use the new OpenTelemetry open-source standard that our industry has > aligned on. OpenTelemetry is the merger of two earlier distributed tracing > projects OpenTracing and OpenCensus. > Next step: Add design document that goes into more detail on the benefits of > distributed tracing and describes how Hive will enhanced. > Also see: > HBASE-22120 Replace HTrace with OpenTelemetry -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25069) Hive Distributed Tracing
[ https://issues.apache.org/jira/browse/HIVE-25069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-25069: Attachment: (was: image-2021-05-10-19-06-02-679.png) > Hive Distributed Tracing > > > Key: HIVE-25069 > URL: https://issues.apache.org/jira/browse/HIVE-25069 > Project: Hive > Issue Type: New Feature >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > > Instrument Hive code to gather distributed traces and export trace data to a > configurable collector. > Distributed tracing is a revolutionary tool for debugging issues. > We will use the new OpenTelemetry open-source standard that our industry has > aligned on. OpenTelemetry is the merger of two earlier distributed tracing > projects OpenTracing and OpenCensus. > Next step: Add design document that goes into more detail on the benefits of > distributed tracing and describes how Hive will enhanced. > Also see: > HBASE-22120 Replace HTrace with OpenTelemetry -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HIVE-25140) Hive Distributed Tracing -- Part 1: Disabled
[ https://issues.apache.org/jira/browse/HIVE-25140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline resolved HIVE-25140. - Resolution: Won't Fix > Hive Distributed Tracing -- Part 1: Disabled > > > Key: HIVE-25140 > URL: https://issues.apache.org/jira/browse/HIVE-25140 > Project: Hive > Issue Type: Sub-task >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > > Infrastructure except exporters to Jaeger or OpenTelementry (OTL) due to > Thrift and protobuf version conflicts. A logging only exporter is used. > There are Spans for BeeLine and Hive. Server 2. The code was developed on > branch-3.1 and porting Spans to the Hive MetaStore on master is taking more > time due to major metastore code refactoring. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25140) Hive Distributed Tracing -- Part 1: Disabled
[ https://issues.apache.org/jira/browse/HIVE-25140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-25140: Attachment: (was: HIVE-25140.03.patch) > Hive Distributed Tracing -- Part 1: Disabled > > > Key: HIVE-25140 > URL: https://issues.apache.org/jira/browse/HIVE-25140 > Project: Hive > Issue Type: Sub-task >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > > Infrastructure except exporters to Jaeger or OpenTelementry (OTL) due to > Thrift and protobuf version conflicts. A logging only exporter is used. > There are Spans for BeeLine and Hive. Server 2. The code was developed on > branch-3.1 and porting Spans to the Hive MetaStore on master is taking more > time due to major metastore code refactoring. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25140) Hive Distributed Tracing -- Part 1: Disabled
[ https://issues.apache.org/jira/browse/HIVE-25140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-25140: Attachment: (was: HIVE-25140.01.patch) > Hive Distributed Tracing -- Part 1: Disabled > > > Key: HIVE-25140 > URL: https://issues.apache.org/jira/browse/HIVE-25140 > Project: Hive > Issue Type: Sub-task >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > > Infrastructure except exporters to Jaeger or OpenTelementry (OTL) due to > Thrift and protobuf version conflicts. A logging only exporter is used. > There are Spans for BeeLine and Hive. Server 2. The code was developed on > branch-3.1 and porting Spans to the Hive MetaStore on master is taking more > time due to major metastore code refactoring. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25140) Hive Distributed Tracing -- Part 1: Disabled
[ https://issues.apache.org/jira/browse/HIVE-25140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-25140: Attachment: (was: HIVE-25140.02.patch) > Hive Distributed Tracing -- Part 1: Disabled > > > Key: HIVE-25140 > URL: https://issues.apache.org/jira/browse/HIVE-25140 > Project: Hive > Issue Type: Sub-task >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > > Infrastructure except exporters to Jaeger or OpenTelementry (OTL) due to > Thrift and protobuf version conflicts. A logging only exporter is used. > There are Spans for BeeLine and Hive. Server 2. The code was developed on > branch-3.1 and porting Spans to the Hive MetaStore on master is taking more > time due to major metastore code refactoring. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25140) Hive Distributed Tracing -- Part 1: Disabled
[ https://issues.apache.org/jira/browse/HIVE-25140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-25140: Status: Open (was: Patch Available) Rethink. > Hive Distributed Tracing -- Part 1: Disabled > > > Key: HIVE-25140 > URL: https://issues.apache.org/jira/browse/HIVE-25140 > Project: Hive > Issue Type: Sub-task >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > Attachments: HIVE-25140.01.patch, HIVE-25140.02.patch, > HIVE-25140.03.patch > > > Infrastructure except exporters to Jaeger or OpenTelementry (OTL) due to > Thrift and protobuf version conflicts. A logging only exporter is used. > There are Spans for BeeLine and Hive. Server 2. The code was developed on > branch-3.1 and porting Spans to the Hive MetaStore on master is taking more > time due to major metastore code refactoring. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25396) Improve uncaught Thread Exception handling in Hive Server 2
[ https://issues.apache.org/jira/browse/HIVE-25396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-25396: Attachment: hive-uncaught-http-servlet-01-3_1.patch > Improve uncaught Thread Exception handling in Hive Server 2 > --- > > Key: HIVE-25396 > URL: https://issues.apache.org/jira/browse/HIVE-25396 > Project: Hive > Issue Type: Bug >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > Attachments: hive-uncaught-http-servlet-01-3_1.patch > > > Hive's org.apache.hive.service.thrift.ThriftHttpServlet.doPost method does > not handle all Exception kinds. This leaves uncaught Exception handling > choices to the Jetty HTTP library. We fix that. > Also, a Thread.UncaughtExceptionHandler is added to Hive Server 2 so uncaught > Exception are handled uniformly, including making them logged and not just > printed to stderr. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-25396) Improve uncaught Thread Exception handling in Hive Server 2
[ https://issues.apache.org/jira/browse/HIVE-25396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline reassigned HIVE-25396: --- > Improve uncaught Thread Exception handling in Hive Server 2 > --- > > Key: HIVE-25396 > URL: https://issues.apache.org/jira/browse/HIVE-25396 > Project: Hive > Issue Type: Bug >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > > Hive's org.apache.hive.service.thrift.ThriftHttpServlet.doPost method does > not handle all Exception kinds. This leaves uncaught Exception handling > choices to the Jetty HTTP library. We fix that. > Also, a Thread.UncaughtExceptionHandler is added to Hive Server 2 so uncaught > Exception are handled uniformly, including making them logged and not just > printed to stderr. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25385) Prevent Hive Server 2 process failures when InterruptedException encountered (Part 1)
[ https://issues.apache.org/jira/browse/HIVE-25385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-25385: Summary: Prevent Hive Server 2 process failures when InterruptedException encountered (Part 1) (was: Prevent Hive Server 2 process failures when InterruptedException encountered) > Prevent Hive Server 2 process failures when InterruptedException encountered > (Part 1) > - > > Key: HIVE-25385 > URL: https://issues.apache.org/jira/browse/HIVE-25385 > Project: Hive > Issue Type: Bug >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > Labels: pull-request-available > Attachments: hive-interrupted-part1-01-3_1.patch, > hive-interrupted-part1-01-master.patch > > Time Spent: 10m > Remaining Estimate: 0h > > To prevent Hive Server 2 process failure, wrap InterruptedException with > another Exception like MetaException, HiveSQLException, etc. Otherwise, > InterruptedException rises to Thread.run and kills the process. > Example of problem stack trace: > java.lang.reflect.UndeclaredThrowableExceptionjava.lang.reflect.UndeclaredThrowableException > at com.sun.proxy.$Proxy44.heartbeat(Unknown Source) at > sun.reflect.GeneratedMethodAccessor127.invoke(Unknown Source) at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) at > org.apache.hadoop.hive.metastore.HiveMetaStoreClient$SynchronizedHandler.invoke(HiveMetaStoreClient.java:2990) > at com.sun.proxy.$Proxy44.heartbeat(Unknown Source) at > org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.heartbeat(DbTxnManager.java:622) > at > org.apache.hadoop.hive.ql.lockmgr.DbTxnManager$Heartbeater.lambda$run$0(DbTxnManager.java:999) > at java.security.AccessController.doPrivileged(Native Method) at > javax.security.auth.Subject.doAs(Subject.java:422) at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730) > at > org.apache.hadoop.hive.ql.lockmgr.DbTxnManager$Heartbeater.run(DbTxnManager.java:998) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748)Caused by: > java.lang.InterruptedException: sleep interrupted at > java.lang.Thread.sleep(Native Method) at > org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:259) > ... 19 more -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25385) Prevent Hive Server 2 process failures when InterruptedException encountered
[ https://issues.apache.org/jira/browse/HIVE-25385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-25385: Attachment: hive-interrupted-part1-01-master.patch > Prevent Hive Server 2 process failures when InterruptedException encountered > > > Key: HIVE-25385 > URL: https://issues.apache.org/jira/browse/HIVE-25385 > Project: Hive > Issue Type: Bug >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > Attachments: hive-interrupted-part1-01-3_1.patch, > hive-interrupted-part1-01-master.patch > > > To prevent Hive Server 2 process failure, wrap InterruptedException with > another Exception like MetaException, HiveSQLException, etc. Otherwise, > InterruptedException rises to Thread.run and kills the process. > Example of problem stack trace: > java.lang.reflect.UndeclaredThrowableExceptionjava.lang.reflect.UndeclaredThrowableException > at com.sun.proxy.$Proxy44.heartbeat(Unknown Source) at > sun.reflect.GeneratedMethodAccessor127.invoke(Unknown Source) at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) at > org.apache.hadoop.hive.metastore.HiveMetaStoreClient$SynchronizedHandler.invoke(HiveMetaStoreClient.java:2990) > at com.sun.proxy.$Proxy44.heartbeat(Unknown Source) at > org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.heartbeat(DbTxnManager.java:622) > at > org.apache.hadoop.hive.ql.lockmgr.DbTxnManager$Heartbeater.lambda$run$0(DbTxnManager.java:999) > at java.security.AccessController.doPrivileged(Native Method) at > javax.security.auth.Subject.doAs(Subject.java:422) at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730) > at > org.apache.hadoop.hive.ql.lockmgr.DbTxnManager$Heartbeater.run(DbTxnManager.java:998) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748)Caused by: > java.lang.InterruptedException: sleep interrupted at > java.lang.Thread.sleep(Native Method) at > org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:259) > ... 19 more -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25385) Prevent Hive Server 2 process failures when InterruptedException encountered
[ https://issues.apache.org/jira/browse/HIVE-25385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-25385: Attachment: hive-interrupted-part1-01-3_1.patch > Prevent Hive Server 2 process failures when InterruptedException encountered > > > Key: HIVE-25385 > URL: https://issues.apache.org/jira/browse/HIVE-25385 > Project: Hive > Issue Type: Bug >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > Attachments: hive-interrupted-part1-01-3_1.patch > > > To prevent Hive Server 2 process failure, wrap InterruptedException with > another Exception like MetaException, HiveSQLException, etc. Otherwise, > InterruptedException rises to Thread.run and kills the process. > Example of problem stack trace: > java.lang.reflect.UndeclaredThrowableExceptionjava.lang.reflect.UndeclaredThrowableException > at com.sun.proxy.$Proxy44.heartbeat(Unknown Source) at > sun.reflect.GeneratedMethodAccessor127.invoke(Unknown Source) at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) at > org.apache.hadoop.hive.metastore.HiveMetaStoreClient$SynchronizedHandler.invoke(HiveMetaStoreClient.java:2990) > at com.sun.proxy.$Proxy44.heartbeat(Unknown Source) at > org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.heartbeat(DbTxnManager.java:622) > at > org.apache.hadoop.hive.ql.lockmgr.DbTxnManager$Heartbeater.lambda$run$0(DbTxnManager.java:999) > at java.security.AccessController.doPrivileged(Native Method) at > javax.security.auth.Subject.doAs(Subject.java:422) at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730) > at > org.apache.hadoop.hive.ql.lockmgr.DbTxnManager$Heartbeater.run(DbTxnManager.java:998) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748)Caused by: > java.lang.InterruptedException: sleep interrupted at > java.lang.Thread.sleep(Native Method) at > org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:259) > ... 19 more -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-25385) Prevent Hive Server 2 process failures when InterruptedException encountered
[ https://issues.apache.org/jira/browse/HIVE-25385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline reassigned HIVE-25385: --- > Prevent Hive Server 2 process failures when InterruptedException encountered > > > Key: HIVE-25385 > URL: https://issues.apache.org/jira/browse/HIVE-25385 > Project: Hive > Issue Type: Bug >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > > To prevent Hive Server 2 process failure, wrap InterruptedException with > another Exception like MetaException, HiveSQLException, etc. Otherwise, > InterruptedException rises to Thread.run and kills the process. > Example of problem stack trace: > java.lang.reflect.UndeclaredThrowableExceptionjava.lang.reflect.UndeclaredThrowableException > at com.sun.proxy.$Proxy44.heartbeat(Unknown Source) at > sun.reflect.GeneratedMethodAccessor127.invoke(Unknown Source) at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) at > org.apache.hadoop.hive.metastore.HiveMetaStoreClient$SynchronizedHandler.invoke(HiveMetaStoreClient.java:2990) > at com.sun.proxy.$Proxy44.heartbeat(Unknown Source) at > org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.heartbeat(DbTxnManager.java:622) > at > org.apache.hadoop.hive.ql.lockmgr.DbTxnManager$Heartbeater.lambda$run$0(DbTxnManager.java:999) > at java.security.AccessController.doPrivileged(Native Method) at > javax.security.auth.Subject.doAs(Subject.java:422) at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730) > at > org.apache.hadoop.hive.ql.lockmgr.DbTxnManager$Heartbeater.run(DbTxnManager.java:998) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748)Caused by: > java.lang.InterruptedException: sleep interrupted at > java.lang.Thread.sleep(Native Method) at > org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:259) > ... 19 more -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HIVE-20475) Hive Thrift Server 2 stops frequently
[ https://issues.apache.org/jira/browse/HIVE-20475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline resolved HIVE-20475. - Resolution: Duplicate HIVE-25307 > Hive Thrift Server 2 stops frequently > - > > Key: HIVE-20475 > URL: https://issues.apache.org/jira/browse/HIVE-20475 > Project: Hive > Issue Type: Bug > Environment: HDP 2.6.5.0 > Hive 1.2.1000 > Spark 2.3.0 >Reporter: Vinod Nerella >Priority: Major > > 18/08/28 02:18:05 ERROR TThreadPoolServer: Error occurred during processing > of message. > java.lang.RuntimeException: > org.apache.thrift.transport.TSaslTransportException: No data or no sasl data > in the stream > at > org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:219) > at > org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:269) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > Caused by: org.apache.thrift.transport.TSaslTransportException: No data or no > sasl data in the stream > at > org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:328) > at > org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:41) > at > org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:216) > ... 4 more -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25307) Hive Server 2 crashes when Thrift library encounters particular security protocol issue
[ https://issues.apache.org/jira/browse/HIVE-25307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-25307: Attachment: hive-thrift-fix2-05-3_1.patch > Hive Server 2 crashes when Thrift library encounters particular security > protocol issue > --- > > Key: HIVE-25307 > URL: https://issues.apache.org/jira/browse/HIVE-25307 > Project: Hive > Issue Type: Bug >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Labels: pull-request-available > Attachments: hive-thrift-fix2-03-3_1.patch, > hive-thrift-fix2-04-3_1.patch, hive-thrift-fix2-05-3_1.patch > > Time Spent: 10m > Remaining Estimate: 0h > > A RuntimeException is thrown by the Thrift library that causes Hive Server 2 > to crash on our customer's machine. If you Google this the exception has been > reported a couple of times over the years but not fixed. A blog (see > references below) says it is an occasional security protocol issue between > Hive Server 2 and a proxy like a Gateway. > One challenge in the older 0.9.3 Thrift version was the Thrift > TTransportFactory getTransport method declaration had throws no Exceptions. > Hence the likely choice of RuntimeException. But that Exception is fatal to > Hive Server 2. > The proposed fix is a work around is we catch RuntimeException in the inner > class TUGIAssumingTransportFactory of the HadoopThriftAuthBridge class in > Hive Server 2. And throw a throw the RuntimeException's (inner) cause (e.g. > TSaslTransportException) as a TTransportException. > Once the Thrift library stops throwing RuntimeException or we catch fatal > Throwable exceptions in the Thrift library's TThreadPoolServer's inner class > WorkerProcess run method and display them, the RuntimeException try/catch > clause can be removed. > ExceptionClassName: > java.lang.RuntimeException > ExceptionStackTrace: > java.lang.RuntimeException: > org.apache.thrift.transport.TSaslTransportException: No data or no sasl data > in the stream > at > org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:219) > at > org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge.java:694) > at > org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge.java:691) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:360) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1710) > at > org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory.getTransport(HadoopThriftAuthBridge.java:691) > at > org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:269) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > Caused by: org.apache.thrift.transport.TSaslTransportException: No data or no > sasl data in the stream > at > org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:326) > at > org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:41) > at > org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:216) > ... 10 more > > References: > [Hive server 2 thrift error - Cloudera Community - > 34293|https://community.cloudera.com/t5/Support-Questions/Hive-server-2-thrift-error/td-p/34293] > Eric Lin blog "“NO DATA OR NO SASL DATA IN THE STREAM” ERROR IN HIVESERVER2 > LOG" > HIVE-12754 AuthTypes.NONE cause exception after HS2 start - ASF JIRA > (apache.org) > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25307) Hive Server 2 crashes when Thrift library encounters particular security protocol issue
[ https://issues.apache.org/jira/browse/HIVE-25307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-25307: Attachment: (was: hive-thrift-fix2-01-3_1.patch) > Hive Server 2 crashes when Thrift library encounters particular security > protocol issue > --- > > Key: HIVE-25307 > URL: https://issues.apache.org/jira/browse/HIVE-25307 > Project: Hive > Issue Type: Bug >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Labels: pull-request-available > Attachments: hive-thrift-fix2-03-3_1.patch, > hive-thrift-fix2-04-3_1.patch > > Time Spent: 10m > Remaining Estimate: 0h > > A RuntimeException is thrown by the Thrift library that causes Hive Server 2 > to crash on our customer's machine. If you Google this the exception has been > reported a couple of times over the years but not fixed. A blog (see > references below) says it is an occasional security protocol issue between > Hive Server 2 and a proxy like a Gateway. > One challenge in the older 0.9.3 Thrift version was the Thrift > TTransportFactory getTransport method declaration had throws no Exceptions. > Hence the likely choice of RuntimeException. But that Exception is fatal to > Hive Server 2. > The proposed fix is a work around is we catch RuntimeException in the inner > class TUGIAssumingTransportFactory of the HadoopThriftAuthBridge class in > Hive Server 2. And throw a throw the RuntimeException's (inner) cause (e.g. > TSaslTransportException) as a TTransportException. > Once the Thrift library stops throwing RuntimeException or we catch fatal > Throwable exceptions in the Thrift library's TThreadPoolServer's inner class > WorkerProcess run method and display them, the RuntimeException try/catch > clause can be removed. > ExceptionClassName: > java.lang.RuntimeException > ExceptionStackTrace: > java.lang.RuntimeException: > org.apache.thrift.transport.TSaslTransportException: No data or no sasl data > in the stream > at > org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:219) > at > org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge.java:694) > at > org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge.java:691) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:360) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1710) > at > org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory.getTransport(HadoopThriftAuthBridge.java:691) > at > org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:269) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > Caused by: org.apache.thrift.transport.TSaslTransportException: No data or no > sasl data in the stream > at > org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:326) > at > org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:41) > at > org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:216) > ... 10 more > > References: > [Hive server 2 thrift error - Cloudera Community - > 34293|https://community.cloudera.com/t5/Support-Questions/Hive-server-2-thrift-error/td-p/34293] > Eric Lin blog "“NO DATA OR NO SASL DATA IN THE STREAM” ERROR IN HIVESERVER2 > LOG" > HIVE-12754 AuthTypes.NONE cause exception after HS2 start - ASF JIRA > (apache.org) > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25307) Hive Server 2 crashes when Thrift library encounters particular security protocol issue
[ https://issues.apache.org/jira/browse/HIVE-25307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-25307: Attachment: (was: hive-thrift-fix2-02-3_1.patch) > Hive Server 2 crashes when Thrift library encounters particular security > protocol issue > --- > > Key: HIVE-25307 > URL: https://issues.apache.org/jira/browse/HIVE-25307 > Project: Hive > Issue Type: Bug >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Labels: pull-request-available > Attachments: hive-thrift-fix2-03-3_1.patch, > hive-thrift-fix2-04-3_1.patch > > Time Spent: 10m > Remaining Estimate: 0h > > A RuntimeException is thrown by the Thrift library that causes Hive Server 2 > to crash on our customer's machine. If you Google this the exception has been > reported a couple of times over the years but not fixed. A blog (see > references below) says it is an occasional security protocol issue between > Hive Server 2 and a proxy like a Gateway. > One challenge in the older 0.9.3 Thrift version was the Thrift > TTransportFactory getTransport method declaration had throws no Exceptions. > Hence the likely choice of RuntimeException. But that Exception is fatal to > Hive Server 2. > The proposed fix is a work around is we catch RuntimeException in the inner > class TUGIAssumingTransportFactory of the HadoopThriftAuthBridge class in > Hive Server 2. And throw a throw the RuntimeException's (inner) cause (e.g. > TSaslTransportException) as a TTransportException. > Once the Thrift library stops throwing RuntimeException or we catch fatal > Throwable exceptions in the Thrift library's TThreadPoolServer's inner class > WorkerProcess run method and display them, the RuntimeException try/catch > clause can be removed. > ExceptionClassName: > java.lang.RuntimeException > ExceptionStackTrace: > java.lang.RuntimeException: > org.apache.thrift.transport.TSaslTransportException: No data or no sasl data > in the stream > at > org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:219) > at > org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge.java:694) > at > org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge.java:691) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:360) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1710) > at > org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory.getTransport(HadoopThriftAuthBridge.java:691) > at > org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:269) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > Caused by: org.apache.thrift.transport.TSaslTransportException: No data or no > sasl data in the stream > at > org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:326) > at > org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:41) > at > org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:216) > ... 10 more > > References: > [Hive server 2 thrift error - Cloudera Community - > 34293|https://community.cloudera.com/t5/Support-Questions/Hive-server-2-thrift-error/td-p/34293] > Eric Lin blog "“NO DATA OR NO SASL DATA IN THE STREAM” ERROR IN HIVESERVER2 > LOG" > HIVE-12754 AuthTypes.NONE cause exception after HS2 start - ASF JIRA > (apache.org) > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25307) Hive Server 2 crashes when Thrift library encounters particular security protocol issue
[ https://issues.apache.org/jira/browse/HIVE-25307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-25307: Attachment: hive-thrift-fix2-04-3_1.patch > Hive Server 2 crashes when Thrift library encounters particular security > protocol issue > --- > > Key: HIVE-25307 > URL: https://issues.apache.org/jira/browse/HIVE-25307 > Project: Hive > Issue Type: Bug >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Labels: pull-request-available > Attachments: hive-thrift-fix2-01-3_1.patch, > hive-thrift-fix2-02-3_1.patch, hive-thrift-fix2-03-3_1.patch, > hive-thrift-fix2-04-3_1.patch > > Time Spent: 10m > Remaining Estimate: 0h > > A RuntimeException is thrown by the Thrift library that causes Hive Server 2 > to crash on our customer's machine. If you Google this the exception has been > reported a couple of times over the years but not fixed. A blog (see > references below) says it is an occasional security protocol issue between > Hive Server 2 and a proxy like a Gateway. > One challenge in the older 0.9.3 Thrift version was the Thrift > TTransportFactory getTransport method declaration had throws no Exceptions. > Hence the likely choice of RuntimeException. But that Exception is fatal to > Hive Server 2. > The proposed fix is a work around is we catch RuntimeException in the inner > class TUGIAssumingTransportFactory of the HadoopThriftAuthBridge class in > Hive Server 2. And throw a throw the RuntimeException's (inner) cause (e.g. > TSaslTransportException) as a TTransportException. > Once the Thrift library stops throwing RuntimeException or we catch fatal > Throwable exceptions in the Thrift library's TThreadPoolServer's inner class > WorkerProcess run method and display them, the RuntimeException try/catch > clause can be removed. > ExceptionClassName: > java.lang.RuntimeException > ExceptionStackTrace: > java.lang.RuntimeException: > org.apache.thrift.transport.TSaslTransportException: No data or no sasl data > in the stream > at > org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:219) > at > org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge.java:694) > at > org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge.java:691) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:360) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1710) > at > org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory.getTransport(HadoopThriftAuthBridge.java:691) > at > org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:269) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > Caused by: org.apache.thrift.transport.TSaslTransportException: No data or no > sasl data in the stream > at > org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:326) > at > org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:41) > at > org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:216) > ... 10 more > > References: > [Hive server 2 thrift error - Cloudera Community - > 34293|https://community.cloudera.com/t5/Support-Questions/Hive-server-2-thrift-error/td-p/34293] > Eric Lin blog "“NO DATA OR NO SASL DATA IN THE STREAM” ERROR IN HIVESERVER2 > LOG" > HIVE-12754 AuthTypes.NONE cause exception after HS2 start - ASF JIRA > (apache.org) > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25307) Hive Server 2 crashes when Thrift library encounters particular security protocol issue
[ https://issues.apache.org/jira/browse/HIVE-25307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-25307: Attachment: hive-thrift-fix2-03-3_1.patch > Hive Server 2 crashes when Thrift library encounters particular security > protocol issue > --- > > Key: HIVE-25307 > URL: https://issues.apache.org/jira/browse/HIVE-25307 > Project: Hive > Issue Type: Bug >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Labels: pull-request-available > Attachments: hive-thrift-fix2-01-3_1.patch, > hive-thrift-fix2-02-3_1.patch, hive-thrift-fix2-03-3_1.patch > > Time Spent: 10m > Remaining Estimate: 0h > > A RuntimeException is thrown by the Thrift library that causes Hive Server 2 > to crash on our customer's machine. If you Google this the exception has been > reported a couple of times over the years but not fixed. A blog (see > references below) says it is an occasional security protocol issue between > Hive Server 2 and a proxy like a Gateway. > One challenge in the older 0.9.3 Thrift version was the Thrift > TTransportFactory getTransport method declaration had throws no Exceptions. > Hence the likely choice of RuntimeException. But that Exception is fatal to > Hive Server 2. > The proposed fix is a work around is we catch RuntimeException in the inner > class TUGIAssumingTransportFactory of the HadoopThriftAuthBridge class in > Hive Server 2. And throw a throw the RuntimeException's (inner) cause (e.g. > TSaslTransportException) as a TTransportException. > Once the Thrift library stops throwing RuntimeException or we catch fatal > Throwable exceptions in the Thrift library's TThreadPoolServer's inner class > WorkerProcess run method and display them, the RuntimeException try/catch > clause can be removed. > ExceptionClassName: > java.lang.RuntimeException > ExceptionStackTrace: > java.lang.RuntimeException: > org.apache.thrift.transport.TSaslTransportException: No data or no sasl data > in the stream > at > org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:219) > at > org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge.java:694) > at > org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge.java:691) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:360) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1710) > at > org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory.getTransport(HadoopThriftAuthBridge.java:691) > at > org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:269) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > Caused by: org.apache.thrift.transport.TSaslTransportException: No data or no > sasl data in the stream > at > org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:326) > at > org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:41) > at > org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:216) > ... 10 more > > References: > [Hive server 2 thrift error - Cloudera Community - > 34293|https://community.cloudera.com/t5/Support-Questions/Hive-server-2-thrift-error/td-p/34293] > Eric Lin blog "“NO DATA OR NO SASL DATA IN THE STREAM” ERROR IN HIVESERVER2 > LOG" > HIVE-12754 AuthTypes.NONE cause exception after HS2 start - ASF JIRA > (apache.org) > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-25337) EXPLAIN Tez/Dag
[ https://issues.apache.org/jira/browse/HIVE-25337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17381562#comment-17381562 ] Matt McCline commented on HIVE-25337: - Great idea. And, when I added EXPLAIN VECTORIZATION to primary audience was 1 person (me)! There is a tremendous value in showing more information. > EXPLAIN Tez/Dag > --- > > Key: HIVE-25337 > URL: https://issues.apache.org/jira/browse/HIVE-25337 > Project: Hive > Issue Type: New Feature >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > > Just an idea, to consider if we can expose some really tez/dag related thing > with Hive's explain command, which user is not interested about under normal > circumstances, e.g. edge types (on class level), input/output classes, output > committers if any, dag plan size, etc. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (HIVE-25337) EXPLAIN Tez/Dag
[ https://issues.apache.org/jira/browse/HIVE-25337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17381562#comment-17381562 ] Matt McCline edited comment on HIVE-25337 at 7/15/21, 7:58 PM: --- Great idea. And, when I added EXPLAIN VECTORIZATION the primary audience was 1 person (me)! There is a tremendous value in showing more information. was (Author: mattmccline): Great idea. And, when I added EXPLAIN VECTORIZATION to primary audience was 1 person (me)! There is a tremendous value in showing more information. > EXPLAIN Tez/Dag > --- > > Key: HIVE-25337 > URL: https://issues.apache.org/jira/browse/HIVE-25337 > Project: Hive > Issue Type: New Feature >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > > Just an idea, to consider if we can expose some really tez/dag related thing > with Hive's explain command, which user is not interested about under normal > circumstances, e.g. edge types (on class level), input/output classes, output > committers if any, dag plan size, etc. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25307) Hive Server 2 crashes when Thrift library encounters particular security protocol issue
[ https://issues.apache.org/jira/browse/HIVE-25307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-25307: Attachment: hive-thrift-fix2-02-3_1.patch > Hive Server 2 crashes when Thrift library encounters particular security > protocol issue > --- > > Key: HIVE-25307 > URL: https://issues.apache.org/jira/browse/HIVE-25307 > Project: Hive > Issue Type: Bug >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Labels: pull-request-available > Attachments: hive-thrift-fix2-01-3_1.patch, > hive-thrift-fix2-02-3_1.patch > > Time Spent: 10m > Remaining Estimate: 0h > > A RuntimeException is thrown by the Thrift library that causes Hive Server 2 > to crash on our customer's machine. If you Google this the exception has been > reported a couple of times over the years but not fixed. A blog (see > references below) says it is an occasional security protocol issue between > Hive Server 2 and a proxy like a Gateway. > One challenge in the older 0.9.3 Thrift version was the Thrift > TTransportFactory getTransport method declaration had throws no Exceptions. > Hence the likely choice of RuntimeException. But that Exception is fatal to > Hive Server 2. > The proposed fix is a work around is we catch RuntimeException in the inner > class TUGIAssumingTransportFactory of the HadoopThriftAuthBridge class in > Hive Server 2. And throw a throw the RuntimeException's (inner) cause (e.g. > TSaslTransportException) as a TTransportException. > Once the Thrift library stops throwing RuntimeException or we catch fatal > Throwable exceptions in the Thrift library's TThreadPoolServer's inner class > WorkerProcess run method and display them, the RuntimeException try/catch > clause can be removed. > ExceptionClassName: > java.lang.RuntimeException > ExceptionStackTrace: > java.lang.RuntimeException: > org.apache.thrift.transport.TSaslTransportException: No data or no sasl data > in the stream > at > org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:219) > at > org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge.java:694) > at > org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge.java:691) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:360) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1710) > at > org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory.getTransport(HadoopThriftAuthBridge.java:691) > at > org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:269) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > Caused by: org.apache.thrift.transport.TSaslTransportException: No data or no > sasl data in the stream > at > org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:326) > at > org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:41) > at > org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:216) > ... 10 more > > References: > [Hive server 2 thrift error - Cloudera Community - > 34293|https://community.cloudera.com/t5/Support-Questions/Hive-server-2-thrift-error/td-p/34293] > Eric Lin blog "“NO DATA OR NO SASL DATA IN THE STREAM” ERROR IN HIVESERVER2 > LOG" > HIVE-12754 AuthTypes.NONE cause exception after HS2 start - ASF JIRA > (apache.org) > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25307) Hive Server 2 crashes when Thrift library encounters particular security protocol issue
[ https://issues.apache.org/jira/browse/HIVE-25307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-25307: Attachment: (was: hive-thrift-fix-02-3_1.patch) > Hive Server 2 crashes when Thrift library encounters particular security > protocol issue > --- > > Key: HIVE-25307 > URL: https://issues.apache.org/jira/browse/HIVE-25307 > Project: Hive > Issue Type: Bug >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Labels: pull-request-available > Attachments: hive-thrift-fix2-01-3_1.patch > > Time Spent: 10m > Remaining Estimate: 0h > > A RuntimeException is thrown by the Thrift library that causes Hive Server 2 > to crash on our customer's machine. If you Google this the exception has been > reported a couple of times over the years but not fixed. A blog (see > references below) says it is an occasional security protocol issue between > Hive Server 2 and a proxy like a Gateway. > One challenge in the older 0.9.3 Thrift version was the Thrift > TTransportFactory getTransport method declaration had throws no Exceptions. > Hence the likely choice of RuntimeException. But that Exception is fatal to > Hive Server 2. > The proposed fix is a work around is we catch RuntimeException in the inner > class TUGIAssumingTransportFactory of the HadoopThriftAuthBridge class in > Hive Server 2. And throw a throw the RuntimeException's (inner) cause (e.g. > TSaslTransportException) as a TTransportException. > Once the Thrift library stops throwing RuntimeException or we catch fatal > Throwable exceptions in the Thrift library's TThreadPoolServer's inner class > WorkerProcess run method and display them, the RuntimeException try/catch > clause can be removed. > ExceptionClassName: > java.lang.RuntimeException > ExceptionStackTrace: > java.lang.RuntimeException: > org.apache.thrift.transport.TSaslTransportException: No data or no sasl data > in the stream > at > org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:219) > at > org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge.java:694) > at > org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge.java:691) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:360) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1710) > at > org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory.getTransport(HadoopThriftAuthBridge.java:691) > at > org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:269) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > Caused by: org.apache.thrift.transport.TSaslTransportException: No data or no > sasl data in the stream > at > org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:326) > at > org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:41) > at > org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:216) > ... 10 more > > References: > [Hive server 2 thrift error - Cloudera Community - > 34293|https://community.cloudera.com/t5/Support-Questions/Hive-server-2-thrift-error/td-p/34293] > Eric Lin blog "“NO DATA OR NO SASL DATA IN THE STREAM” ERROR IN HIVESERVER2 > LOG" > HIVE-12754 AuthTypes.NONE cause exception after HS2 start - ASF JIRA > (apache.org) > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25307) Hive Server 2 crashes when Thrift library encounters particular security protocol issue
[ https://issues.apache.org/jira/browse/HIVE-25307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-25307: Attachment: hive-thrift-fix-02-3_1.patch > Hive Server 2 crashes when Thrift library encounters particular security > protocol issue > --- > > Key: HIVE-25307 > URL: https://issues.apache.org/jira/browse/HIVE-25307 > Project: Hive > Issue Type: Bug >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Labels: pull-request-available > Attachments: hive-thrift-fix-02-3_1.patch, > hive-thrift-fix2-01-3_1.patch > > Time Spent: 10m > Remaining Estimate: 0h > > A RuntimeException is thrown by the Thrift library that causes Hive Server 2 > to crash on our customer's machine. If you Google this the exception has been > reported a couple of times over the years but not fixed. A blog (see > references below) says it is an occasional security protocol issue between > Hive Server 2 and a proxy like a Gateway. > One challenge in the older 0.9.3 Thrift version was the Thrift > TTransportFactory getTransport method declaration had throws no Exceptions. > Hence the likely choice of RuntimeException. But that Exception is fatal to > Hive Server 2. > The proposed fix is a work around is we catch RuntimeException in the inner > class TUGIAssumingTransportFactory of the HadoopThriftAuthBridge class in > Hive Server 2. And throw a throw the RuntimeException's (inner) cause (e.g. > TSaslTransportException) as a TTransportException. > Once the Thrift library stops throwing RuntimeException or we catch fatal > Throwable exceptions in the Thrift library's TThreadPoolServer's inner class > WorkerProcess run method and display them, the RuntimeException try/catch > clause can be removed. > ExceptionClassName: > java.lang.RuntimeException > ExceptionStackTrace: > java.lang.RuntimeException: > org.apache.thrift.transport.TSaslTransportException: No data or no sasl data > in the stream > at > org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:219) > at > org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge.java:694) > at > org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge.java:691) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:360) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1710) > at > org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory.getTransport(HadoopThriftAuthBridge.java:691) > at > org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:269) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > Caused by: org.apache.thrift.transport.TSaslTransportException: No data or no > sasl data in the stream > at > org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:326) > at > org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:41) > at > org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:216) > ... 10 more > > References: > [Hive server 2 thrift error - Cloudera Community - > 34293|https://community.cloudera.com/t5/Support-Questions/Hive-server-2-thrift-error/td-p/34293] > Eric Lin blog "“NO DATA OR NO SASL DATA IN THE STREAM” ERROR IN HIVESERVER2 > LOG" > HIVE-12754 AuthTypes.NONE cause exception after HS2 start - ASF JIRA > (apache.org) > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25307) Hive Server 2 crashes when Thrift library encounters particular security protocol issue
[ https://issues.apache.org/jira/browse/HIVE-25307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-25307: Attachment: hive-thrift-fix2-01-3_1.patch > Hive Server 2 crashes when Thrift library encounters particular security > protocol issue > --- > > Key: HIVE-25307 > URL: https://issues.apache.org/jira/browse/HIVE-25307 > Project: Hive > Issue Type: Bug >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Labels: pull-request-available > Attachments: hive-thrift-fix2-01-3_1.patch > > Time Spent: 10m > Remaining Estimate: 0h > > A RuntimeException is thrown by the Thrift library that causes Hive Server 2 > to crash on our customer's machine. If you Google this the exception has been > reported a couple of times over the years but not fixed. A blog (see > references below) says it is an occasional security protocol issue between > Hive Server 2 and a proxy like a Gateway. > One challenge in the older 0.9.3 Thrift version was the Thrift > TTransportFactory getTransport method declaration had throws no Exceptions. > Hence the likely choice of RuntimeException. But that Exception is fatal to > Hive Server 2. > The proposed fix is a work around is we catch RuntimeException in the inner > class TUGIAssumingTransportFactory of the HadoopThriftAuthBridge class in > Hive Server 2. And throw a throw the RuntimeException's (inner) cause (e.g. > TSaslTransportException) as a TTransportException. > Once the Thrift library stops throwing RuntimeException or we catch fatal > Throwable exceptions in the Thrift library's TThreadPoolServer's inner class > WorkerProcess run method and display them, the RuntimeException try/catch > clause can be removed. > ExceptionClassName: > java.lang.RuntimeException > ExceptionStackTrace: > java.lang.RuntimeException: > org.apache.thrift.transport.TSaslTransportException: No data or no sasl data > in the stream > at > org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:219) > at > org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge.java:694) > at > org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge.java:691) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:360) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1710) > at > org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory.getTransport(HadoopThriftAuthBridge.java:691) > at > org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:269) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > Caused by: org.apache.thrift.transport.TSaslTransportException: No data or no > sasl data in the stream > at > org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:326) > at > org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:41) > at > org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:216) > ... 10 more > > References: > [Hive server 2 thrift error - Cloudera Community - > 34293|https://community.cloudera.com/t5/Support-Questions/Hive-server-2-thrift-error/td-p/34293] > Eric Lin blog "“NO DATA OR NO SASL DATA IN THE STREAM” ERROR IN HIVESERVER2 > LOG" > HIVE-12754 AuthTypes.NONE cause exception after HS2 start - ASF JIRA > (apache.org) > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25307) Hive Server 2 crashes when Thrift library encounters particular security protocol issue
[ https://issues.apache.org/jira/browse/HIVE-25307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-25307: Attachment: (was: hive-thrift-fix-02-3_1.patch) > Hive Server 2 crashes when Thrift library encounters particular security > protocol issue > --- > > Key: HIVE-25307 > URL: https://issues.apache.org/jira/browse/HIVE-25307 > Project: Hive > Issue Type: Bug >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Labels: pull-request-available > Attachments: hive-thrift-fix2-01-3_1.patch > > Time Spent: 10m > Remaining Estimate: 0h > > A RuntimeException is thrown by the Thrift library that causes Hive Server 2 > to crash on our customer's machine. If you Google this the exception has been > reported a couple of times over the years but not fixed. A blog (see > references below) says it is an occasional security protocol issue between > Hive Server 2 and a proxy like a Gateway. > One challenge in the older 0.9.3 Thrift version was the Thrift > TTransportFactory getTransport method declaration had throws no Exceptions. > Hence the likely choice of RuntimeException. But that Exception is fatal to > Hive Server 2. > The proposed fix is a work around is we catch RuntimeException in the inner > class TUGIAssumingTransportFactory of the HadoopThriftAuthBridge class in > Hive Server 2. And throw a throw the RuntimeException's (inner) cause (e.g. > TSaslTransportException) as a TTransportException. > Once the Thrift library stops throwing RuntimeException or we catch fatal > Throwable exceptions in the Thrift library's TThreadPoolServer's inner class > WorkerProcess run method and display them, the RuntimeException try/catch > clause can be removed. > ExceptionClassName: > java.lang.RuntimeException > ExceptionStackTrace: > java.lang.RuntimeException: > org.apache.thrift.transport.TSaslTransportException: No data or no sasl data > in the stream > at > org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:219) > at > org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge.java:694) > at > org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge.java:691) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:360) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1710) > at > org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory.getTransport(HadoopThriftAuthBridge.java:691) > at > org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:269) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > Caused by: org.apache.thrift.transport.TSaslTransportException: No data or no > sasl data in the stream > at > org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:326) > at > org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:41) > at > org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:216) > ... 10 more > > References: > [Hive server 2 thrift error - Cloudera Community - > 34293|https://community.cloudera.com/t5/Support-Questions/Hive-server-2-thrift-error/td-p/34293] > Eric Lin blog "“NO DATA OR NO SASL DATA IN THE STREAM” ERROR IN HIVESERVER2 > LOG" > HIVE-12754 AuthTypes.NONE cause exception after HS2 start - ASF JIRA > (apache.org) > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25307) Hive Server 2 crashes when Thrift library encounters particular security protocol issue
[ https://issues.apache.org/jira/browse/HIVE-25307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-25307: Description: A RuntimeException is thrown by the Thrift library that causes Hive Server 2 to crash on our customer's machine. If you Google this the exception has been reported a couple of times over the years but not fixed. A blog (see references below) says it is an occasional security protocol issue between Hive Server 2 and a proxy like a Gateway. One challenge in the older 0.9.3 Thrift version was the Thrift TTransportFactory getTransport method declaration had throws no Exceptions. Hence the likely choice of RuntimeException. But that Exception is fatal to Hive Server 2. The proposed fix is a work around is we catch RuntimeException in the inner class TUGIAssumingTransportFactory of the HadoopThriftAuthBridge class in Hive Server 2. And throw a throw the RuntimeException's (inner) cause (e.g. TSaslTransportException) as a TTransportException. Once the Thrift library stops throwing RuntimeException or we catch fatal Throwable exceptions in the Thrift library's TThreadPoolServer's inner class WorkerProcess run method and display them, the RuntimeException try/catch clause can be removed. ExceptionClassName: java.lang.RuntimeException ExceptionStackTrace: java.lang.RuntimeException: org.apache.thrift.transport.TSaslTransportException: No data or no sasl data in the stream at org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:219) at org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge.java:694) at org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge.java:691) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:360) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1710) at org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory.getTransport(HadoopThriftAuthBridge.java:691) at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:269) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: org.apache.thrift.transport.TSaslTransportException: No data or no sasl data in the stream at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:326) at org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:41) at org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:216) ... 10 more References: [Hive server 2 thrift error - Cloudera Community - 34293|https://community.cloudera.com/t5/Support-Questions/Hive-server-2-thrift-error/td-p/34293] Eric Lin blog "“NO DATA OR NO SASL DATA IN THE STREAM” ERROR IN HIVESERVER2 LOG" HIVE-12754 AuthTypes.NONE cause exception after HS2 start - ASF JIRA (apache.org) was: A RuntimeException is thrown by the Thrift library that causes Hive Server 2 to crash on our customer's machine. If you Google this the exception has been reported a couple of times over the years but not fixed. A blog (see references below) says it is an occasional security protocol issue between Hive Server 2 and a proxy like a Gateway. One challenge in the older 0.9.3 Thrift version was the Thrift TTransportFactory getTransport method declaration had throws no Exceptions. Hence the likely choice of RuntimeException. But that Exception is fatal to Hive Server 2. The proposed fix is a work around is we catch RuntimeException in the inner class TUGIAssumingTransportFactory of the HadoopThriftAuthBridge class in Hive Server 2. And throw a throw the RuntimeException's (inner) cause as a TTransportException. ExceptionClassName: java.lang.RuntimeException ExceptionStackTrace: java.lang.RuntimeException: org.apache.thrift.transport.TSaslTransportException: No data or no sasl data in the stream at org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:219) at org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge.java:694) at org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge.java:691)
[jira] [Updated] (HIVE-25307) Hive Server 2 crashes when Thrift library encounters particular security protocol issue
[ https://issues.apache.org/jira/browse/HIVE-25307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-25307: Description: A RuntimeException is thrown by the Thrift library that causes Hive Server 2 to crash on our customer's machine. If you Google this the exception has been reported a couple of times over the years but not fixed. A blog (see references below) says it is an occasional security protocol issue between Hive Server 2 and a proxy like a Gateway. One challenge in the older 0.9.3 Thrift version was the Thrift TTransportFactory getTransport method declaration had throws no Exceptions. Hence the likely choice of RuntimeException. But that Exception is fatal to Hive Server 2. The proposed fix is a work around is we catch RuntimeException in the inner class TUGIAssumingTransportFactory of the HadoopThriftAuthBridge class in Hive Server 2. And throw a throw the RuntimeException's (inner) cause as a TTransportException. ExceptionClassName: java.lang.RuntimeException ExceptionStackTrace: java.lang.RuntimeException: org.apache.thrift.transport.TSaslTransportException: No data or no sasl data in the stream at org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:219) at org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge.java:694) at org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge.java:691) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:360) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1710) at org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory.getTransport(HadoopThriftAuthBridge.java:691) at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:269) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: org.apache.thrift.transport.TSaslTransportException: No data or no sasl data in the stream at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:326) at org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:41) at org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:216) ... 10 more References: [Hive server 2 thrift error - Cloudera Community - 34293|https://community.cloudera.com/t5/Support-Questions/Hive-server-2-thrift-error/td-p/34293] Eric Lin blog "“NO DATA OR NO SASL DATA IN THE STREAM” ERROR IN HIVESERVER2 LOG" HIVE-12754 AuthTypes.NONE cause exception after HS2 start - ASF JIRA (apache.org) was: A RuntimeException is thrown by the Thrift library that causes Hive Server 2 to crash on our customer's machine. If you Google this the exception has been reported a couple of times over the years but not fixed. A blog (see references below) says it is an occasional security protocol issue between Hive Server 2 and a proxy like a Gateway. One challenge is the Thrift TTransportFactory getTransport method declaration throws no Exceptions hence the likely choice of RuntimeException. But that Exception is fatal to Hive Server 2. The proposed fix is a work around that catches RuntimeException in Hive Server 2, saves the Exception cause in a dummy TTransport object, and throws the cause when TTransport's open method is called later. ExceptionClassName: java.lang.RuntimeException ExceptionStackTrace: java.lang.RuntimeException: org.apache.thrift.transport.TSaslTransportException: No data or no sasl data in the stream at org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:219) at org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge.java:694) at org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge.java:691) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:360) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1710) at
[jira] [Updated] (HIVE-25307) Hive Server 2 crashes when Thrift library encounters particular security protocol issue
[ https://issues.apache.org/jira/browse/HIVE-25307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-25307: Attachment: (was: hive-thrift-fix-01-3_1.patch) > Hive Server 2 crashes when Thrift library encounters particular security > protocol issue > --- > > Key: HIVE-25307 > URL: https://issues.apache.org/jira/browse/HIVE-25307 > Project: Hive > Issue Type: Bug >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Labels: pull-request-available > Attachments: hive-thrift-fix-02-3_1.patch > > Time Spent: 10m > Remaining Estimate: 0h > > A RuntimeException is thrown by the Thrift library that causes Hive Server 2 > to crash on our customer's machine. If you Google this the exception has been > reported a couple of times over the years but not fixed. A blog (see > references below) says it is an occasional security protocol issue between > Hive Server 2 and a proxy like a Gateway. > One challenge is the Thrift TTransportFactory getTransport method declaration > throws no Exceptions hence the likely choice of RuntimeException. But that > Exception is fatal to Hive Server 2. > The proposed fix is a work around that catches RuntimeException in Hive > Server 2, saves the Exception cause in a dummy TTransport object, and throws > the cause when TTransport's open method is called later. > > ExceptionClassName: > java.lang.RuntimeException > ExceptionStackTrace: > java.lang.RuntimeException: > org.apache.thrift.transport.TSaslTransportException: No data or no sasl data > in the stream > at > org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:219) > at > org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge.java:694) > at > org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge.java:691) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:360) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1710) > at > org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory.getTransport(HadoopThriftAuthBridge.java:691) > at > org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:269) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > Caused by: org.apache.thrift.transport.TSaslTransportException: No data or no > sasl data in the stream > at > org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:326) > at > org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:41) > at > org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:216) > ... 10 more > > References: > [Hive server 2 thrift error - Cloudera Community - > 34293|https://community.cloudera.com/t5/Support-Questions/Hive-server-2-thrift-error/td-p/34293] > Eric Lin blog "“NO DATA OR NO SASL DATA IN THE STREAM” ERROR IN HIVESERVER2 > LOG" > [HIVE-12754] AuthTypes.NONE cause exception after HS2 start - ASF JIRA > (apache.org) > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25307) Hive Server 2 crashes when Thrift library encounters particular security protocol issue
[ https://issues.apache.org/jira/browse/HIVE-25307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-25307: Attachment: hive-thrift-fix-02-3_1.patch > Hive Server 2 crashes when Thrift library encounters particular security > protocol issue > --- > > Key: HIVE-25307 > URL: https://issues.apache.org/jira/browse/HIVE-25307 > Project: Hive > Issue Type: Bug >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Labels: pull-request-available > Attachments: hive-thrift-fix-01-3_1.patch, > hive-thrift-fix-02-3_1.patch > > Time Spent: 10m > Remaining Estimate: 0h > > A RuntimeException is thrown by the Thrift library that causes Hive Server 2 > to crash on our customer's machine. If you Google this the exception has been > reported a couple of times over the years but not fixed. A blog (see > references below) says it is an occasional security protocol issue between > Hive Server 2 and a proxy like a Gateway. > One challenge is the Thrift TTransportFactory getTransport method declaration > throws no Exceptions hence the likely choice of RuntimeException. But that > Exception is fatal to Hive Server 2. > The proposed fix is a work around that catches RuntimeException in Hive > Server 2, saves the Exception cause in a dummy TTransport object, and throws > the cause when TTransport's open method is called later. > > ExceptionClassName: > java.lang.RuntimeException > ExceptionStackTrace: > java.lang.RuntimeException: > org.apache.thrift.transport.TSaslTransportException: No data or no sasl data > in the stream > at > org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:219) > at > org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge.java:694) > at > org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge.java:691) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:360) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1710) > at > org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory.getTransport(HadoopThriftAuthBridge.java:691) > at > org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:269) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > Caused by: org.apache.thrift.transport.TSaslTransportException: No data or no > sasl data in the stream > at > org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:326) > at > org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:41) > at > org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:216) > ... 10 more > > References: > [Hive server 2 thrift error - Cloudera Community - > 34293|https://community.cloudera.com/t5/Support-Questions/Hive-server-2-thrift-error/td-p/34293] > Eric Lin blog "“NO DATA OR NO SASL DATA IN THE STREAM” ERROR IN HIVESERVER2 > LOG" > [HIVE-12754] AuthTypes.NONE cause exception after HS2 start - ASF JIRA > (apache.org) > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-25307) Hive Server 2 crashes when Thrift library encounters particular security protocol issue
[ https://issues.apache.org/jira/browse/HIVE-25307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17375104#comment-17375104 ] Matt McCline commented on HIVE-25307: - Work-in-progress patch attached. Next step: create a pull request. FYI: [~ashish-kumar-sharma] [~sankarh] > Hive Server 2 crashes when Thrift library encounters particular security > protocol issue > --- > > Key: HIVE-25307 > URL: https://issues.apache.org/jira/browse/HIVE-25307 > Project: Hive > Issue Type: Bug >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: hive-thrift-fix-01-3_1.patch > > > A RuntimeException is thrown by the Thrift library that causes Hive Server 2 > to crash on our customer's machine. If you Google this the exception has been > reported a couple of times over the years but not fixed. A blog (see > references below) says it is an occasional security protocol issue between > Hive Server 2 and a proxy like a Gateway. > One challenge is the Thrift TTransportFactory getTransport method declaration > throws no Exceptions hence the likely choice of RuntimeException. But that > Exception is fatal to Hive Server 2. > The proposed fix is a work around that catches RuntimeException in Hive > Server 2, saves the Exception cause in a dummy TTransport object, and throws > the cause when TTransport's open method is called later. > > ExceptionClassName: > java.lang.RuntimeException > ExceptionStackTrace: > java.lang.RuntimeException: > org.apache.thrift.transport.TSaslTransportException: No data or no sasl data > in the stream > at > org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:219) > at > org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge.java:694) > at > org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge.java:691) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:360) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1710) > at > org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory.getTransport(HadoopThriftAuthBridge.java:691) > at > org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:269) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > Caused by: org.apache.thrift.transport.TSaslTransportException: No data or no > sasl data in the stream > at > org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:326) > at > org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:41) > at > org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:216) > ... 10 more > > References: > [Hive server 2 thrift error - Cloudera Community - > 34293|https://community.cloudera.com/t5/Support-Questions/Hive-server-2-thrift-error/td-p/34293] > Eric Lin blog "“NO DATA OR NO SASL DATA IN THE STREAM” ERROR IN HIVESERVER2 > LOG" > [HIVE-12754] AuthTypes.NONE cause exception after HS2 start - ASF JIRA > (apache.org) > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25307) Hive Server 2 crashes when Thrift library encounters particular security protocol issue
[ https://issues.apache.org/jira/browse/HIVE-25307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-25307: Attachment: hive-thrift-fix-01-3_1.patch > Hive Server 2 crashes when Thrift library encounters particular security > protocol issue > --- > > Key: HIVE-25307 > URL: https://issues.apache.org/jira/browse/HIVE-25307 > Project: Hive > Issue Type: Bug >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: hive-thrift-fix-01-3_1.patch > > > A RuntimeException is thrown by the Thrift library that causes Hive Server 2 > to crash on our customer's machine. If you Google this the exception has been > reported a couple of times over the years but not fixed. A blog (see > references below) says it is an occasional security protocol issue between > Hive Server 2 and a proxy like a Gateway. > One challenge is the Thrift TTransportFactory getTransport method declaration > throws no Exceptions hence the likely choice of RuntimeException. But that > Exception is fatal to Hive Server 2. > The proposed fix is a work around that catches RuntimeException in Hive > Server 2, saves the Exception cause in a dummy TTransport object, and throws > the cause when TTransport's open method is called later. > > ExceptionClassName: > java.lang.RuntimeException > ExceptionStackTrace: > java.lang.RuntimeException: > org.apache.thrift.transport.TSaslTransportException: No data or no sasl data > in the stream > at > org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:219) > at > org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge.java:694) > at > org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge.java:691) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:360) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1710) > at > org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory.getTransport(HadoopThriftAuthBridge.java:691) > at > org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:269) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > Caused by: org.apache.thrift.transport.TSaslTransportException: No data or no > sasl data in the stream > at > org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:326) > at > org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:41) > at > org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:216) > ... 10 more > > References: > [Hive server 2 thrift error - Cloudera Community - > 34293|https://community.cloudera.com/t5/Support-Questions/Hive-server-2-thrift-error/td-p/34293] > Eric Lin blog "“NO DATA OR NO SASL DATA IN THE STREAM” ERROR IN HIVESERVER2 > LOG" > [HIVE-12754] AuthTypes.NONE cause exception after HS2 start - ASF JIRA > (apache.org) > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-25307) Hive Server 2 crashes when Thrift library encounters particular security protocol issue
[ https://issues.apache.org/jira/browse/HIVE-25307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline reassigned HIVE-25307: --- > Hive Server 2 crashes when Thrift library encounters particular security > protocol issue > --- > > Key: HIVE-25307 > URL: https://issues.apache.org/jira/browse/HIVE-25307 > Project: Hive > Issue Type: Bug >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > > A RuntimeException is thrown by the Thrift library that causes Hive Server 2 > to crash on our customer's machine. If you Google this the exception has been > reported a couple of times over the years but not fixed. A blog (see > references below) says it is an occasional security protocol issue between > Hive Server 2 and a proxy like a Gateway. > One challenge is the Thrift TTransportFactory getTransport method declaration > throws no Exceptions hence the likely choice of RuntimeException. But that > Exception is fatal to Hive Server 2. > The proposed fix is a work around that catches RuntimeException in Hive > Server 2, saves the Exception cause in a dummy TTransport object, and throws > the cause when TTransport's open method is called later. > > ExceptionClassName: > java.lang.RuntimeException > ExceptionStackTrace: > java.lang.RuntimeException: > org.apache.thrift.transport.TSaslTransportException: No data or no sasl data > in the stream > at > org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:219) > at > org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge.java:694) > at > org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge.java:691) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:360) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1710) > at > org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory.getTransport(HadoopThriftAuthBridge.java:691) > at > org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:269) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > Caused by: org.apache.thrift.transport.TSaslTransportException: No data or no > sasl data in the stream > at > org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:326) > at > org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:41) > at > org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:216) > ... 10 more > > References: > [Hive server 2 thrift error - Cloudera Community - > 34293|https://community.cloudera.com/t5/Support-Questions/Hive-server-2-thrift-error/td-p/34293] > Eric Lin blog "“NO DATA OR NO SASL DATA IN THE STREAM” ERROR IN HIVESERVER2 > LOG" > [HIVE-12754] AuthTypes.NONE cause exception after HS2 start - ASF JIRA > (apache.org) > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HIVE-25228) Thrift CLI Service Protocol: Watch for lack of interest by client and kill queries faster
[ https://issues.apache.org/jira/browse/HIVE-25228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline resolved HIVE-25228. - Resolution: Won't Fix Tabling for now. > Thrift CLI Service Protocol: Watch for lack of interest by client and kill > queries faster > - > > Key: HIVE-25228 > URL: https://issues.apache.org/jira/browse/HIVE-25228 > Project: Hive > Issue Type: Improvement >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > > CONSIDER: Have Hive Server 2 monitor operations (queries) for continuing > client interest. If a client does not ask for status every 15 seconds, then > automatically kill a query and release its txn locks and job resources. > > Users will experience queries cleaning up much faster (15 to 30 seconds > instead of minutes and possibly many minutes) when client communication is > lost. Cleaning up those queries prevents other queries from being blocked on > EXCLUSIVE txn locks and blocking of scheduling of their queries including > retries of the original query. Today, users can get timeouts when they retry > a query that got a connection error causing understandably upset users. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HIVE-25227) Thrift CLI Service Protocol: Eliminate long compile requests than proxies can timeout
[
https://issues.apache.org/jira/browse/HIVE-25227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt McCline resolved HIVE-25227.
-
Resolution: Invalid
_HIVE_SERVER2_ASYNC_EXEC_ASYNC_COMPILE_("hive.server2.async.exec.async.compile",
*false*,
"Whether to enable compiling async query asynchronously. If enabled, it is
unknown if the query will have any resultset before compilation completed."),
Asynchronous prepare is already implemented!
> Thrift CLI Service Protocol: Eliminate long compile requests than proxies can
> timeout
> -
>
> Key: HIVE-25227
> URL: https://issues.apache.org/jira/browse/HIVE-25227
> Project: Hive
> Issue Type: Improvement
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Major
>
> CONSIDER: Avoid proxy (GW) timeouts on long Hive query compiles. Use request
> to start the operation; then poll for status like we do for execution.
> E.g. a HTTP Gateway can have arbitrary timeouts and cause queries with long
> compile times to fail.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-25140) Hive Distributed Tracing -- Part 1: Disabled
[ https://issues.apache.org/jira/browse/HIVE-25140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17361327#comment-17361327 ] Matt McCline commented on HIVE-25140: - [~kgyrtkirk] I have thought about this. I think using something like aspect oriented approach would make the Distributed Tracing (DT) feature much weaker since to add Spans you have to learn a new tool and DT could be out of sight and probably get neglected. The pattern of code used to create Spans is well established. OpenTelemetry (OTL) and its predecessors OpenTracing and OpenCensus use this pattern. You see the pattern in the DT books and in numerous tutorials on the web. And a bunch of programming languages are supported by OTL. Also I do not think it adds a lot of code. It does sprinkle changes across the code base yes. The very nature of manually instrumenting code like Hive to do tracing is to start at the top of execution (e.g. BeeLine's SQL Statement) and judicially look for large areas of execution that would provide us benefit from a Span. The decision process for adding a new Span becomes a design process. I think it will be good for people to encounter Span creation in the code and explore what it does. It is easy with IDE autocomplete to add a new Span and go see how it looks in the UI. > Hive Distributed Tracing -- Part 1: Disabled > > > Key: HIVE-25140 > URL: https://issues.apache.org/jira/browse/HIVE-25140 > Project: Hive > Issue Type: Sub-task >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > Attachments: HIVE-25140.01.patch, HIVE-25140.02.patch, > HIVE-25140.03.patch > > > Infrastructure except exporters to Jaeger or OpenTelementry (OTL) due to > Thrift and protobuf version conflicts. A logging only exporter is used. > There are Spans for BeeLine and Hive. Server 2. The code was developed on > branch-3.1 and porting Spans to the Hive MetaStore on master is taking more > time due to major metastore code refactoring. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25237) Thrift CLI Service Protocol: Enhance HTTP variant to be more resilient
[ https://issues.apache.org/jira/browse/HIVE-25237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-25237: Summary: Thrift CLI Service Protocol: Enhance HTTP variant to be more resilient (was: Thrift CLI Service Protocol: Enhance HTTP variant) > Thrift CLI Service Protocol: Enhance HTTP variant to be more resilient > -- > > Key: HIVE-25237 > URL: https://issues.apache.org/jira/browse/HIVE-25237 > Project: Hive > Issue Type: Improvement >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > > I have been thinking about the (Thrift) CLI Service protocol between the > client and server. > Cloudera's Prashanth Jayachandran (private e-mail) told me that its original > BINARY (TCP/IP) transport is designed +_differently_+ than the newer HTTP > transport. HTTP is used when we go through a Gateway. The design for HTTP is > stateless and different in nature than the direct BINARY TCP/IP connection. > Which means today when we see that a Hive Server 2 response to a HTTP query > request can be lost and that is part of the design... It is the WARNING we > have seen when the Gateway drops its HTTP connection to Hive Server 2. We had > been thinking this was a bug but it is by design. > I think the HTTP design needs a rethink. > When I worked for Tandem computers a long time ago messages were > fault-tolerant. They used a message sequence #. When you send a message to a > Tandem server it is a process pair. The message gets routed to the current > process called the primary. The primary computes the message work and tells > the backup process to remember the results before replying in case there is a > failure. You can see where this goes -- if there is a failure before the > client gets the result it retries and the backup process can resiliently give > back the result the primary sent it. This isn't unique to Tandem -- without a > process-pair -- this is a general resilient protocol. > In the HTTP design says message lost is possible both directions (request and > response). I think we adopt a better scheme but not necessarily a process > pair. > The first principle of rethink is the +_client_+ needs to generate a new > operation num (an integer) that replaces the server-side generated random > GUID. And the client generates a new msg num within its new operation. So > beeline might say ExecuteStatement operationNum = 57 NEW, operationMsgNum = > 1. If the client gets an OS connection kind of error, it retries with those > (57, 1) numbers. Hive Server 2 will remember the last response. When Hive > Server 2 gets a message, there are 3 cases: > 1) The sessionId GUID is not valid -- for now we reject the request because > it is likely Hive Server 2 killed the session perhaps because it was > restarted. > 2) The operationNum or operationMsgNum is new. (Assert the msg num increases > monotonically.) Perform the request and save the response. And respond. > 3) The (operationNum, operationMsgNum) matches the last request. Resiliently > respond with the saved result. > I think this message handling is in alignment with the HTTP stateless and any > messages in-between can be lost philosophy. And it will shield the client > from suffering a whole category of message failures that unnecessarily kill > queries. > This also allows to not worry about which request is idempotent or not but > instead requests are resilient. > - > Link to earlier HTTP change: [HIVE-24786: JDBC HttpClient should retry for > idempotent and unsent http methods by prasanthj · Pull Request #1983 · > apache/hive (github.com)|https://github.com/apache/hive/pull/1983/files] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-25237) Thrift CLI Service Protocol: Enhance HTTP variant
[ https://issues.apache.org/jira/browse/HIVE-25237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline reassigned HIVE-25237: --- > Thrift CLI Service Protocol: Enhance HTTP variant > - > > Key: HIVE-25237 > URL: https://issues.apache.org/jira/browse/HIVE-25237 > Project: Hive > Issue Type: Improvement >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > > I have been thinking about the (Thrift) CLI Service protocol between the > client and server. > Cloudera's Prashanth Jayachandran (private e-mail) told me that its original > BINARY (TCP/IP) transport is designed +_differently_+ than the newer HTTP > transport. HTTP is used when we go through a Gateway. The design for HTTP is > stateless and different in nature than the direct BINARY TCP/IP connection. > Which means today when we see that a Hive Server 2 response to a HTTP query > request can be lost and that is part of the design... It is the WARNING we > have seen when the Gateway drops its HTTP connection to Hive Server 2. We had > been thinking this was a bug but it is by design. > I think the HTTP design needs a rethink. > When I worked for Tandem computers a long time ago messages were > fault-tolerant. They used a message sequence #. When you send a message to a > Tandem server it is a process pair. The message gets routed to the current > process called the primary. The primary computes the message work and tells > the backup process to remember the results before replying in case there is a > failure. You can see where this goes -- if there is a failure before the > client gets the result it retries and the backup process can resiliently give > back the result the primary sent it. This isn't unique to Tandem -- without a > process-pair -- this is a general resilient protocol. > In the HTTP design says message lost is possible both directions (request and > response). I think we adopt a better scheme but not necessarily a process > pair. > The first principle of rethink is the +_client_+ needs to generate a new > operation num (an integer) that replaces the server-side generated random > GUID. And the client generates a new msg num within its new operation. So > beeline might say ExecuteStatement operationNum = 57 NEW, operationMsgNum = > 1. If the client gets an OS connection kind of error, it retries with those > (57, 1) numbers. Hive Server 2 will remember the last response. When Hive > Server 2 gets a message, there are 3 cases: > 1) The sessionId GUID is not valid -- for now we reject the request because > it is likely Hive Server 2 killed the session perhaps because it was > restarted. > 2) The operationNum or operationMsgNum is new. (Assert the msg num increases > monotonically.) Perform the request and save the response. And respond. > 3) The (operationNum, operationMsgNum) matches the last request. Resiliently > respond with the saved result. > I think this message handling is in alignment with the HTTP stateless and any > messages in-between can be lost philosophy. And it will shield the client > from suffering a whole category of message failures that unnecessarily kill > queries. > This also allows to not worry about which request is idempotent or not but > instead requests are resilient. > - > Link to earlier HTTP change: [HIVE-24786: JDBC HttpClient should retry for > idempotent and unsent http methods by prasanthj · Pull Request #1983 · > apache/hive (github.com)|https://github.com/apache/hive/pull/1983/files] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HIVE-25191) Modernize Hive Thrift CLI Service Protocol
[ https://issues.apache.org/jira/browse/HIVE-25191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline resolved HIVE-25191. - Resolution: Invalid > Modernize Hive Thrift CLI Service Protocol > -- > > Key: HIVE-25191 > URL: https://issues.apache.org/jira/browse/HIVE-25191 > Project: Hive > Issue Type: Improvement >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > > Unnecessary errors are occurring with the advent of proxy use such as > Gateways between the Hive client and Hive Server 2. Query failures can be due > to arbitrary proxy timeouts. This proposal avoids the timeouts by changing > the protocol to do regular polling. Currently, the Hive client uses one > request for the query compile request. Long query compile times make those > requests vulnerable to the arbitrary proxy timeouts. > Another issue is Hive Server 2 sometimes does not notice the client has > failed or has lost interest in a potentially long running query. This causes > Hive locks and Big Data query resources to be held unnecessarily. The > assumption is the client issues a cancel query request when it gets an error. > This assumption does not always hold. If the proxy returned an error itself, > that proxy may reject the subsequent cancel request, too. And, if the client > is killed or the network is down, the client cannot complete a cancel > request. The proposed solution here is for Hive Server 2 to watch that the > client is sending regular polling requests for status. If a client ceases > those requests, then Hive Server 2 will cancel the query. > Hive owns the JDBC path (i.e. HiveDriver). The ODBC path may be more > challenging because vendors provide ODBC drivers and Hive does not own the > ODBC protocol. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-25191) Modernize Hive Thrift CLI Service Protocol
[ https://issues.apache.org/jira/browse/HIVE-25191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17360326#comment-17360326 ] Matt McCline commented on HIVE-25191: - Closing this Jira for more specific JIRAs: [HIVE-25227] Thrift CLI Service Protocol: Eliminate long compile requests than proxies can timeout - ASF JIRA (apache.org) [HIVE-25228] Thrift CLI Service Protocol: Watch for lack of interest by client and kill queries faster - ASF JIRA (apache.org) > Modernize Hive Thrift CLI Service Protocol > -- > > Key: HIVE-25191 > URL: https://issues.apache.org/jira/browse/HIVE-25191 > Project: Hive > Issue Type: Improvement >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > > Unnecessary errors are occurring with the advent of proxy use such as > Gateways between the Hive client and Hive Server 2. Query failures can be due > to arbitrary proxy timeouts. This proposal avoids the timeouts by changing > the protocol to do regular polling. Currently, the Hive client uses one > request for the query compile request. Long query compile times make those > requests vulnerable to the arbitrary proxy timeouts. > Another issue is Hive Server 2 sometimes does not notice the client has > failed or has lost interest in a potentially long running query. This causes > Hive locks and Big Data query resources to be held unnecessarily. The > assumption is the client issues a cancel query request when it gets an error. > This assumption does not always hold. If the proxy returned an error itself, > that proxy may reject the subsequent cancel request, too. And, if the client > is killed or the network is down, the client cannot complete a cancel > request. The proposed solution here is for Hive Server 2 to watch that the > client is sending regular polling requests for status. If a client ceases > those requests, then Hive Server 2 will cancel the query. > Hive owns the JDBC path (i.e. HiveDriver). The ODBC path may be more > challenging because vendors provide ODBC drivers and Hive does not own the > ODBC protocol. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-25228) Thrift CLI Service Protocol: Watch for lack of interest by client and kill queries faster
[ https://issues.apache.org/jira/browse/HIVE-25228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline reassigned HIVE-25228: --- > Thrift CLI Service Protocol: Watch for lack of interest by client and kill > queries faster > - > > Key: HIVE-25228 > URL: https://issues.apache.org/jira/browse/HIVE-25228 > Project: Hive > Issue Type: Improvement >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > > CONSIDER: Have Hive Server 2 monitor operations (queries) for continuing > client interest. If a client does not ask for status every 15 seconds, then > automatically kill a query and release its txn locks and job resources. > > Users will experience queries cleaning up much faster (15 to 30 seconds > instead of minutes and possibly many minutes) when client communication is > lost. Cleaning up those queries prevents other queries from being blocked on > EXCLUSIVE txn locks and blocking of scheduling of their queries including > retries of the original query. Today, users can get timeouts when they retry > a query that got a connection error causing understandably upset users. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25227) Thrift CLI Service Protocol: Eliminate long compile requests than proxies can timeout
[ https://issues.apache.org/jira/browse/HIVE-25227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-25227: Description: CONSIDER: Avoid proxy (GW) timeouts on long Hive query compiles. Use request to start the operation; then poll for status like we do for execution. E.g. a HTTP Gateway can have arbitrary timeouts and cause queries with long compile times to fail. was:CONSIDER: Avoid proxy (GW) timeouts on long Hive query compiles. Use request to start the operation; then poll for status like we do for execution. > Thrift CLI Service Protocol: Eliminate long compile requests than proxies can > timeout > - > > Key: HIVE-25227 > URL: https://issues.apache.org/jira/browse/HIVE-25227 > Project: Hive > Issue Type: Improvement >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > > CONSIDER: Avoid proxy (GW) timeouts on long Hive query compiles. Use request > to start the operation; then poll for status like we do for execution. > E.g. a HTTP Gateway can have arbitrary timeouts and cause queries with long > compile times to fail. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-25227) Thrift CLI Service Protocol: Eliminate long compile requests than proxies can timeout
[ https://issues.apache.org/jira/browse/HIVE-25227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline reassigned HIVE-25227: --- > Thrift CLI Service Protocol: Eliminate long compile requests than proxies can > timeout > - > > Key: HIVE-25227 > URL: https://issues.apache.org/jira/browse/HIVE-25227 > Project: Hive > Issue Type: Improvement >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > > CONSIDER: Avoid proxy (GW) timeouts on long Hive query compiles. Use request > to start the operation; then poll for status like we do for execution. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-25196) Native Vectorization of GenericUDFSplit function
[ https://issues.apache.org/jira/browse/HIVE-25196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline reassigned HIVE-25196: --- > Native Vectorization of GenericUDFSplit function > > > Key: HIVE-25196 > URL: https://issues.apache.org/jira/browse/HIVE-25196 > Project: Hive > Issue Type: Improvement >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > > Provide faster 'split' function for vector-mode. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-25191) Modernize Hive Thrift CLI Service Protocol
[ https://issues.apache.org/jira/browse/HIVE-25191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17356510#comment-17356510 ] Matt McCline commented on HIVE-25191: - [~thejas] Thank you very much for your comments! – studying. > Modernize Hive Thrift CLI Service Protocol > -- > > Key: HIVE-25191 > URL: https://issues.apache.org/jira/browse/HIVE-25191 > Project: Hive > Issue Type: Improvement >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > > Unnecessary errors are occurring with the advent of proxy use such as > Gateways between the Hive client and Hive Server 2. Query failures can be due > to arbitrary proxy timeouts. This proposal avoids the timeouts by changing > the protocol to do regular polling. Currently, the Hive client uses one > request for the query compile request. Long query compile times make those > requests vulnerable to the arbitrary proxy timeouts. > Another issue is Hive Server 2 sometimes does not notice the client has > failed or has lost interest in a potentially long running query. This causes > Hive locks and Big Data query resources to be held unnecessarily. The > assumption is the client issues a cancel query request when it gets an error. > This assumption does not always hold. If the proxy returned an error itself, > that proxy may reject the subsequent cancel request, too. And, if the client > is killed or the network is down, the client cannot complete a cancel > request. The proposed solution here is for Hive Server 2 to watch that the > client is sending regular polling requests for status. If a client ceases > those requests, then Hive Server 2 will cancel the query. > Hive owns the JDBC path (i.e. HiveDriver). The ODBC path may be more > challenging because vendors provide ODBC drivers and Hive does not own the > ODBC protocol. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25191) Modernize Hive Thrift CLI Service Protocol
[ https://issues.apache.org/jira/browse/HIVE-25191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-25191: Issue Type: Improvement (was: Bug) > Modernize Hive Thrift CLI Service Protocol > -- > > Key: HIVE-25191 > URL: https://issues.apache.org/jira/browse/HIVE-25191 > Project: Hive > Issue Type: Improvement >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > > Unnecessary errors are occurring with the advent of proxy use such as > Gateways between the Hive client and Hive Server 2. Query failures can be due > to arbitrary proxy timeouts. This proposal avoids the timeouts by changing > the protocol to do regular polling. Currently, the Hive client uses one > request for the query compile request. Long query compile times make those > requests vulnerable to the arbitrary proxy timeouts. > Another issue is Hive Server 2 sometimes does not notice the client has > failed or has lost interest in a potentially long running query. This causes > Hive locks and Big Data query resources to be held unnecessarily. The > assumption is the client issues a cancel query request when it gets an error. > This assumption does not always hold. If the proxy returned an error itself, > that proxy may reject the subsequent cancel request, too. And, if the client > is killed or the network is down, the client cannot complete a cancel > request. The proposed solution here is for Hive Server 2 to watch that the > client is sending regular polling requests for status. If a client ceases > those requests, then Hive Server 2 will cancel the query. > Hive owns the JDBC path (i.e. HiveDriver). The ODBC path may be more > challenging because vendors provide ODBC drivers and Hive does not own the > ODBC protocol. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-25191) Modernize Hive Thrift CLI Service Protocol
[ https://issues.apache.org/jira/browse/HIVE-25191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline reassigned HIVE-25191: --- > Modernize Hive Thrift CLI Service Protocol > -- > > Key: HIVE-25191 > URL: https://issues.apache.org/jira/browse/HIVE-25191 > Project: Hive > Issue Type: Bug >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > > Unnecessary errors are occurring with the advent of proxy use such as > Gateways between the Hive client and Hive Server 2. Query failures can be due > to arbitrary proxy timeouts. This proposal avoids the timeouts by changing > the protocol to do regular polling. Currently, the Hive client uses one > request for the query compile request. Long query compile times make those > requests vulnerable to the arbitrary proxy timeouts. > Another issue is Hive Server 2 sometimes does not notice the client has > failed or has lost interest in a potentially long running query. This causes > Hive locks and Big Data query resources to be held unnecessarily. The > assumption is the client issues a cancel query request when it gets an error. > This assumption does not always hold. If the proxy returned an error itself, > that proxy may reject the subsequent cancel request, too. And, if the client > is killed or the network is down, the client cannot complete a cancel > request. The proposed solution here is for Hive Server 2 to watch that the > client is sending regular polling requests for status. If a client ceases > those requests, then Hive Server 2 will cancel the query. > Hive owns the JDBC path (i.e. HiveDriver). The ODBC path may be more > challenging because vendors provide ODBC drivers and Hive does not own the > ODBC protocol. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-25190) BytesColumnVector fails when the aggregate size is > 1gb
[
https://issues.apache.org/jira/browse/HIVE-25190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17355902#comment-17355902
]
Matt McCline commented on HIVE-25190:
-
Oh I see – approaching 1/2 of java.lang.Integer MAX_VALUE = 2147483647.
> BytesColumnVector fails when the aggregate size is > 1gb
>
>
> Key: HIVE-25190
> URL: https://issues.apache.org/jira/browse/HIVE-25190
> Project: Hive
> Issue Type: Bug
>Reporter: Owen O'Malley
>Assignee: Owen O'Malley
>Priority: Major
>
> Currently, BytesColumnVector will allocate a buffer for small values (< 1mb),
> but fail with:
> {code:java}
> new RuntimeException("Overflow of newLength. smallBuffer.length="
> + smallBuffer.length + ", nextElemLength=" + nextElemLength);
> {code:java}
> if the aggregate size of the buffer crosses over 1gb.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-25140) Hive Distributed Tracing -- Part 1: Disabled
[ https://issues.apache.org/jira/browse/HIVE-25140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-25140: Attachment: HIVE-25140.03.patch > Hive Distributed Tracing -- Part 1: Disabled > > > Key: HIVE-25140 > URL: https://issues.apache.org/jira/browse/HIVE-25140 > Project: Hive > Issue Type: Sub-task >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > Attachments: HIVE-25140.01.patch, HIVE-25140.02.patch, > HIVE-25140.03.patch > > > Infrastructure except exporters to Jaeger or OpenTelementry (OTL) due to > Thrift and protobuf version conflicts. A logging only exporter is used. > There are Spans for BeeLine and Hive. Server 2. The code was developed on > branch-3.1 and porting Spans to the Hive MetaStore on master is taking more > time due to major metastore code refactoring. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25140) Hive Distributed Tracing -- Part 1: Disabled
[ https://issues.apache.org/jira/browse/HIVE-25140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-25140: Attachment: HIVE-25140.02.patch > Hive Distributed Tracing -- Part 1: Disabled > > > Key: HIVE-25140 > URL: https://issues.apache.org/jira/browse/HIVE-25140 > Project: Hive > Issue Type: Sub-task >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > Attachments: HIVE-25140.01.patch, HIVE-25140.02.patch > > > Infrastructure except exporters to Jaeger or OpenTelementry (OTL) due to > Thrift and protobuf version conflicts. A logging only exporter is used. > There are Spans for BeeLine and Hive. Server 2. The code was developed on > branch-3.1 and porting Spans to the Hive MetaStore on master is taking more > time due to major metastore code refactoring. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-25093) date_format() UDF is returning values in UTC time zone only
[
https://issues.apache.org/jira/browse/HIVE-25093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17349031#comment-17349031
]
Matt McCline commented on HIVE-25093:
-
[~jcamacho] [~ashutoshc] Can you please review this change yourself since both
of you were in on the original HIVE-12192 design, +*and/or*+ have others look
at this soon (we have a customer inquiring about this as part of a v1.3 to 3.1
upgrade). Thank you.
> date_format() UDF is returning values in UTC time zone only
>
>
> Key: HIVE-25093
> URL: https://issues.apache.org/jira/browse/HIVE-25093
> Project: Hive
> Issue Type: Bug
> Components: UDF
>Affects Versions: 3.1.2
>Reporter: Ashish Sharma
>Assignee: Ashish Sharma
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> *HIVE - 1.2*
> sshuser@hn0-dateti:~$ *timedatectl*
> Local time: Thu 2021-05-06 11:56:08 IST
> Universal time: Thu 2021-05-06 06:26:08 UTC
> RTC time: Thu 2021-05-06 06:26:08
>Time zone: Asia/Kolkata (IST, +0530)
> Network time on: yes
> NTP synchronized: yes
> RTC in local TZ: no
> sshuser@hn0-dateti:~$ beeline
> 0: jdbc:hive2://localhost:10001/default> *select
> date_format(current_timestamp,"-MM-dd HH:mm:ss.SSS z");*
> +--+--+
> | _c0 |
> +--+--+
> | 2021-05-06 11:58:53.760 IST |
> +--+--+
> 1 row selected (1.271 seconds)
> *HIVE - 3.1.0*
> sshuser@hn0-testja:~$ *timedatectl*
> Local time: Thu 2021-05-06 12:03:32 IST
> Universal time: Thu 2021-05-06 06:33:32 UTC
> RTC time: Thu 2021-05-06 06:33:32
>Time zone: Asia/Kolkata (IST, +0530)
> Network time on: yes
> NTP synchronized: yes
> RTC in local TZ: no
> sshuser@hn0-testja:~$ beeline
> 0: jdbc:hive2://zk0-testja.e0mrrixnyxde5h1suy> *select
> date_format(current_timestamp,"-MM-dd HH:mm:ss.SSS z");*
> +--+
> | _c0 |
> +--+
> | *2021-05-06 06:33:59.078 UTC* |
> +--+
> 1 row selected (13.396 seconds)
> 0: jdbc:hive2://zk0-testja.e0mrrixnyxde5h1suy> *set
> hive.local.time.zone=Asia/Kolkata;*
> No rows affected (0.025 seconds)
> 0: jdbc:hive2://zk0-testja.e0mrrixnyxde5h1suy> *select
> date_format(current_timestamp,"-MM-dd HH:mm:ss.SSS z");*
> +--+
> | _c0 |
> +--+
> | *{color:red}2021-05-06 12:08:15.118 UTC{color}* |
> +--+
> 1 row selected (1.074 seconds)
> expected result was *2021-05-06 12:08:15.118 IST*
> As part of HIVE-12192 it was decided to have a common time zone for all
> computation i.e. "UTC". Due to which data_format() function was hard coded to
> "UTC".
> But later in HIVE-21039 it was decided that user session time zone value
> should be the default not UTC.
> date_format() was not fixed as part of HIVE-21039.
> what should be the ideal time zone value of date_format().
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (HIVE-25093) date_format() UDF is returning values in UTC time zone only
[
https://issues.apache.org/jira/browse/HIVE-25093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17349023#comment-17349023
]
Matt McCline edited comment on HIVE-25093 at 5/21/21, 6:56 AM:
---
I think Ashish's basic observation is correct that the original HIVE-12192 says
compute internally with UTC and let the Hive session setting for time zone
instruct us on how to display the date / time (unless overridden by the z / Z
format character). And, his observation that the dateformat function wasn't
change needed to this new model as part of the HIVE-12192 changes is correct.
was (Author: mattmccline):
I think Ashish's basic observation is correct that the original HIVE-12192 says
compute internally with UTC and let the Hive session setting for time zone
instruct us on how to display the date / time (unless overridden by the z / Z
format character). And, his observation that the dateformat function wasn't
change needed to this new model as part of the HIVE-12192 changes.
> date_format() UDF is returning values in UTC time zone only
>
>
> Key: HIVE-25093
> URL: https://issues.apache.org/jira/browse/HIVE-25093
> Project: Hive
> Issue Type: Bug
> Components: UDF
>Affects Versions: 3.1.2
>Reporter: Ashish Sharma
>Assignee: Ashish Sharma
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> *HIVE - 1.2*
> sshuser@hn0-dateti:~$ *timedatectl*
> Local time: Thu 2021-05-06 11:56:08 IST
> Universal time: Thu 2021-05-06 06:26:08 UTC
> RTC time: Thu 2021-05-06 06:26:08
>Time zone: Asia/Kolkata (IST, +0530)
> Network time on: yes
> NTP synchronized: yes
> RTC in local TZ: no
> sshuser@hn0-dateti:~$ beeline
> 0: jdbc:hive2://localhost:10001/default> *select
> date_format(current_timestamp,"-MM-dd HH:mm:ss.SSS z");*
> +--+--+
> | _c0 |
> +--+--+
> | 2021-05-06 11:58:53.760 IST |
> +--+--+
> 1 row selected (1.271 seconds)
> *HIVE - 3.1.0*
> sshuser@hn0-testja:~$ *timedatectl*
> Local time: Thu 2021-05-06 12:03:32 IST
> Universal time: Thu 2021-05-06 06:33:32 UTC
> RTC time: Thu 2021-05-06 06:33:32
>Time zone: Asia/Kolkata (IST, +0530)
> Network time on: yes
> NTP synchronized: yes
> RTC in local TZ: no
> sshuser@hn0-testja:~$ beeline
> 0: jdbc:hive2://zk0-testja.e0mrrixnyxde5h1suy> *select
> date_format(current_timestamp,"-MM-dd HH:mm:ss.SSS z");*
> +--+
> | _c0 |
> +--+
> | *2021-05-06 06:33:59.078 UTC* |
> +--+
> 1 row selected (13.396 seconds)
> 0: jdbc:hive2://zk0-testja.e0mrrixnyxde5h1suy> *set
> hive.local.time.zone=Asia/Kolkata;*
> No rows affected (0.025 seconds)
> 0: jdbc:hive2://zk0-testja.e0mrrixnyxde5h1suy> *select
> date_format(current_timestamp,"-MM-dd HH:mm:ss.SSS z");*
> +--+
> | _c0 |
> +--+
> | *{color:red}2021-05-06 12:08:15.118 UTC{color}* |
> +--+
> 1 row selected (1.074 seconds)
> expected result was *2021-05-06 12:08:15.118 IST*
> As part of HIVE-12192 it was decided to have a common time zone for all
> computation i.e. "UTC". Due to which data_format() function was hard coded to
> "UTC".
> But later in HIVE-21039 it was decided that user session time zone value
> should be the default not UTC.
> date_format() was not fixed as part of HIVE-21039.
> what should be the ideal time zone value of date_format().
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-25093) date_format() UDF is returning values in UTC time zone only
[
https://issues.apache.org/jira/browse/HIVE-25093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17349023#comment-17349023
]
Matt McCline commented on HIVE-25093:
-
I think Ashish's basic observation is correct that the original HIVE-12192 says
compute internally with UTC and let the Hive session setting for time zone
instruct us on how to display the date / time (unless overridden by the z / Z
format character). And, his observation that the dateformat function wasn't
change needed to this new model as part of the HIVE-12192 changes.
> date_format() UDF is returning values in UTC time zone only
>
>
> Key: HIVE-25093
> URL: https://issues.apache.org/jira/browse/HIVE-25093
> Project: Hive
> Issue Type: Bug
> Components: UDF
>Affects Versions: 3.1.2
>Reporter: Ashish Sharma
>Assignee: Ashish Sharma
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> *HIVE - 1.2*
> sshuser@hn0-dateti:~$ *timedatectl*
> Local time: Thu 2021-05-06 11:56:08 IST
> Universal time: Thu 2021-05-06 06:26:08 UTC
> RTC time: Thu 2021-05-06 06:26:08
>Time zone: Asia/Kolkata (IST, +0530)
> Network time on: yes
> NTP synchronized: yes
> RTC in local TZ: no
> sshuser@hn0-dateti:~$ beeline
> 0: jdbc:hive2://localhost:10001/default> *select
> date_format(current_timestamp,"-MM-dd HH:mm:ss.SSS z");*
> +--+--+
> | _c0 |
> +--+--+
> | 2021-05-06 11:58:53.760 IST |
> +--+--+
> 1 row selected (1.271 seconds)
> *HIVE - 3.1.0*
> sshuser@hn0-testja:~$ *timedatectl*
> Local time: Thu 2021-05-06 12:03:32 IST
> Universal time: Thu 2021-05-06 06:33:32 UTC
> RTC time: Thu 2021-05-06 06:33:32
>Time zone: Asia/Kolkata (IST, +0530)
> Network time on: yes
> NTP synchronized: yes
> RTC in local TZ: no
> sshuser@hn0-testja:~$ beeline
> 0: jdbc:hive2://zk0-testja.e0mrrixnyxde5h1suy> *select
> date_format(current_timestamp,"-MM-dd HH:mm:ss.SSS z");*
> +--+
> | _c0 |
> +--+
> | *2021-05-06 06:33:59.078 UTC* |
> +--+
> 1 row selected (13.396 seconds)
> 0: jdbc:hive2://zk0-testja.e0mrrixnyxde5h1suy> *set
> hive.local.time.zone=Asia/Kolkata;*
> No rows affected (0.025 seconds)
> 0: jdbc:hive2://zk0-testja.e0mrrixnyxde5h1suy> *select
> date_format(current_timestamp,"-MM-dd HH:mm:ss.SSS z");*
> +--+
> | _c0 |
> +--+
> | *{color:red}2021-05-06 12:08:15.118 UTC{color}* |
> +--+
> 1 row selected (1.074 seconds)
> expected result was *2021-05-06 12:08:15.118 IST*
> As part of HIVE-12192 it was decided to have a common time zone for all
> computation i.e. "UTC". Due to which data_format() function was hard coded to
> "UTC".
> But later in HIVE-21039 it was decided that user session time zone value
> should be the default not UTC.
> date_format() was not fixed as part of HIVE-21039.
> what should be the ideal time zone value of date_format().
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-25140) Hive Distributed Tracing -- Part 1: Disabled
[ https://issues.apache.org/jira/browse/HIVE-25140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-25140: Status: Patch Available (was: Open) Work-In-Progress (WIP) first patch to do a Hive QA run. Not ready for code review yet. > Hive Distributed Tracing -- Part 1: Disabled > > > Key: HIVE-25140 > URL: https://issues.apache.org/jira/browse/HIVE-25140 > Project: Hive > Issue Type: Sub-task >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > Attachments: HIVE-25140.01.patch > > > Infrastructure except exporters to Jaeger or OpenTelementry (OTL) due to > Thrift and protobuf version conflicts. A logging only exporter is used. > There are Spans for BeeLine and Hive. Server 2. The code was developed on > branch-3.1 and porting Spans to the Hive MetaStore on master is taking more > time due to major metastore code refactoring. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25140) Hive Distributed Tracing -- Part 1: Disabled
[ https://issues.apache.org/jira/browse/HIVE-25140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-25140: Attachment: HIVE-25140.01.patch > Hive Distributed Tracing -- Part 1: Disabled > > > Key: HIVE-25140 > URL: https://issues.apache.org/jira/browse/HIVE-25140 > Project: Hive > Issue Type: Sub-task >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > Attachments: HIVE-25140.01.patch > > > Infrastructure except exporters to Jaeger or OpenTelementry (OTL) due to > Thrift and protobuf version conflicts. A logging only exporter is used. > There are Spans for BeeLine and Hive. Server 2. The code was developed on > branch-3.1 and porting Spans to the Hive MetaStore on master is taking more > time due to major metastore code refactoring. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25140) Hive Distributed Tracing -- Part 1: Disabled
[ https://issues.apache.org/jira/browse/HIVE-25140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-25140: Description: Infrastructure except exporters to Jaeger or OpenTelementry (OTL) due to Thrift and protobuf version conflicts. A logging only exporter is used. There are Spans for BeeLine and Hive. Server 2. The code was developed on branch-3.1 and porting Spans to the Hive MetaStore on master is taking more time due to major metastore code refactoring. was: Infrastructure except exporters to Jaeger or OpenTelementry (OTL) due to Thrift and protobuf version conflicts. Has Spans for BeeLine and Hive. Server 2. The code was developed on branch-3.1 and porting Spans to the Hive MetaStore on master is taking more time due to major code refactoring. > Hive Distributed Tracing -- Part 1: Disabled > > > Key: HIVE-25140 > URL: https://issues.apache.org/jira/browse/HIVE-25140 > Project: Hive > Issue Type: Sub-task >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > > Infrastructure except exporters to Jaeger or OpenTelementry (OTL) due to > Thrift and protobuf version conflicts. A logging only exporter is used. > There are Spans for BeeLine and Hive. Server 2. The code was developed on > branch-3.1 and porting Spans to the Hive MetaStore on master is taking more > time due to major metastore code refactoring. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-25140) Hive Distributed Tracing -- Part 1: Disabled
[ https://issues.apache.org/jira/browse/HIVE-25140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline reassigned HIVE-25140: --- > Hive Distributed Tracing -- Part 1: Disabled > > > Key: HIVE-25140 > URL: https://issues.apache.org/jira/browse/HIVE-25140 > Project: Hive > Issue Type: Sub-task >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > > Infrastructure except exporters to Jaeger or OpenTelementry (OTL) due to > Thrift and protobuf version conflicts. > Has Spans for BeeLine and Hive. Server 2. The code was developed on > branch-3.1 and porting Spans to the Hive MetaStore on master is taking more > time due to major code refactoring. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25069) Hive Distributed Tracing
[ https://issues.apache.org/jira/browse/HIVE-25069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-25069: Attachment: (was: HIVE-25069.01.patch) > Hive Distributed Tracing > > > Key: HIVE-25069 > URL: https://issues.apache.org/jira/browse/HIVE-25069 > Project: Hive > Issue Type: New Feature >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > Attachments: image-2021-05-10-09-20-54-688.png, > image-2021-05-10-09-30-44-570.png, image-2021-05-10-19-06-02-679.png > > > Instrument Hive code to gather distributed traces and export trace data to a > configurable collector. > Distributed tracing is a revolutionary tool for debugging issues. > We will use the new OpenTelemetry open-source standard that our industry has > aligned on. OpenTelemetry is the merger of two earlier distributed tracing > projects OpenTracing and OpenCensus. > Next step: Add design document that goes into more detail on the benefits of > distributed tracing and describes how Hive will enhanced. > Also see: > HBASE-22120 Replace HTrace with OpenTelemetry -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Issue Comment Deleted] (HIVE-25069) Hive Distributed Tracing
[ https://issues.apache.org/jira/browse/HIVE-25069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-25069: Comment: was deleted (was: A first Work-in-Progress patch. Work was done on branch-3.1 and manually merging changes to master is tedious. The Tracing infrastructure modules are in but only a few Hive classes have been merged. Enough though to give Hive QA a run. Tracing will exported to a logging-only exporter..) > Hive Distributed Tracing > > > Key: HIVE-25069 > URL: https://issues.apache.org/jira/browse/HIVE-25069 > Project: Hive > Issue Type: New Feature >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > Attachments: image-2021-05-10-09-20-54-688.png, > image-2021-05-10-09-30-44-570.png, image-2021-05-10-19-06-02-679.png > > > Instrument Hive code to gather distributed traces and export trace data to a > configurable collector. > Distributed tracing is a revolutionary tool for debugging issues. > We will use the new OpenTelemetry open-source standard that our industry has > aligned on. OpenTelemetry is the merger of two earlier distributed tracing > projects OpenTracing and OpenCensus. > Next step: Add design document that goes into more detail on the benefits of > distributed tracing and describes how Hive will enhanced. > Also see: > HBASE-22120 Replace HTrace with OpenTelemetry -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Issue Comment Deleted] (HIVE-25069) Hive Distributed Tracing
[ https://issues.apache.org/jira/browse/HIVE-25069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-25069: Comment: was deleted (was: I'll try a pull request instead.) > Hive Distributed Tracing > > > Key: HIVE-25069 > URL: https://issues.apache.org/jira/browse/HIVE-25069 > Project: Hive > Issue Type: New Feature >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > Attachments: image-2021-05-10-09-20-54-688.png, > image-2021-05-10-09-30-44-570.png, image-2021-05-10-19-06-02-679.png > > > Instrument Hive code to gather distributed traces and export trace data to a > configurable collector. > Distributed tracing is a revolutionary tool for debugging issues. > We will use the new OpenTelemetry open-source standard that our industry has > aligned on. OpenTelemetry is the merger of two earlier distributed tracing > projects OpenTracing and OpenCensus. > Next step: Add design document that goes into more detail on the benefits of > distributed tracing and describes how Hive will enhanced. > Also see: > HBASE-22120 Replace HTrace with OpenTelemetry -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25069) Hive Distributed Tracing
[ https://issues.apache.org/jira/browse/HIVE-25069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-25069: Status: Open (was: Patch Available) I'll try a pull request instead. > Hive Distributed Tracing > > > Key: HIVE-25069 > URL: https://issues.apache.org/jira/browse/HIVE-25069 > Project: Hive > Issue Type: New Feature >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > Attachments: HIVE-25069.01.patch, image-2021-05-10-09-20-54-688.png, > image-2021-05-10-09-30-44-570.png, image-2021-05-10-19-06-02-679.png > > > Instrument Hive code to gather distributed traces and export trace data to a > configurable collector. > Distributed tracing is a revolutionary tool for debugging issues. > We will use the new OpenTelemetry open-source standard that our industry has > aligned on. OpenTelemetry is the merger of two earlier distributed tracing > projects OpenTracing and OpenCensus. > Next step: Add design document that goes into more detail on the benefits of > distributed tracing and describes how Hive will enhanced. > Also see: > HBASE-22120 Replace HTrace with OpenTelemetry -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25069) Hive Distributed Tracing
[ https://issues.apache.org/jira/browse/HIVE-25069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-25069: Status: Patch Available (was: In Progress) A first Work-in-Progress patch. Work was done on branch-3.1 and manually merging changes to master is tedious. The Tracing infrastructure modules are in but only a few Hive classes have been merged. Enough though to give Hive QA a run. Tracing will exported to a logging-only exporter.. > Hive Distributed Tracing > > > Key: HIVE-25069 > URL: https://issues.apache.org/jira/browse/HIVE-25069 > Project: Hive > Issue Type: New Feature >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > Attachments: HIVE-25069.01.patch, image-2021-05-10-09-20-54-688.png, > image-2021-05-10-09-30-44-570.png, image-2021-05-10-19-06-02-679.png > > > Instrument Hive code to gather distributed traces and export trace data to a > configurable collector. > Distributed tracing is a revolutionary tool for debugging issues. > We will use the new OpenTelemetry open-source standard that our industry has > aligned on. OpenTelemetry is the merger of two earlier distributed tracing > projects OpenTracing and OpenCensus. > Next step: Add design document that goes into more detail on the benefits of > distributed tracing and describes how Hive will enhanced. > Also see: > HBASE-22120 Replace HTrace with OpenTelemetry -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25069) Hive Distributed Tracing
[ https://issues.apache.org/jira/browse/HIVE-25069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-25069: Attachment: HIVE-25069.01.patch > Hive Distributed Tracing > > > Key: HIVE-25069 > URL: https://issues.apache.org/jira/browse/HIVE-25069 > Project: Hive > Issue Type: New Feature >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > Attachments: HIVE-25069.01.patch, image-2021-05-10-09-20-54-688.png, > image-2021-05-10-09-30-44-570.png, image-2021-05-10-19-06-02-679.png > > > Instrument Hive code to gather distributed traces and export trace data to a > configurable collector. > Distributed tracing is a revolutionary tool for debugging issues. > We will use the new OpenTelemetry open-source standard that our industry has > aligned on. OpenTelemetry is the merger of two earlier distributed tracing > projects OpenTracing and OpenCensus. > Next step: Add design document that goes into more detail on the benefits of > distributed tracing and describes how Hive will enhanced. > Also see: > HBASE-22120 Replace HTrace with OpenTelemetry -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work started] (HIVE-25069) Hive Distributed Tracing
[ https://issues.apache.org/jira/browse/HIVE-25069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-25069 started by Matt McCline. --- > Hive Distributed Tracing > > > Key: HIVE-25069 > URL: https://issues.apache.org/jira/browse/HIVE-25069 > Project: Hive > Issue Type: New Feature >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > Attachments: image-2021-05-10-09-20-54-688.png, > image-2021-05-10-09-30-44-570.png, image-2021-05-10-19-06-02-679.png > > > Instrument Hive code to gather distributed traces and export trace data to a > configurable collector. > Distributed tracing is a revolutionary tool for debugging issues. > We will use the new OpenTelemetry open-source standard that our industry has > aligned on. OpenTelemetry is the merger of two earlier distributed tracing > projects OpenTracing and OpenCensus. > Next step: Add design document that goes into more detail on the benefits of > distributed tracing and describes how Hive will enhanced. > Also see: > HBASE-22120 Replace HTrace with OpenTelemetry -- This message was sent by Atlassian Jira (v8.3.4#803005)
