date:20210227

[jira] [Work logged] (HIVE-24834) Cannot add comment for kafka table

2021-02-27 Thread ASF GitHub Bot (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-24834?focusedWorklogId=559016=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-559016
 ]

ASF GitHub Bot logged work on HIVE-24834:
-

Author: ASF GitHub Bot
Created on: 28/Feb/21 03:55
Start Date: 28/Feb/21 03:55
Worklog Time Spent: 10m 
  Work Description: ChangjiGuo closed pull request #2024:
URL: https://github.com/apache/hive/pull/2024


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 559016)
Time Spent: 40m  (was: 0.5h)

> Cannot add comment for kafka table
> --
>
> Key: HIVE-24834
> URL: https://issues.apache.org/jira/browse/HIVE-24834
> Project: Hive
>  Issue Type: Bug
>  Components: kafka integration
>Affects Versions: 2.3.7
>Reporter: ChangjiGuo
>Assignee: ChangjiGuo
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> when using kafka-handler to create a kafka table, no matter whether the user 
> specifies column comment or not, the comment will become 'from deserializer' 
> when the 'show create table' command is used to view the table structure.
> You can refer to the following example:
> {code:sql}
> CREATE EXTERNAL TABLE `kafka_table`(
>   `id` string, 
>   `info` string comment 'comment 1', 
>   `jsoninfo` struct comment 'comment 2')
> ROW FORMAT SERDE 
>   'org.apache.hadoop.hive.kafka.KafkaSerDe' 
> STORED BY 
>   'org.apache.hadoop.hive.kafka.KafkaStorageHandler' 
> WITH SERDEPROPERTIES ( 
>   'serialization.format'='1')
> LOCATION
>   'hdfs://offlinehdfs/user/hive/warehouse/kafka_table'
> TBLPROPERTIES (
>   'properties.bootstrap.servers'='', 
>   'topic'=''
>   ..)
> {code}
> The result is as follows:
> {code:sql}
> CREATE EXTERNAL TABLE `kafka_table`(
>   `id` string COMMENT 'from deserializer', 
>   `info` string COMMENT 'from deserializer', 
>   `jsoninfo` struct COMMENT 'from deserializer', 
>   `__key` binary COMMENT 'from deserializer', 
>   `__partition` int COMMENT 'from deserializer', 
>   `__offset` bigint COMMENT 'from deserializer', 
>   `__timestamp` bigint COMMENT 'from deserializer')
> ROW FORMAT SERDE 
>   'org.apache.hadoop.hive.kafka.KafkaSerDe' 
> STORED BY 
>   'org.apache.hadoop.hive.kafka.KafkaStorageHandler' 
> WITH SERDEPROPERTIES ( 
>   'serialization.format'='1')
> LOCATION
>   'hdfs://offlinehdfs/user/hive/warehouse/kafka_table'
> TBLPROPERTIES (
>   ...)
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (HIVE-24752) Returned operation's drilldown link may be broken

2021-02-27 Thread ASF GitHub Bot (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-24752?focusedWorklogId=558976=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-558976
 ]

ASF GitHub Bot logged work on HIVE-24752:
-

Author: ASF GitHub Bot
Created on: 27/Feb/21 20:59
Start Date: 27/Feb/21 20:59
Worklog Time Spent: 10m 
  Work Description: searlapati commented on pull request #1958:
URL: https://github.com/apache/hive/pull/1958#issuecomment-787134393


   Please unsubscribe me
   
   On Tue, Feb 16, 2021, 5:49 PM dengzh  @kgyrtkirk  @belugabehr
   >  could you please take a look? thanks...
   >
   > —
   > You are receiving this because you are subscribed to this thread.
   > Reply to this email directly, view it on GitHub
   > , or
   > unsubscribe
   > 

   > .
   >
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 558976)
Time Spent: 2h 40m  (was: 2.5h)

> Returned operation's drilldown link may be broken
> -
>
> Key: HIVE-24752
> URL: https://issues.apache.org/jira/browse/HIVE-24752
> Project: Hive
>  Issue Type: Improvement
>  Components: HiveServer2
>Affects Versions: 4.0.0
>Reporter: Zhihua Deng
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 2h 40m
>  Remaining Estimate: 0h
>
> Since HIVE-23625, the path spec for the query page has changed from 
> _query_page_ to _query_page.html_,
> {code:java}
> webServer.addServlet("query_page", "/query_page.html", 
> QueryProfileServlet.class);{code}
> the drilldown link of the operation returned may be broken if 
> hive.server2.show.operation.drilldown.link is enabled...



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (HIVE-24752) Returned operation's drilldown link may be broken

2021-02-27 Thread ASF GitHub Bot (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-24752?focusedWorklogId=558949=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-558949
 ]

ASF GitHub Bot logged work on HIVE-24752:
-

Author: ASF GitHub Bot
Created on: 27/Feb/21 15:11
Start Date: 27/Feb/21 15:11
Worklog Time Spent: 10m 
  Work Description: dengzhhu653 opened a new pull request #1958:
URL: https://github.com/apache/hive/pull/1958


   https://issues.apache.org/jira/browse/HIVE-24752
   
   
   ### What changes were proposed in this pull request?
   
   
   
   ### Why are the changes needed?
   
   
   
   ### Does this PR introduce _any_ user-facing change?
   
   
   
   ### How was this patch tested?
   Local machine
   
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 558949)
Time Spent: 2.5h  (was: 2h 20m)

> Returned operation's drilldown link may be broken
> -
>
> Key: HIVE-24752
> URL: https://issues.apache.org/jira/browse/HIVE-24752
> Project: Hive
>  Issue Type: Improvement
>  Components: HiveServer2
>Affects Versions: 4.0.0
>Reporter: Zhihua Deng
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 2.5h
>  Remaining Estimate: 0h
>
> Since HIVE-23625, the path spec for the query page has changed from 
> _query_page_ to _query_page.html_,
> {code:java}
> webServer.addServlet("query_page", "/query_page.html", 
> QueryProfileServlet.class);{code}
> the drilldown link of the operation returned may be broken if 
> hive.server2.show.operation.drilldown.link is enabled...



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (HIVE-24752) Returned operation's drilldown link may be broken

2021-02-27 Thread ASF GitHub Bot (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-24752?focusedWorklogId=558948=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-558948
 ]

ASF GitHub Bot logged work on HIVE-24752:
-

Author: ASF GitHub Bot
Created on: 27/Feb/21 15:08
Start Date: 27/Feb/21 15:08
Worklog Time Spent: 10m 
  Work Description: dengzhhu653 closed pull request #1958:
URL: https://github.com/apache/hive/pull/1958


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 558948)
Time Spent: 2h 20m  (was: 2h 10m)

> Returned operation's drilldown link may be broken
> -
>
> Key: HIVE-24752
> URL: https://issues.apache.org/jira/browse/HIVE-24752
> Project: Hive
>  Issue Type: Improvement
>  Components: HiveServer2
>Affects Versions: 4.0.0
>Reporter: Zhihua Deng
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 2h 20m
>  Remaining Estimate: 0h
>
> Since HIVE-23625, the path spec for the query page has changed from 
> _query_page_ to _query_page.html_,
> {code:java}
> webServer.addServlet("query_page", "/query_page.html", 
> QueryProfileServlet.class);{code}
> the drilldown link of the operation returned may be broken if 
> hive.server2.show.operation.drilldown.link is enabled...



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (HIVE-24792) Potential thread leak in Operation

2021-02-27 Thread ASF GitHub Bot (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-24792?focusedWorklogId=558921=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-558921
 ]

ASF GitHub Bot logged work on HIVE-24792:
-

Author: ASF GitHub Bot
Created on: 27/Feb/21 12:16
Start Date: 27/Feb/21 12:16
Worklog Time Spent: 10m 
  Work Description: dengzhhu653 opened a new pull request #1992:
URL: https://github.com/apache/hive/pull/1992


   
   
   ### What changes were proposed in this pull request?
   
   
   
   ### Why are the changes needed?
   
   
   
   ### Does this PR introduce _any_ user-facing change?
   
   
   
   ### How was this patch tested?
   
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 558921)
Time Spent: 1.5h  (was: 1h 20m)

> Potential thread leak in Operation
> --
>
> Key: HIVE-24792
> URL: https://issues.apache.org/jira/browse/HIVE-24792
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2
>Reporter: Zhihua Deng
>Assignee: Zhihua Deng
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 1.5h
>  Remaining Estimate: 0h
>
> The _scheduledExecutorService_  in _Operation_ does not shut down after 
> scheduling delay operationlog cleanup, which may result to thread leak in 
> hiveserver2...



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (HIVE-24752) Returned operation's drilldown link may be broken

2021-02-27 Thread ASF GitHub Bot (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-24752?focusedWorklogId=558920=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-558920
 ]

ASF GitHub Bot logged work on HIVE-24752:
-

Author: ASF GitHub Bot
Created on: 27/Feb/21 12:16
Start Date: 27/Feb/21 12:16
Worklog Time Spent: 10m 
  Work Description: dengzhhu653 opened a new pull request #1958:
URL: https://github.com/apache/hive/pull/1958


   https://issues.apache.org/jira/browse/HIVE-24752
   
   
   ### What changes were proposed in this pull request?
   
   
   
   ### Why are the changes needed?
   
   
   
   ### Does this PR introduce _any_ user-facing change?
   
   
   
   ### How was this patch tested?
   Local machine
   
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 558920)
Time Spent: 2h 10m  (was: 2h)

> Returned operation's drilldown link may be broken
> -
>
> Key: HIVE-24752
> URL: https://issues.apache.org/jira/browse/HIVE-24752
> Project: Hive
>  Issue Type: Improvement
>  Components: HiveServer2
>Affects Versions: 4.0.0
>Reporter: Zhihua Deng
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 2h 10m
>  Remaining Estimate: 0h
>
> Since HIVE-23625, the path spec for the query page has changed from 
> _query_page_ to _query_page.html_,
> {code:java}
> webServer.addServlet("query_page", "/query_page.html", 
> QueryProfileServlet.class);{code}
> the drilldown link of the operation returned may be broken if 
> hive.server2.show.operation.drilldown.link is enabled...



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (HIVE-24752) Returned operation's drilldown link may be broken

2021-02-27 Thread ASF GitHub Bot (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-24752?focusedWorklogId=558919=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-558919
 ]

ASF GitHub Bot logged work on HIVE-24752:
-

Author: ASF GitHub Bot
Created on: 27/Feb/21 12:09
Start Date: 27/Feb/21 12:09
Worklog Time Spent: 10m 
  Work Description: dengzhhu653 closed pull request #1958:
URL: https://github.com/apache/hive/pull/1958


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 558919)
Time Spent: 2h  (was: 1h 50m)

> Returned operation's drilldown link may be broken
> -
>
> Key: HIVE-24752
> URL: https://issues.apache.org/jira/browse/HIVE-24752
> Project: Hive
>  Issue Type: Improvement
>  Components: HiveServer2
>Affects Versions: 4.0.0
>Reporter: Zhihua Deng
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 2h
>  Remaining Estimate: 0h
>
> Since HIVE-23625, the path spec for the query page has changed from 
> _query_page_ to _query_page.html_,
> {code:java}
> webServer.addServlet("query_page", "/query_page.html", 
> QueryProfileServlet.class);{code}
> the drilldown link of the operation returned may be broken if 
> hive.server2.show.operation.drilldown.link is enabled...



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (HIVE-24792) Potential thread leak in Operation

2021-02-27 Thread ASF GitHub Bot (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-24792?focusedWorklogId=558918=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-558918
 ]

ASF GitHub Bot logged work on HIVE-24792:
-

Author: ASF GitHub Bot
Created on: 27/Feb/21 12:08
Start Date: 27/Feb/21 12:08
Worklog Time Spent: 10m 
  Work Description: dengzhhu653 closed pull request #1992:
URL: https://github.com/apache/hive/pull/1992


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 558918)
Time Spent: 1h 20m  (was: 1h 10m)

> Potential thread leak in Operation
> --
>
> Key: HIVE-24792
> URL: https://issues.apache.org/jira/browse/HIVE-24792
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2
>Reporter: Zhihua Deng
>Assignee: Zhihua Deng
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 1h 20m
>  Remaining Estimate: 0h
>
> The _scheduledExecutorService_  in _Operation_ does not shut down after 
> scheduling delay operationlog cleanup, which may result to thread leak in 
> hiveserver2...



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-24750) Create a single copy task for external tables within default DB location

2021-02-27 Thread Pravin Sinha (Jira)



[ 
https://issues.apache.org/jira/browse/HIVE-24750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17292077#comment-17292077
 ] 

Pravin Sinha commented on HIVE-24750:
-

+1

> Create a single copy task for external tables within default DB location
> 
>
> Key: HIVE-24750
> URL: https://issues.apache.org/jira/browse/HIVE-24750
> Project: Hive
>  Issue Type: Improvement
>Reporter: Ayush Saxena
>Assignee: Ayush Saxena
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> Presently we create single task for each table, but for the tables within 
> default DB location, we can copy the DB location in one task.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Assigned] (HIVE-24790) Batch column stats updates to HMS

2021-02-27 Thread Swaminathan Balachandran (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-24790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Swaminathan Balachandran reassigned HIVE-24790:
---

Assignee: (was: Swaminathan Balachandran)

> Batch column stats updates to HMS
> -
>
> Key: HIVE-24790
> URL: https://issues.apache.org/jira/browse/HIVE-24790
> Project: Hive
>  Issue Type: Improvement
>  Components: HiveServer2
>Reporter: Rajesh Balamohan
>Priority: Major
>
> When large number of partitions are inserted/updated, it would be good to 
> batch column statistics updates to HMS.
> Currently, HS2 ends up throwing read timeout exception when updating HMS.
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/stats/ColStatsProcessor.java#L180
>  {noformat}
> Caused by: java.net.SocketTimeoutException: Read timed out
> at java.net.SocketInputStream.socketRead0(Native Method) 
> ~[?:1.8.0_252]
> at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) 
> ~[?:1.8.0_252]
> at java.net.SocketInputStream.read(SocketInputStream.java:171) 
> ~[?:1.8.0_252]
> at java.net.SocketInputStream.read(SocketInputStream.java:141) 
> ~[?:1.8.0_252]
> at java.io.BufferedInputStream.fill(BufferedInputStream.java:246) 
> ~[?:1.8.0_252]
> at java.io.BufferedInputStream.read1(BufferedInputStream.java:286) 
> ~[?:1.8.0_252]
> at java.io.BufferedInputStream.read(BufferedInputStream.java:345) 
> ~[?:1.8.0_252]
> at 
> org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:127)
>  ~[hive-exec-3.1]
> at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86) 
> ~[hive-exec-3.1]
> at 
> org.apache.thrift.transport.TSaslTransport.readLength(TSaslTransport.java:374)
>  ~[hive-exec-3.1]
> at 
> org.apache.thrift.transport.TSaslTransport.readFrame(TSaslTransport.java:451) 
> ~[hive-exec-3.1]
> at 
> org.apache.thrift.transport.TSaslTransport.read(TSaslTransport.java:433) 
> ~[hive-exec-3.1]
> at 
> org.apache.thrift.transport.TSaslClientTransport.read(TSaslClientTransport.java:37)
>  ~[hive-exec-3.1]
> at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86) 
> ~[hive-exec-3.1]
> at 
> org.apache.hadoop.hive.metastore.security.TFilterTransport.readAll(TFilterTransport.java:62)
>  ~[hive-exec-3.1]
> at 
> org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:429) 
> ~[hive-exec-3.1]
> at 
> org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:318) 
> ~[hive-exec-3.1]
> at 
> org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:219)
>  ~[hive-exec-3.1]
> at 
> org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:77) 
> ~[hive-exec-3.1]
> at 
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_set_aggr_stats_for(ThriftHiveMetastore.java:4561)
>  ~[hive-exec-3.1]
> at 
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.set_aggr_stats_for(ThriftHiveMetastore.java:4548)
>  ~[hive-exec-3.1]
> at 
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.setPartitionColumnStatistics(HiveMetaStoreClient.java:2496)
>  ~[hive-exec-3.1]
> at 
> org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.setPartitionColumnStatistics(SessionHiveMetaStoreClient.java:515)
>  ~[hive-exec-3.1]
> at sun.reflect.GeneratedMethodAccessor194.invoke(Unknown Source) 
> ~[?:?]
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>  ~[?:1.8.0_252]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_252]
> at 
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:212)
>  ~[hive-exec-3.1]
> at com.sun.proxy.$Proxy60.setPartitionColumnStatistics(Unknown 
> Source) ~[?:?]
> at sun.reflect.GeneratedMethodAccessor194.invoke(Unknown Source) 
> ~[?:?]
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>  ~[?:1.8.0_252]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_252]
> at 
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient$SynchronizedHandler.invoke(HiveMetaStoreClient.java:3431)
>  ~[hive-exec-3.1]
> at com.sun.proxy.$Proxy60.setPartitionColumnStatistics(Unknown 
> Source) ~[?:?]
> at 
> org.apache.hadoop.hive.ql.metadata.Hive.setPartitionColumnStatistics(Hive.java:5213)
>  ~[hive-exec-3.1]
> at 
> org.apache.hadoop.hive.ql.stats.ColStatsProcessor.persistColumnStats(ColStatsProcessor.java:192)
>  ~[hive-exec-3.1]
> at 
>

[jira] [Assigned] (HIVE-24790) Batch column stats updates to HMS

2021-02-27 Thread Swaminathan Balachandran (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-24790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Swaminathan Balachandran reassigned HIVE-24790:
---

Assignee: Swaminathan Balachandran

> Batch column stats updates to HMS
> -
>
> Key: HIVE-24790
> URL: https://issues.apache.org/jira/browse/HIVE-24790
> Project: Hive
>  Issue Type: Improvement
>  Components: HiveServer2
>Reporter: Rajesh Balamohan
>Assignee: Swaminathan Balachandran
>Priority: Major
>
> When large number of partitions are inserted/updated, it would be good to 
> batch column statistics updates to HMS.
> Currently, HS2 ends up throwing read timeout exception when updating HMS.
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/stats/ColStatsProcessor.java#L180
>  {noformat}
> Caused by: java.net.SocketTimeoutException: Read timed out
> at java.net.SocketInputStream.socketRead0(Native Method) 
> ~[?:1.8.0_252]
> at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) 
> ~[?:1.8.0_252]
> at java.net.SocketInputStream.read(SocketInputStream.java:171) 
> ~[?:1.8.0_252]
> at java.net.SocketInputStream.read(SocketInputStream.java:141) 
> ~[?:1.8.0_252]
> at java.io.BufferedInputStream.fill(BufferedInputStream.java:246) 
> ~[?:1.8.0_252]
> at java.io.BufferedInputStream.read1(BufferedInputStream.java:286) 
> ~[?:1.8.0_252]
> at java.io.BufferedInputStream.read(BufferedInputStream.java:345) 
> ~[?:1.8.0_252]
> at 
> org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:127)
>  ~[hive-exec-3.1]
> at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86) 
> ~[hive-exec-3.1]
> at 
> org.apache.thrift.transport.TSaslTransport.readLength(TSaslTransport.java:374)
>  ~[hive-exec-3.1]
> at 
> org.apache.thrift.transport.TSaslTransport.readFrame(TSaslTransport.java:451) 
> ~[hive-exec-3.1]
> at 
> org.apache.thrift.transport.TSaslTransport.read(TSaslTransport.java:433) 
> ~[hive-exec-3.1]
> at 
> org.apache.thrift.transport.TSaslClientTransport.read(TSaslClientTransport.java:37)
>  ~[hive-exec-3.1]
> at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86) 
> ~[hive-exec-3.1]
> at 
> org.apache.hadoop.hive.metastore.security.TFilterTransport.readAll(TFilterTransport.java:62)
>  ~[hive-exec-3.1]
> at 
> org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:429) 
> ~[hive-exec-3.1]
> at 
> org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:318) 
> ~[hive-exec-3.1]
> at 
> org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:219)
>  ~[hive-exec-3.1]
> at 
> org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:77) 
> ~[hive-exec-3.1]
> at 
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_set_aggr_stats_for(ThriftHiveMetastore.java:4561)
>  ~[hive-exec-3.1]
> at 
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.set_aggr_stats_for(ThriftHiveMetastore.java:4548)
>  ~[hive-exec-3.1]
> at 
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.setPartitionColumnStatistics(HiveMetaStoreClient.java:2496)
>  ~[hive-exec-3.1]
> at 
> org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.setPartitionColumnStatistics(SessionHiveMetaStoreClient.java:515)
>  ~[hive-exec-3.1]
> at sun.reflect.GeneratedMethodAccessor194.invoke(Unknown Source) 
> ~[?:?]
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>  ~[?:1.8.0_252]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_252]
> at 
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:212)
>  ~[hive-exec-3.1]
> at com.sun.proxy.$Proxy60.setPartitionColumnStatistics(Unknown 
> Source) ~[?:?]
> at sun.reflect.GeneratedMethodAccessor194.invoke(Unknown Source) 
> ~[?:?]
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>  ~[?:1.8.0_252]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_252]
> at 
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient$SynchronizedHandler.invoke(HiveMetaStoreClient.java:3431)
>  ~[hive-exec-3.1]
> at com.sun.proxy.$Proxy60.setPartitionColumnStatistics(Unknown 
> Source) ~[?:?]
> at 
> org.apache.hadoop.hive.ql.metadata.Hive.setPartitionColumnStatistics(Hive.java:5213)
>  ~[hive-exec-3.1]
> at 
> org.apache.hadoop.hive.ql.stats.ColStatsProcessor.persistColumnStats(ColStatsProcessor.java:192)
>  ~[hive-exec-3.1]
> at 
>

[jira] [Work logged] (HIVE-24834) Cannot add comment for kafka table

[jira] [Work logged] (HIVE-24752) Returned operation's drilldown link may be broken

[jira] [Work logged] (HIVE-24752) Returned operation's drilldown link may be broken

[jira] [Work logged] (HIVE-24752) Returned operation's drilldown link may be broken

[jira] [Work logged] (HIVE-24792) Potential thread leak in Operation

[jira] [Work logged] (HIVE-24752) Returned operation's drilldown link may be broken

[jira] [Work logged] (HIVE-24752) Returned operation's drilldown link may be broken

[jira] [Work logged] (HIVE-24792) Potential thread leak in Operation

[jira] [Commented] (HIVE-24750) Create a single copy task for external tables within default DB location

[jira] [Assigned] (HIVE-24790) Batch column stats updates to HMS

[jira] [Assigned] (HIVE-24790) Batch column stats updates to HMS

11 matches

Site Navigation

Mail list logo

Footer information