[jira] [Work logged] (HIVE-27172) Add the HMS client connection timeout config
[
https://issues.apache.org/jira/browse/HIVE-27172?focusedWorklogId=860213=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-860213
]
ASF GitHub Bot logged work on HIVE-27172:
-
Author: ASF GitHub Bot
Created on: 03/May/23 06:45
Start Date: 03/May/23 06:45
Worklog Time Spent: 10m
Work Description: deniskuzZ merged PR #4150:
URL: https://github.com/apache/hive/pull/4150
Issue Time Tracking
---
Worklog Id: (was: 860213)
Time Spent: 3h 10m (was: 3h)
> Add the HMS client connection timeout config
>
>
> Key: HIVE-27172
> URL: https://issues.apache.org/jira/browse/HIVE-27172
> Project: Hive
> Issue Type: Task
> Components: Hive
>Reporter: Wechar
>Assignee: Wechar
>Priority: Major
> Labels: pull-request-available
> Time Spent: 3h 10m
> Remaining Estimate: 0h
>
> Currently {{HiveMetastoreClient}} use {{CLIENT_SOCKET_TIMEOUT}} as both
> socket timeout and connection timeout, it's not convenient for users to set a
> smaller connection timeout.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27172) Add the HMS client connection timeout config
[
https://issues.apache.org/jira/browse/HIVE-27172?focusedWorklogId=859775=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-859775
]
ASF GitHub Bot logged work on HIVE-27172:
-
Author: ASF GitHub Bot
Created on: 29/Apr/23 12:35
Start Date: 29/Apr/23 12:35
Worklog Time Spent: 10m
Work Description: sonarcloud[bot] commented on PR #4150:
URL: https://github.com/apache/hive/pull/4150#issuecomment-1528778411
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive=4150)
[](https://sonarcloud.io/project/issues?id=apache_hive=4150=false=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive=4150=false=BUG)
[0
Bugs](https://sonarcloud.io/project/issues?id=apache_hive=4150=false=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive=4150=false=VULNERABILITY)
[](https://sonarcloud.io/project/issues?id=apache_hive=4150=false=VULNERABILITY)
[0
Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive=4150=false=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4150=false=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4150=false=SECURITY_HOTSPOT)
[0 Security
Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4150=false=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive=4150=false=CODE_SMELL)
[](https://sonarcloud.io/project/issues?id=apache_hive=4150=false=CODE_SMELL)
[1 Code
Smell](https://sonarcloud.io/project/issues?id=apache_hive=4150=false=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive=4150=coverage=list)
No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive=4150=duplicated_lines_density=list)
No Duplication information
Issue Time Tracking
---
Worklog Id: (was: 859775)
Time Spent: 3h (was: 2h 50m)
> Add the HMS client connection timeout config
>
>
> Key: HIVE-27172
> URL: https://issues.apache.org/jira/browse/HIVE-27172
> Project: Hive
> Issue Type: Task
> Components: Hive
>Reporter: Wechar
>Assignee: Wechar
>Priority: Major
> Labels: pull-request-available
> Time Spent: 3h
> Remaining Estimate: 0h
>
> Currently {{HiveMetastoreClient}} use {{CLIENT_SOCKET_TIMEOUT}} as both
> socket timeout and connection timeout, it's not convenient for users to set a
> smaller connection timeout.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27172) Add the HMS client connection timeout config
[
https://issues.apache.org/jira/browse/HIVE-27172?focusedWorklogId=859771=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-859771
]
ASF GitHub Bot logged work on HIVE-27172:
-
Author: ASF GitHub Bot
Created on: 29/Apr/23 11:27
Start Date: 29/Apr/23 11:27
Worklog Time Spent: 10m
Work Description: wecharyu commented on code in PR #4150:
URL: https://github.com/apache/hive/pull/4150#discussion_r1181066086
##
standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/TestHiveMetaStoreTimeout.java:
##
@@ -43,9 +46,10 @@ public class TestHiveMetaStoreTimeout {
protected static HiveMetaStoreClient client;
protected static Configuration conf;
protected static Warehouse warehouse;
+ protected static int port;
- @BeforeClass
- public static void setUp() throws Exception {
+ @Before
+ public void setUp() throws Exception {
HMSHandler.testTimeoutEnabled = true;
Review Comment:
Addressed! Have separated the start of MetaStore Server into a `BeforeClass`
method.
Issue Time Tracking
---
Worklog Id: (was: 859771)
Time Spent: 2h 50m (was: 2h 40m)
> Add the HMS client connection timeout config
>
>
> Key: HIVE-27172
> URL: https://issues.apache.org/jira/browse/HIVE-27172
> Project: Hive
> Issue Type: Task
> Components: Hive
>Reporter: Wechar
>Assignee: Wechar
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2h 50m
> Remaining Estimate: 0h
>
> Currently {{HiveMetastoreClient}} use {{CLIENT_SOCKET_TIMEOUT}} as both
> socket timeout and connection timeout, it's not convenient for users to set a
> smaller connection timeout.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27172) Add the HMS client connection timeout config
[
https://issues.apache.org/jira/browse/HIVE-27172?focusedWorklogId=859770=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-859770
]
ASF GitHub Bot logged work on HIVE-27172:
-
Author: ASF GitHub Bot
Created on: 29/Apr/23 11:21
Start Date: 29/Apr/23 11:21
Worklog Time Spent: 10m
Work Description: wecharyu commented on code in PR #4150:
URL: https://github.com/apache/hive/pull/4150#discussion_r1181062821
##
standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/TestHiveMetaStoreTimeout.java:
##
@@ -117,7 +123,7 @@ public void testResetTimeout() throws Exception {
.build(conf);
try {
client.createDatabase(db);
-} catch (MetaException e) {
+} catch (Exception e) {
Review Comment:
Because we have changed to use the remote MetaStore server in this test
class, so the timeout exception was wrapped in `TTransportException` rather
than `MetaException`.
Issue Time Tracking
---
Worklog Id: (was: 859770)
Time Spent: 2h 40m (was: 2.5h)
> Add the HMS client connection timeout config
>
>
> Key: HIVE-27172
> URL: https://issues.apache.org/jira/browse/HIVE-27172
> Project: Hive
> Issue Type: Task
> Components: Hive
>Reporter: Wechar
>Assignee: Wechar
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2h 40m
> Remaining Estimate: 0h
>
> Currently {{HiveMetastoreClient}} use {{CLIENT_SOCKET_TIMEOUT}} as both
> socket timeout and connection timeout, it's not convenient for users to set a
> smaller connection timeout.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27172) Add the HMS client connection timeout config
[
https://issues.apache.org/jira/browse/HIVE-27172?focusedWorklogId=859425=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-859425
]
ASF GitHub Bot logged work on HIVE-27172:
-
Author: ASF GitHub Bot
Created on: 27/Apr/23 15:13
Start Date: 27/Apr/23 15:13
Worklog Time Spent: 10m
Work Description: deniskuzZ commented on code in PR #4150:
URL: https://github.com/apache/hive/pull/4150#discussion_r1179317009
##
standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/TestHiveMetaStoreTimeout.java:
##
@@ -117,7 +123,7 @@ public void testResetTimeout() throws Exception {
.build(conf);
try {
client.createDatabase(db);
-} catch (MetaException e) {
+} catch (Exception e) {
Review Comment:
why is this change?
Issue Time Tracking
---
Worklog Id: (was: 859425)
Time Spent: 2.5h (was: 2h 20m)
> Add the HMS client connection timeout config
>
>
> Key: HIVE-27172
> URL: https://issues.apache.org/jira/browse/HIVE-27172
> Project: Hive
> Issue Type: Task
> Components: Hive
>Reporter: Wechar
>Assignee: Wechar
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> Currently {{HiveMetastoreClient}} use {{CLIENT_SOCKET_TIMEOUT}} as both
> socket timeout and connection timeout, it's not convenient for users to set a
> smaller connection timeout.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27172) Add the HMS client connection timeout config
[
https://issues.apache.org/jira/browse/HIVE-27172?focusedWorklogId=859424=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-859424
]
ASF GitHub Bot logged work on HIVE-27172:
-
Author: ASF GitHub Bot
Created on: 27/Apr/23 15:13
Start Date: 27/Apr/23 15:13
Worklog Time Spent: 10m
Work Description: deniskuzZ commented on code in PR #4150:
URL: https://github.com/apache/hive/pull/4150#discussion_r1179316177
##
standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/TestHiveMetaStoreTimeout.java:
##
@@ -43,9 +46,10 @@ public class TestHiveMetaStoreTimeout {
protected static HiveMetaStoreClient client;
protected static Configuration conf;
protected static Warehouse warehouse;
+ protected static int port;
- @BeforeClass
- public static void setUp() throws Exception {
+ @Before
+ public void setUp() throws Exception {
HMSHandler.testTimeoutEnabled = true;
Review Comment:
do we need to repeat the whole setup for every test or it's possible to
extract just the HMSHandler part?
Issue Time Tracking
---
Worklog Id: (was: 859424)
Time Spent: 2h 20m (was: 2h 10m)
> Add the HMS client connection timeout config
>
>
> Key: HIVE-27172
> URL: https://issues.apache.org/jira/browse/HIVE-27172
> Project: Hive
> Issue Type: Task
> Components: Hive
>Reporter: Wechar
>Assignee: Wechar
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
> Currently {{HiveMetastoreClient}} use {{CLIENT_SOCKET_TIMEOUT}} as both
> socket timeout and connection timeout, it's not convenient for users to set a
> smaller connection timeout.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27172) Add the HMS client connection timeout config
[
https://issues.apache.org/jira/browse/HIVE-27172?focusedWorklogId=859035=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-859035
]
ASF GitHub Bot logged work on HIVE-27172:
-
Author: ASF GitHub Bot
Created on: 26/Apr/23 00:08
Start Date: 26/Apr/23 00:08
Worklog Time Spent: 10m
Work Description: sonarcloud[bot] commented on PR #4150:
URL: https://github.com/apache/hive/pull/4150#issuecomment-1522566937
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive=4150)
[](https://sonarcloud.io/project/issues?id=apache_hive=4150=false=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive=4150=false=BUG)
[0
Bugs](https://sonarcloud.io/project/issues?id=apache_hive=4150=false=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive=4150=false=VULNERABILITY)
[](https://sonarcloud.io/project/issues?id=apache_hive=4150=false=VULNERABILITY)
[0
Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive=4150=false=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4150=false=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4150=false=SECURITY_HOTSPOT)
[0 Security
Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4150=false=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive=4150=false=CODE_SMELL)
[](https://sonarcloud.io/project/issues?id=apache_hive=4150=false=CODE_SMELL)
[1 Code
Smell](https://sonarcloud.io/project/issues?id=apache_hive=4150=false=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive=4150=coverage=list)
No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive=4150=duplicated_lines_density=list)
No Duplication information
Issue Time Tracking
---
Worklog Id: (was: 859035)
Time Spent: 2h 10m (was: 2h)
> Add the HMS client connection timeout config
>
>
> Key: HIVE-27172
> URL: https://issues.apache.org/jira/browse/HIVE-27172
> Project: Hive
> Issue Type: Task
> Components: Hive
>Reporter: Wechar
>Assignee: Wechar
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> Currently {{HiveMetastoreClient}} use {{CLIENT_SOCKET_TIMEOUT}} as both
> socket timeout and connection timeout, it's not convenient for users to set a
> smaller connection timeout.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27172) Add the HMS client connection timeout config
[
https://issues.apache.org/jira/browse/HIVE-27172?focusedWorklogId=858987=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-858987
]
ASF GitHub Bot logged work on HIVE-27172:
-
Author: ASF GitHub Bot
Created on: 25/Apr/23 17:08
Start Date: 25/Apr/23 17:08
Worklog Time Spent: 10m
Work Description: wecharyu commented on code in PR #4150:
URL: https://github.com/apache/hive/pull/4150#discussion_r1176806249
##
standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/conf/MetastoreConf.java:
##
@@ -402,6 +402,8 @@ public enum ConfVars {
"has an infinite lifetime."),
CLIENT_SOCKET_TIMEOUT("metastore.client.socket.timeout",
"hive.metastore.client.socket.timeout", 600,
TimeUnit.SECONDS, "MetaStore Client socket timeout in seconds"),
+CLIENT_CONNECTION_TIMEOUT("metastore.client.connection.timeout",
"hive.metastore.client.connection.timeout", 10,
Review Comment:
Yes, I have changed the default connectionTimeout to be the same as
socketTimeout. But IMHO it's not a good practice because if we set
socketTimeout and not set connectionTimeout, the default long time will be
gained for connection, I think we should change the behavior by default where
socketTimeout = connectionTimeout.
Issue Time Tracking
---
Worklog Id: (was: 858987)
Time Spent: 2h (was: 1h 50m)
> Add the HMS client connection timeout config
>
>
> Key: HIVE-27172
> URL: https://issues.apache.org/jira/browse/HIVE-27172
> Project: Hive
> Issue Type: Task
> Components: Hive
>Reporter: Wechar
>Assignee: Wechar
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2h
> Remaining Estimate: 0h
>
> Currently {{HiveMetastoreClient}} use {{CLIENT_SOCKET_TIMEOUT}} as both
> socket timeout and connection timeout, it's not convenient for users to set a
> smaller connection timeout.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27172) Add the HMS client connection timeout config
[
https://issues.apache.org/jira/browse/HIVE-27172?focusedWorklogId=858793=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-858793
]
ASF GitHub Bot logged work on HIVE-27172:
-
Author: ASF GitHub Bot
Created on: 24/Apr/23 21:10
Start Date: 24/Apr/23 21:10
Worklog Time Spent: 10m
Work Description: deniskuzZ commented on code in PR #4150:
URL: https://github.com/apache/hive/pull/4150#discussion_r1175784708
##
standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/conf/MetastoreConf.java:
##
@@ -402,6 +402,8 @@ public enum ConfVars {
"has an infinite lifetime."),
CLIENT_SOCKET_TIMEOUT("metastore.client.socket.timeout",
"hive.metastore.client.socket.timeout", 600,
TimeUnit.SECONDS, "MetaStore Client socket timeout in seconds"),
+CLIENT_CONNECTION_TIMEOUT("metastore.client.connection.timeout",
"hive.metastore.client.connection.timeout", 10,
Review Comment:
@wecharyu, won't be socketTimeout = connectTimeout if not explicitly set?
public TSocket(TConfiguration config, String host, int port, int timeout) {
this(config, host, port, timeout, timeout);
}
public TSocket(TConfiguration config, String host, int port, int
socketTimeout, int connectTimeout) {}
Issue Time Tracking
---
Worklog Id: (was: 858793)
Time Spent: 1h 50m (was: 1h 40m)
> Add the HMS client connection timeout config
>
>
> Key: HIVE-27172
> URL: https://issues.apache.org/jira/browse/HIVE-27172
> Project: Hive
> Issue Type: Task
> Components: Hive
>Reporter: Wechar
>Assignee: Wechar
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> Currently {{HiveMetastoreClient}} use {{CLIENT_SOCKET_TIMEOUT}} as both
> socket timeout and connection timeout, it's not convenient for users to set a
> smaller connection timeout.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27172) Add the HMS client connection timeout config
[
https://issues.apache.org/jira/browse/HIVE-27172?focusedWorklogId=858792=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-858792
]
ASF GitHub Bot logged work on HIVE-27172:
-
Author: ASF GitHub Bot
Created on: 24/Apr/23 21:09
Start Date: 24/Apr/23 21:09
Worklog Time Spent: 10m
Work Description: deniskuzZ commented on code in PR #4150:
URL: https://github.com/apache/hive/pull/4150#discussion_r1175784708
##
standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/conf/MetastoreConf.java:
##
@@ -402,6 +402,8 @@ public enum ConfVars {
"has an infinite lifetime."),
CLIENT_SOCKET_TIMEOUT("metastore.client.socket.timeout",
"hive.metastore.client.socket.timeout", 600,
TimeUnit.SECONDS, "MetaStore Client socket timeout in seconds"),
+CLIENT_CONNECTION_TIMEOUT("metastore.client.connection.timeout",
"hive.metastore.client.connection.timeout", 10,
Review Comment:
@wecharyu, won't be socketTimeout = connectTimeout if not explicitly set?
{code}
public TSocket(TConfiguration config, String host, int port, int timeout) {
this(config, host, port, timeout, timeout);
}
public TSocket(TConfiguration config, String host, int port, int
socketTimeout, int connectTimeout) {}
{code}
Issue Time Tracking
---
Worklog Id: (was: 858792)
Time Spent: 1h 40m (was: 1.5h)
> Add the HMS client connection timeout config
>
>
> Key: HIVE-27172
> URL: https://issues.apache.org/jira/browse/HIVE-27172
> Project: Hive
> Issue Type: Task
> Components: Hive
>Reporter: Wechar
>Assignee: Wechar
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> Currently {{HiveMetastoreClient}} use {{CLIENT_SOCKET_TIMEOUT}} as both
> socket timeout and connection timeout, it's not convenient for users to set a
> smaller connection timeout.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27172) Add the HMS client connection timeout config
[
https://issues.apache.org/jira/browse/HIVE-27172?focusedWorklogId=858789=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-858789
]
ASF GitHub Bot logged work on HIVE-27172:
-
Author: ASF GitHub Bot
Created on: 24/Apr/23 21:00
Start Date: 24/Apr/23 21:00
Worklog Time Spent: 10m
Work Description: deniskuzZ commented on code in PR #4150:
URL: https://github.com/apache/hive/pull/4150#discussion_r1175777494
##
standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/conf/MetastoreConf.java:
##
@@ -402,6 +402,8 @@ public enum ConfVars {
"has an infinite lifetime."),
CLIENT_SOCKET_TIMEOUT("metastore.client.socket.timeout",
"hive.metastore.client.socket.timeout", 600,
TimeUnit.SECONDS, "MetaStore Client socket timeout in seconds"),
+CLIENT_CONNECTION_TIMEOUT("metastore.client.connection.timeout",
"hive.metastore.client.connection.timeout", 10,
Review Comment:
@wecharyu isn't the default socket timeout 30min?
Issue Time Tracking
---
Worklog Id: (was: 858789)
Time Spent: 1.5h (was: 1h 20m)
> Add the HMS client connection timeout config
>
>
> Key: HIVE-27172
> URL: https://issues.apache.org/jira/browse/HIVE-27172
> Project: Hive
> Issue Type: Task
> Components: Hive
>Reporter: Wechar
>Assignee: Wechar
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> Currently {{HiveMetastoreClient}} use {{CLIENT_SOCKET_TIMEOUT}} as both
> socket timeout and connection timeout, it's not convenient for users to set a
> smaller connection timeout.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27172) Add the HMS client connection timeout config
[
https://issues.apache.org/jira/browse/HIVE-27172?focusedWorklogId=855596=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-855596
]
ASF GitHub Bot logged work on HIVE-27172:
-
Author: ASF GitHub Bot
Created on: 08/Apr/23 12:40
Start Date: 08/Apr/23 12:40
Worklog Time Spent: 10m
Work Description: wecharyu commented on PR #4150:
URL: https://github.com/apache/hive/pull/4150#issuecomment-1500883854
@zabetak @dengzhhu653: Could you also take a look?
Issue Time Tracking
---
Worklog Id: (was: 855596)
Time Spent: 1h 20m (was: 1h 10m)
> Add the HMS client connection timeout config
>
>
> Key: HIVE-27172
> URL: https://issues.apache.org/jira/browse/HIVE-27172
> Project: Hive
> Issue Type: Task
> Components: Hive
>Reporter: Wechar
>Assignee: Wechar
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> Currently {{HiveMetastoreClient}} use {{CLIENT_SOCKET_TIMEOUT}} as both
> socket timeout and connection timeout, it's not convenient for users to set a
> smaller connection timeout.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27172) Add the HMS client connection timeout config
[
https://issues.apache.org/jira/browse/HIVE-27172?focusedWorklogId=853592=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-853592
]
ASF GitHub Bot logged work on HIVE-27172:
-
Author: ASF GitHub Bot
Created on: 29/Mar/23 07:33
Start Date: 29/Mar/23 07:33
Worklog Time Spent: 10m
Work Description: wecharyu commented on PR #4150:
URL: https://github.com/apache/hive/pull/4150#issuecomment-1488086581
@ayushtkn @deniskuzZ @kasakrisz: Could you help review this PR?
Issue Time Tracking
---
Worklog Id: (was: 853592)
Time Spent: 1h 10m (was: 1h)
> Add the HMS client connection timeout config
>
>
> Key: HIVE-27172
> URL: https://issues.apache.org/jira/browse/HIVE-27172
> Project: Hive
> Issue Type: Task
> Components: Hive
>Reporter: Wechar
>Assignee: Wechar
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> Currently {{HiveMetastoreClient}} use {{CLIENT_SOCKET_TIMEOUT}} as both
> socket timeout and connection timeout, it's not convenient for users to set a
> smaller connection timeout.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27172) Add the HMS client connection timeout config
[
https://issues.apache.org/jira/browse/HIVE-27172?focusedWorklogId=852946=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-852946
]
ASF GitHub Bot logged work on HIVE-27172:
-
Author: ASF GitHub Bot
Created on: 25/Mar/23 04:35
Start Date: 25/Mar/23 04:35
Worklog Time Spent: 10m
Work Description: wecharyu commented on code in PR #4150:
URL: https://github.com/apache/hive/pull/4150#discussion_r1148302834
##
standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/conf/MetastoreConf.java:
##
@@ -402,6 +402,8 @@ public enum ConfVars {
"has an infinite lifetime."),
CLIENT_SOCKET_TIMEOUT("metastore.client.socket.timeout",
"hive.metastore.client.socket.timeout", 600,
TimeUnit.SECONDS, "MetaStore Client socket timeout in seconds"),
+CLIENT_CONNECTION_TIMEOUT("metastore.client.connection.timeout",
"hive.metastore.client.connection.timeout", 10,
Review Comment:
Nice catch, if there is any concern I will keep the default connection
timeout value same as socket timeout.
Issue Time Tracking
---
Worklog Id: (was: 852946)
Time Spent: 1h (was: 50m)
> Add the HMS client connection timeout config
>
>
> Key: HIVE-27172
> URL: https://issues.apache.org/jira/browse/HIVE-27172
> Project: Hive
> Issue Type: Task
> Components: Hive
>Reporter: Wechar
>Assignee: Wechar
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h
> Remaining Estimate: 0h
>
> Currently {{HiveMetastoreClient}} use {{CLIENT_SOCKET_TIMEOUT}} as both
> socket timeout and connection timeout, it's not convenient for users to set a
> smaller connection timeout.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27172) Add the HMS client connection timeout config
[
https://issues.apache.org/jira/browse/HIVE-27172?focusedWorklogId=852921=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-852921
]
ASF GitHub Bot logged work on HIVE-27172:
-
Author: ASF GitHub Bot
Created on: 24/Mar/23 20:51
Start Date: 24/Mar/23 20:51
Worklog Time Spent: 10m
Work Description: sonarcloud[bot] commented on PR #4150:
URL: https://github.com/apache/hive/pull/4150#issuecomment-1483391646
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive=4150)
[](https://sonarcloud.io/project/issues?id=apache_hive=4150=false=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive=4150=false=BUG)
[0
Bugs](https://sonarcloud.io/project/issues?id=apache_hive=4150=false=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive=4150=false=VULNERABILITY)
[](https://sonarcloud.io/project/issues?id=apache_hive=4150=false=VULNERABILITY)
[0
Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive=4150=false=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4150=false=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4150=false=SECURITY_HOTSPOT)
[0 Security
Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4150=false=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive=4150=false=CODE_SMELL)
[](https://sonarcloud.io/project/issues?id=apache_hive=4150=false=CODE_SMELL)
[3 Code
Smells](https://sonarcloud.io/project/issues?id=apache_hive=4150=false=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive=4150=coverage=list)
No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive=4150=duplicated_lines_density=list)
No Duplication information
Issue Time Tracking
---
Worklog Id: (was: 852921)
Time Spent: 50m (was: 40m)
> Add the HMS client connection timeout config
>
>
> Key: HIVE-27172
> URL: https://issues.apache.org/jira/browse/HIVE-27172
> Project: Hive
> Issue Type: Task
> Components: Hive
>Reporter: Wechar
>Assignee: Wechar
>Priority: Major
> Labels: pull-request-available
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Currently {{HiveMetastoreClient}} use {{CLIENT_SOCKET_TIMEOUT}} as both
> socket timeout and connection timeout, it's not convenient for users to set a
> smaller connection timeout.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27172) Add the HMS client connection timeout config
[
https://issues.apache.org/jira/browse/HIVE-27172?focusedWorklogId=852901=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-852901
]
ASF GitHub Bot logged work on HIVE-27172:
-
Author: ASF GitHub Bot
Created on: 24/Mar/23 18:39
Start Date: 24/Mar/23 18:39
Worklog Time Spent: 10m
Work Description: pan3793 commented on code in PR #4150:
URL: https://github.com/apache/hive/pull/4150#discussion_r1147940693
##
standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/conf/MetastoreConf.java:
##
@@ -402,6 +402,8 @@ public enum ConfVars {
"has an infinite lifetime."),
CLIENT_SOCKET_TIMEOUT("metastore.client.socket.timeout",
"hive.metastore.client.socket.timeout", 600,
TimeUnit.SECONDS, "MetaStore Client socket timeout in seconds"),
+CLIENT_CONNECTION_TIMEOUT("metastore.client.connection.timeout",
"hive.metastore.client.connection.timeout", 10,
Review Comment:
It makes sense to set connection timeout a small value, because generally,
establish the connection is quick, and small timeout make it quick failover if
one HMS is down, but it silent changes the default behavior, I'm not sure if
such change is acceptable in Hive community
Issue Time Tracking
---
Worklog Id: (was: 852901)
Time Spent: 40m (was: 0.5h)
> Add the HMS client connection timeout config
>
>
> Key: HIVE-27172
> URL: https://issues.apache.org/jira/browse/HIVE-27172
> Project: Hive
> Issue Type: Task
> Components: Hive
>Reporter: Wechar
>Assignee: Wechar
>Priority: Major
> Labels: pull-request-available
> Time Spent: 40m
> Remaining Estimate: 0h
>
> Currently {{HiveMetastoreClient}} use {{CLIENT_SOCKET_TIMEOUT}} as both
> socket timeout and connection timeout, it's not convenient for users to set a
> smaller connection timeout.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27172) Add the HMS client connection timeout config
[
https://issues.apache.org/jira/browse/HIVE-27172?focusedWorklogId=852898=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-852898
]
ASF GitHub Bot logged work on HIVE-27172:
-
Author: ASF GitHub Bot
Created on: 24/Mar/23 18:30
Start Date: 24/Mar/23 18:30
Worklog Time Spent: 10m
Work Description: pan3793 commented on PR #4150:
URL: https://github.com/apache/hive/pull/4150#issuecomment-1483243678
A similar PR https://github.com/apache/hive/pull/3379 for JDBC thrift
client, IMO the connection timeout is quite important, please help reopen the
closed PR if any committer think it's useful.
Issue Time Tracking
---
Worklog Id: (was: 852898)
Time Spent: 0.5h (was: 20m)
> Add the HMS client connection timeout config
>
>
> Key: HIVE-27172
> URL: https://issues.apache.org/jira/browse/HIVE-27172
> Project: Hive
> Issue Type: Task
> Components: Hive
>Reporter: Wechar
>Assignee: Wechar
>Priority: Major
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Currently {{HiveMetastoreClient}} use {{CLIENT_SOCKET_TIMEOUT}} as both
> socket timeout and connection timeout, it's not convenient for users to set a
> smaller connection timeout.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27172) Add the HMS client connection timeout config
[
https://issues.apache.org/jira/browse/HIVE-27172?focusedWorklogId=852896=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-852896
]
ASF GitHub Bot logged work on HIVE-27172:
-
Author: ASF GitHub Bot
Created on: 24/Mar/23 18:24
Start Date: 24/Mar/23 18:24
Worklog Time Spent: 10m
Work Description: wecharyu commented on code in PR #4150:
URL: https://github.com/apache/hive/pull/4150#discussion_r1147927316
##
standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/TestHiveMetaStoreTimeout.java:
##
@@ -43,9 +46,10 @@ public class TestHiveMetaStoreTimeout {
protected static HiveMetaStoreClient client;
protected static Configuration conf;
protected static Warehouse warehouse;
+ protected static int port;
- @BeforeClass
- public static void setUp() throws Exception {
+ @Before
+ public void setUp() throws Exception {
HMSHandler.testTimeoutEnabled = true;
Review Comment:
Nit: This test code hacks `HMSHandler` class, I will refactor this in a new
PR.
Issue Time Tracking
---
Worklog Id: (was: 852896)
Time Spent: 20m (was: 10m)
> Add the HMS client connection timeout config
>
>
> Key: HIVE-27172
> URL: https://issues.apache.org/jira/browse/HIVE-27172
> Project: Hive
> Issue Type: Task
> Components: Hive
>Reporter: Wechar
>Assignee: Wechar
>Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Currently {{HiveMetastoreClient}} use {{CLIENT_SOCKET_TIMEOUT}} as both
> socket timeout and connection timeout, it's not convenient for users to set a
> smaller connection timeout.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27172) Add the HMS client connection timeout config
[
https://issues.apache.org/jira/browse/HIVE-27172?focusedWorklogId=852892=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-852892
]
ASF GitHub Bot logged work on HIVE-27172:
-
Author: ASF GitHub Bot
Created on: 24/Mar/23 18:16
Start Date: 24/Mar/23 18:16
Worklog Time Spent: 10m
Work Description: wecharyu opened a new pull request, #4150:
URL: https://github.com/apache/hive/pull/4150
### What changes were proposed in this pull request?
Currently `HiveMetaStoreClient` use `CLIENT_SOCKET_TIMEOUT` as both socket
timeout and connection timeout, we want to introduce a new HiveMetaStoreClient
config for connection timeout:
```java
CLIENT_CONNECTION_TIMEOUT("metastore.client.connection.timeout",
"hive.metastore.client.connection.timeout", 10,
TimeUnit.SECONDS, "MetaStore Client connection timeout in
seconds"),
```
### Why are the changes needed?
1. achieve a more flexible config for client socket timeout and connection
timeout.
2. we can use a smaller connection timeout to fail-fast reconnect to another
HMS server.
### Does this PR introduce _any_ user-facing change?
Yes, user can set the client connection timeout by this conf.
### How was this patch tested?
Add a unit test.
Issue Time Tracking
---
Worklog Id: (was: 852892)
Remaining Estimate: 0h
Time Spent: 10m
> Add the HMS client connection timeout config
>
>
> Key: HIVE-27172
> URL: https://issues.apache.org/jira/browse/HIVE-27172
> Project: Hive
> Issue Type: Task
> Components: Hive
>Reporter: Wechar
>Assignee: Wechar
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Currently {{HiveMetastoreClient}} use {{CLIENT_SOCKET_TIMEOUT}} as both
> socket timeout and connection timeout, it's not convenient for users to set a
> smaller connection timeout.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
