ayushtkn commented on code in PR #4597:
URL: https://github.com/apache/hadoop/pull/4597#discussion_r928012699
##########
hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RBFConfigKeys.java:
##########
@@ -354,6 +354,10 @@ public class RBFConfigKeys extends
CommonConfigurationKeysPublic {
NoRouterRpcFairnessPolicyController.class;
public static final String DFS_ROUTER_FAIR_HANDLER_COUNT_KEY_PREFIX =
FEDERATION_ROUTER_FAIRNESS_PREFIX + "handler.count.";
+ public static final String DFS_ROUTER_FAIRNESS_ACQUIRE_TIMEOUT =
+ FEDERATION_ROUTER_FAIRNESS_PREFIX + "acquire.timeout";
+ public static final long DFS_ROUTER_FAIRNESS_ACQUIRE_TIMEOUT_MS_DEFAULT =
Review Comment:
The variable name should be similar to the original config name +
``_DEFAULT``, remove the ``_MS_`` in middle.
Should be ``DFS_ROUTER_FAIRNESS_ACQUIRE_TIMEOUT_DEFAULT``
##########
hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/fairness/AbstractRouterRpcFairnessPolicyController.java:
##########
@@ -42,15 +45,22 @@
/** Hash table to hold semaphore for each configured name service. */
private Map<String, Semaphore> permits;
+ private long acquireTimeoutMs =
DFS_ROUTER_FAIRNESS_ACQUIRE_TIMEOUT_MS_DEFAULT;
+
public void init(Configuration conf) {
this.permits = new HashMap<>();
+ long timeoutMs = conf.getTimeDuration(DFS_ROUTER_FAIRNESS_ACQUIRE_TIMEOUT,
+ DFS_ROUTER_FAIRNESS_ACQUIRE_TIMEOUT_MS_DEFAULT, TimeUnit.MILLISECONDS);
+ if (timeoutMs >= 0) {
+ acquireTimeoutMs = timeoutMs;
Review Comment:
if there is an invalid entry configured and we are moving to using the
default value. We should atleast have a warn log. Kind of ``Invalid value -1
configured for dfs.... should be greater than or equal to 0, Using default
value of : 1s instead``
something like this
##########
hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/fairness/TestRouterRpcFairnessPolicyController.java:
##########
@@ -83,6 +87,29 @@ public void testHandlerAllocationPreconfigured() {
assertFalse(routerRpcFairnessPolicyController.acquirePermit(CONCURRENT_NS));
}
+ @Test
+ public void testAcquireTimeout() {
+ Configuration conf = createConf(40);
+ conf.setInt(DFS_ROUTER_FAIR_HANDLER_COUNT_KEY_PREFIX + "ns1", 30);
+ conf.setTimeDuration(DFS_ROUTER_FAIRNESS_ACQUIRE_TIMEOUT, 100,
TimeUnit.MILLISECONDS);
+ RouterRpcFairnessPolicyController routerRpcFairnessPolicyController =
+ FederationUtil.newFairnessPolicyController(conf);
+
+ // ns1 should have 30 permits allocated
+ for (int i = 0; i < 30; i++) {
+ assertTrue(routerRpcFairnessPolicyController.acquirePermit("ns1"));
+ }
+ long acquireBeginTimeMs = Time.monotonicNow();
+ assertFalse(routerRpcFairnessPolicyController.acquirePermit("ns1"));
+ long acquireEndTimeMs = Time.monotonicNow();
+
+ long acquireTimeMs = acquireEndTimeMs - acquireBeginTimeMs;
+
+ // There are some other operations, so acquireTimeMs >= 100ms.
+ assertTrue(acquireTimeMs >= 100);
+ assertTrue(acquireTimeMs < 100 + 50);
Review Comment:
Is this 50 added as safe margin, kind of that other operations won't take
more than 50ms? In that case I doubt this test is gonna go flaky in future,
Double check to confirm it isn't gonna cross the safe limit of 50 ever.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
