[
https://issues.apache.org/jira/browse/HDFS-17751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17932845#comment-17932845
]
ASF GitHub Bot commented on HDFS-17751:
---------------------------------------
slfan1989 commented on PR #7470:
URL: https://github.com/apache/hadoop/pull/7470#issuecomment-2702889071
@hfutatzhanghb Thank you very much for being willing to add unit tests, as
it will significantly improve the robustness of the system. However, we need to
have a deeper discussion regarding these changes to the unit tests. Recently,
I’ve been upgrading to JUnit 5, so I’m particularly focused on testing-related
topics.
I’ve initially reviewed the relevant tests and found that the main
difference between the synchronous and asynchronous routers lies in the
configuration within the `Configuration`: The
`RBFConfigKeys.DFS_ROUTER_ASYNC_RPC_ENABLE_KEY` parameter determines the type
of router. If the parameter is set to `true`, it enables the asynchronous
router; if set to `false`, the synchronous router is used.
My initial suggestion is to use JUnit 5’s Parameterized Tests so that we can
validate both configurations in the same test class.
However, after discussing with hfutatzhanghb, I learned an important point:
HDFS Router tests typically require initializing a minicluster, and
miniclusters are resource-intensive components. If we use Parameterized Tests,
it would mean starting a minicluster for each test case, which would incur a
significant overhead.
I have considered two alternative approaches for the current situation:
**Option 1**: When using Parameterized Tests, we can use the `@MethodSource`
annotation during initialization. In a custom method, we can initialize two
different router miniclusters, then pass them into the test method for further
testing.
After the test completes, we would need to destroy the corresponding
MiniCluster to release the resources.
Here is an example code:
```
@ParameterizedTest
@MethodSource("getRouterMiniCluster")
void testWithExplicitLocalMethodSource(String argument) {
assertNotNull(argument);
}
Stream<String> getRouterMiniCluster() {
return Stream.of("MiniCluster1", "MiniCluster2");
}
```
**Option 2**: Referencing `TestJettyHelper`, we can define a utility class
in JUnit 5 to manage the MiniCluster. This utility class would extend JUnit 5's
`BeforeEachCallback` and `AfterEachCallback`.
Before each test method is executed, we can check whether the synchronous or
asynchronous cluster has been created. If not, we create the cluster and store
the information in `ThreadLocal`, which helps avoid creating the cluster
multiple times.
Here is a part of the code:
```
private static final ThreadLocal<TestJettyHelper> TEST_JETTY_TL =
new InheritableThreadLocal<TestJettyHelper>();
private Server createJettyServer() {
try {
InetAddress localhost = InetAddress.getByName("localhost");
String host = "localhost";
ServerSocket ss = new ServerSocket(0, 50, localhost);
int port = ss.getLocalPort();
ss.close();
Server server = new Server();
ServerConnector conn = new ServerConnector(server);
HttpConfiguration http_config = new HttpConfiguration();
http_config.setRequestHeaderSize(JettyUtils.HEADER_SIZE);
http_config.setResponseHeaderSize(JettyUtils.HEADER_SIZE);
http_config.setSecureScheme("https");
http_config.addCustomizer(new SecureRequestCustomizer());
ConnectionFactory connFactory = new HttpConnectionFactory(http_config);
conn.addConnectionFactory(connFactory);
conn.setHost(host);
conn.setPort(port);
......
} catch (Exception ex) {
throw new RuntimeException("Could not start embedded servlet
container, " + ex.getMessage(), ex);
}
}
```
> [ARR] Add unit tests using asynchronous router rpc for all in
> org.apache.hadoop.hdfs.server.federation.router
> -------------------------------------------------------------------------------------------------------------
>
> Key: HDFS-17751
> URL: https://issues.apache.org/jira/browse/HDFS-17751
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: rbf, test
> Reporter: farmmamba
> Assignee: farmmamba
> Priority: Major
> Labels: pull-request-available
>
> This Jira aims to add unit tests using asynchronous router rpc for those
> existed unit tests which in org.apache.hadoop.hdfs.server.federation.router
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
