TakuyaInoue-github opened a new pull request, #20509: URL: https://github.com/apache/hudi/pull/20509
> [!NOTE] > This is a draft PR stacked on top of #19892. > > The current diff includes the parent PR. Once #19892 is merged, this branch will be rebased onto the latest `master` so that only the changes for #19894 remain. > > This draft is being opened early to confirm the following implementation decisions: > > - Should the Hive and ZooKeeper changes remain in one PR? > - Is retrying the actual port selection and bind operation up to five times the preferred approach? > - Is the test-only Hive service used to propagate the asynchronous HiveServer2 bind failure instead of calling `System.exit(-1)` acceptable? ### Describe the issue this Pull Request addresses Closes #19894. `HiveTestService` and `ZookeeperTestService` select an available port using `NetworkTestUtils.nextFreePort()` and bind to it later. Another process or a parallel test fork can acquire the port between those operations. Previously, the selected ports were also stored statically, so callers could not observe a new port if service startup needed to be retried. ### Summary and Changelog This change makes the Hive and ZooKeeper test services retry their actual bind operations after port conflicts. - Retry ZooKeeper port selection and binding up to five times when startup fails with a `BindException`. - Retry HiveServer2 and Hive Metastore startup up to five times when the exception cause chain contains a `BindException`. - Select new HiveServer2 and Metastore ports for each retry attempt. - Clean up partially started services and executors before retrying or propagating a startup failure. - Replace static ZooKeeper and Hive port values with instance-level accessors so callers use the port that was successfully bound. - Start the integration-test ZooKeeper service before creating `WriterContext`, propagate its runtime connection settings, and stop the same service instance during cleanup. - Add tests that occupy the initially selected ports and verify that the services retry using different ports. - Add a ZooKeeper provider lifecycle test. `RetryableThriftBinaryCLIService` is a test-only adaptation of the relevant Apache Hive 2.3.10 `ThriftBinaryCLIService` startup behavior. It allows the asynchronous HiveServer2 bind failure to be returned to the test service instead of reaching Hive's `System.exit(-1)` path. Validation performed with JDK 11: - `TestZookeeperTestService`: 1 test passed. - `TestZookeeperServiceProvider`: 1 test passed. - `TestHiveTestService`: 2 tests passed. - The corresponding Maven reactor builds completed successfully. - Checkstyle completed with no violations. - `git diff --check` completed successfully. ### Impact No public API, storage format, configuration, or user-facing behavior changes. The changes are limited to test services and integration-test infrastructure. Tests using the embedded Hive and ZooKeeper services will use the ports actually bound by their service instances instead of JVM-wide static port values. ### Risk Level Low. The changes affect test infrastructure only. The retry behavior is limited to port-binding failures and is capped at five attempts. Non-bind startup failures are cleaned up and propagated without retrying. ### Documentation Update None. There are no new user-facing features or configuration changes. ### Contributor's checklist - [x] Read through [[contributor's guide](https://hudi.apache.org/contribute/how-to-contribute)](https://hudi.apache.org/contribute/how-to-contribute) - [x] Enough context is provided in the sections above - [x] Adequate tests were added if applicable -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
