jordepic commented on code in PR #2441:
URL: https://github.com/apache/iceberg-rust/pull/2441#discussion_r3282495848
##########
dev/docker-compose.yaml:
##########
@@ -147,6 +147,50 @@ services:
timeout: 5s
retries: 5
+ #
=============================================================================
+ # HDFS - single-node NameNode + DataNode for HDFS tests
+ #
=============================================================================
+ # Mirrors apache/opendal's fixtures/hdfs/docker-compose-hdfs-cluster.yml:
+ # same bde2020 images, host networking on both services. Host networking
+ # is required because hdfs-native 0.13.5 connects to the DataNode by IP
+ # from `DatanodeIdProto.ip_addr` (not by hostname). On a docker bridge
+ # the DN would register with an unroutable bridge IP; host networking
+ # lets it bind directly on the host network namespace so the registered
+ # address is host-reachable.
+ #
+ # This works on Linux CI runners. On macOS / Windows Docker Desktop
+ # host networking has known issues (e.g. unresolvable VM hostname), so
+ # the HDFS integration tests are `#[ignore]`d; CI explicitly opts them
+ # in via `cargo nextest --run-ignored=only` (see .github/workflows/ci.yml).
+ hdfs-namenode:
+ image: bde2020/hadoop-namenode:2.0.0-hadoop3.2.1-java8
+ network_mode: "host"
+ environment:
+ CLUSTER_NAME: iceberg-rust-test
+ CORE_CONF_fs_defaultFS: hdfs://localhost:8020
+ CORE_CONF_hadoop_http_staticuser_user: root
+ HDFS_CONF_dfs_permissions_enabled: false
+ HDFS_CONF_dfs_replication: 1
+ healthcheck:
+ test: ["CMD-SHELL", "hdfs dfsadmin -safemode get | grep -q OFF"]
+ interval: 5s
+ timeout: 5s
+ retries: 30
+ start_period: 30s
+
+ hdfs-datanode:
+ image: bde2020/hadoop-datanode:2.0.0-hadoop3.2.1-java8
Review Comment:
Thanks for the feedback @xanderbailey ! Image is upgraded, let's see if
these tests pass :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
