leekeiabstraction commented on code in PR #2569: URL: https://github.com/apache/fluss/pull/2569#discussion_r2782381448
########## website/docs/quickstart/lakehouse.md: ########## @@ -224,25 +264,62 @@ You can add more jars to this `lib` directory based on your requirements: ```yaml services: + #begin RustFS (S3-compatible storage) + rustfs: + image: rustfs/rustfs:latest + ports: + - "9000:9000" + - "9001:9001" + environment: + - RUSTFS_ACCESS_KEY=rustfsadmin + - RUSTFS_SECRET_KEY=rustfsadmin + - RUSTFS_CONSOLE_ENABLE=true + volumes: + - rustfs-data:/data + command: /data + rustfs-init: + image: minio/mc + depends_on: + - rustfs + entrypoint: > + /bin/sh -c " + until mc alias set rustfs http://rustfs:9000 rustfsadmin rustfsadmin; do + echo 'Waiting for RustFS...'; + sleep 1; + done; + mc mb --ignore-existing rustfs/fluss; + " + #end coordinator-server: image: apache/fluss:$FLUSS_DOCKER_VERSION$ command: coordinatorServer depends_on: - zookeeper + - rustfs-init environment: - | FLUSS_PROPERTIES= zookeeper.address: zookeeper:2181 bind.listeners: FLUSS://coordinator-server:9123 - remote.data.dir: /tmp/fluss/remote-data + remote.data.dir: s3://fluss/remote-data + s3.endpoint: http://rustfs:9000 + s3.access-key: rustfsadmin + s3.secret-key: rustfsadmin + s3.path.style.access: true datalake.format: iceberg datalake.iceberg.type: hadoop - datalake.iceberg.warehouse: /tmp/iceberg + datalake.iceberg.warehouse: s3a://fluss/iceberg + datalake.iceberg.iceberg.hadoop.fs.s3a.endpoint: http://rustfs:9000 + datalake.iceberg.iceberg.hadoop.fs.s3a.access.key: rustfsadmin + datalake.iceberg.iceberg.hadoop.fs.s3a.secret.key: rustfsadmin + datalake.iceberg.iceberg.hadoop.fs.s3a.path.style.access: true volumes: - - shared-tmpfs:/tmp/iceberg - - shared-tmpfs:/tmp/fluss - ./lib/fluss-lake-iceberg-$FLUSS_DOCKER_VERSION$.jar:/opt/fluss/plugins/iceberg/fluss-lake-iceberg-$FLUSS_DOCKER_VERSION$.jar - - ./lib/hadoop-apache-3.3.5-2.jar:/opt/fluss/plugins/iceberg/hadoop-apache-3.3.5-2.jar + - ./lib/hadoop-client-api-3.3.5.jar:/opt/fluss/plugins/iceberg/hadoop-client-api-3.3.5.jar Review Comment: Unfortunately not, `hadoop-apache-3.3.5-2` is a trino shaded jar, some of its dependencies have their namespaces moved and it does not include `hadoop-aws`. When I tried to use `hadoop-aws` with `hadoop-apache-3.3.5-2`, binary incompatibility occurred because `hadoop-aws` was trying to load classes which have been shaded. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
